2024-04-25T12:22:44.563 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T12:22:44.563 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146 --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --verbose -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/orig.config.yaml 2024-04-25T12:22:44.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T12:22:44.693 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673146' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO - MDS_TRIM sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZq9ZGjFaEDy096n+o5IXK+gnVnfHxqzRTfyqL5jdmWMpctx/3t/ZBr6psedrmqaoYwJVf0dhA4uqxbif9q1VE= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+17CMczDPuJ1mO2BaF92RtCZmOljpAgviJyKwmuEL0S81N3k/EC7U6WgWhqcmm8HC4aNMINzLTtgTHA0G6e6I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - thrashosds: null - workunit: clients: all: - fs/misc/trivial_sync.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T12:22:44.693 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T12:22:44.694 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T12:22:44.694 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T12:22:44.696 INFO:teuthology.task.internal:Checking packages... 2024-04-25T12:22:44.726 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T12:22:44.726 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T12:22:44.726 INFO:teuthology.packaging:ref: None 2024-04-25T12:22:44.726 INFO:teuthology.packaging:tag: None 2024-04-25T12:22:44.726 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T12:22:44.726 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T12:22:44.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T12:22:44.870 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T12:22:44.872 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T12:22:44.878 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T12:22:44.878 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T12:22:44.899 INFO:teuthology.task.internal:Saving configuration 2024-04-25T12:22:44.916 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T12:22:44.925 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T12:22:44.951 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:17:35.548651', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZq9ZGjFaEDy096n+o5IXK+gnVnfHxqzRTfyqL5jdmWMpctx/3t/ZBr6psedrmqaoYwJVf0dhA4uqxbif9q1VE='} 2024-04-25T12:22:44.974 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:17:35.550160', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+17CMczDPuJ1mO2BaF92RtCZmOljpAgviJyKwmuEL0S81N3k/EC7U6WgWhqcmm8HC4aNMINzLTtgTHA0G6e6I='} 2024-04-25T12:22:44.974 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T12:22:44.982 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-25T12:22:44.982 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-25T12:22:44.982 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T12:22:45.114 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f782d50df70>, signals=[15]) 2024-04-25T12:22:45.114 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T12:22:45.121 INFO:teuthology.task.internal:Opening connections... 2024-04-25T12:22:45.122 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:22:45.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:22:45.200 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:22:45.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:22:45.271 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T12:22:45.281 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-04-25T12:22:45.299 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-04-25T12:22:45.299 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-25T12:22:45.355 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:22:45.356 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:22:45.357 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-25T12:22:45.382 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-04-25T12:22:45.400 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-04-25T12:22:45.400 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-04-25T12:22:45.458 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T12:22:45.459 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T12:22:45.459 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T12:22:45.459 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-04-25T12:22:45.482 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T12:22:45.493 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T12:22:45.503 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T12:22:45.504 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:22:45.506 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-04-25T12:22:45.523 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T12:22:45.532 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T12:22:45.532 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:22:45.563 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-04-25T12:22:45.676 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T12:22:45.731 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T12:22:45.747 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T12:22:45.747 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T12:22:45.747 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T12:22:45.747 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:22:45.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:22:45.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T12:22:46.045 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T12:22:46.045 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T12:22:46.045 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-25T12:22:46.062 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-437.el9.x86_64 2024-04-25T12:22:46.063 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T12:22:46.063 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T12:22:46.063 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum upgrade -y linux-firmware 2024-04-25T12:22:48.717 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-04-25T12:22:50.735 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T12:22:54.927 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T12:22:57.675 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-25T12:22:58.535 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-04-25T12:23:03.818 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 5.5 kB/s | 1.7 kB 00:00 2024-04-25T12:23:05.793 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T12:23:05.794 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-25T12:23:05.794 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T12:23:05.909 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T12:23:05.910 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T12:23:06.061 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T12:23:06.061 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T12:23:06.064 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi123.front.sepia.ceph.com to complete... 2024-04-25T12:23:23.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25 12:23:23 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T12:23:23.114 INFO:teuthology.orchestra.run.smithi123.stderr:FINISHED --2024-04-25 12:23:23-- 2024-04-25T12:23:23.114 INFO:teuthology.orchestra.run.smithi123.stderr:Total wall clock time: 17s 2024-04-25T12:23:23.114 INFO:teuthology.orchestra.run.smithi123.stderr:Downloaded: 1 files, 341M in 17s (20.5 MB/s) 2024-04-25T12:23:23.116 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T12:23:23.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T12:23:25.319 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying... ######################################## 2024-04-25T12:23:25.468 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing... ######################################## 2024-04-25T12:23:25.582 INFO:teuthology.orchestra.run.smithi123.stdout:Updating / installing... 2024-04-25T12:23:33.544 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T12:24:12.354 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T12:24:12.355 DEBUG:teuthology.orchestra.run.smithi123:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T12:24:12.429 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T12:24:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T12:24:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T12:24:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T12:24:12.433 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T12:24:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T12:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T12:24:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T12:24:12.437 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T12:24:12.438 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T12:24:12.439 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T12:24:12.440 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T12:24:12.441 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T12:24:12.442 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T12:24:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T12:24:12.444 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T12:24:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T12:24:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T12:24:12.447 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T12:24:12.448 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T12:24:12.449 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T12:24:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T12:24:12.451 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T12:24:12.452 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T12:24:12.453 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T12:24:12.454 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T12:24:12.455 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T12:24:12.456 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T12:24:12.457 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T12:24:12.458 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T12:24:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T12:24:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T12:24:12.461 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T12:24:12.462 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T12:24:12.463 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T12:24:12.464 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T12:24:12.465 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T12:24:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T12:24:12.467 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T12:24:12.468 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T12:24:12.469 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T12:24:12.470 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T12:24:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T12:24:12.472 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T12:24:12.473 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T12:24:12.474 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T12:24:12.475 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T12:24:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T12:24:12.477 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T12:24:12.478 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T12:24:12.479 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T12:24:12.480 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T12:24:12.481 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T12:24:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T12:24:12.483 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T12:24:12.484 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T12:24:12.485 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T12:24:12.486 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T12:24:12.487 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T12:24:12.488 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T12:24:12.489 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T12:24:12.490 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T12:24:12.491 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T12:24:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T12:24:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T12:24:12.494 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T12:24:12.495 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T12:24:12.496 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T12:24:12.497 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T12:24:12.498 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T12:24:12.499 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T12:24:12.500 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T12:24:12.501 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T12:24:12.502 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T12:24:12.503 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T12:24:12.504 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T12:24:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T12:24:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T12:24:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T12:24:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T12:24:12.509 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T12:24:12.510 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T12:24:12.511 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T12:24:12.512 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T12:24:12.513 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T12:24:12.514 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T12:24:12.515 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T12:24:12.516 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T12:24:12.517 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T12:24:12.518 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T12:24:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T12:24:12.520 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T12:24:12.521 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T12:24:12.522 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T12:24:12.523 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T12:24:12.524 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T12:24:12.525 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T12:24:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T12:24:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T12:24:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T12:24:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T12:24:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T12:24:12.531 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T12:24:12.532 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T12:24:12.533 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T12:24:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T12:24:12.535 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T12:24:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T12:24:12.537 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T12:24:12.538 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T12:24:12.539 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T12:24:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T12:24:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T12:24:12.542 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T12:24:12.543 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T12:24:12.544 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T12:24:12.545 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T12:24:12.546 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T12:24:12.547 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T12:24:12.548 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T12:24:12.549 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T12:24:12.550 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T12:24:12.551 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T12:24:12.552 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T12:24:12.553 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T12:24:12.554 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T12:24:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T12:24:12.556 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T12:24:12.557 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T12:24:12.558 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T12:24:12.559 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T12:24:12.560 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T12:24:12.561 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T12:24:12.562 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T12:24:12.566 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:12.567 DEBUG:teuthology.orchestra.run.smithi123:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T12:24:12.589 INFO:teuthology.orchestra.run.smithi123.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T12:24:12.589 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T12:24:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:for file in vmlinuz System.map config; do 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: fi 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:done 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T12:24:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:12.591 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T12:24:12.591 INFO:teuthology.orchestra.run.smithi123.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T12:24:12.591 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T12:24:12.591 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:12.591 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T12:24:12.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 68.el9 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-04-25T12:24:12.664 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8264254 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T12:24:12.665 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-04-25T12:24:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:24:12.666 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T12:24:12.667 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T12:24:12.667 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:12.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T12:24:13.214 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-04-25T12:24:14.543 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T12:24:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-04-25T12:24:14.576 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-25T12:24:14.607 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.LWFbtSb3NG 2024-04-25T12:24:14.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LWFbtSb3NG 2024-04-25T12:24:14.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.LWFbtSb3NG 2024-04-25T12:24:14.823 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.LWFbtSb3NG is 6KB 2024-04-25T12:24:14.872 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.LWFbtSb3NG 2024-04-25T12:24:14.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T12:24:14.948 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T12:24:14.949 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T12:24:14.949 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T12:24:14.949 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T12:24:14.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T12:24:15.144 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-04-25T12:24:45.148 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T12:24:45.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:24:45.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:25:03.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-25T12:25:12.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:25:12.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:25:15.586 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-25T12:25:27.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:25:27.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:25:46.018 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-25T12:26:01.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:26:01.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:26:02.938 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-25T12:26:03.568 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:26:03.569 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T12:26:03.569 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-25T12:26:03.606 INFO:teuthology.orchestra.run.smithi123.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T12:26:03.607 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T12:26:03.607 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T12:26:03.607 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T12:26:04.610 DEBUG:teuthology.parallel:result is None 2024-04-25T12:26:04.611 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T12:26:04.620 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T12:26:04.620 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:26:04.627 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T12:26:04.666 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T12:26:04.702 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T12:26:04.709 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T12:26:04.709 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:26:04.713 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T12:26:04.825 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T12:26:04.832 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T12:26:04.832 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T12:26:04.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:26:04.849 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:26:04.856 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T12:26:04.926 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:26:04.959 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:26:05.002 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:26:05.067 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T12:26:05.071 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T12:26:05.080 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T12:26:05.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:26:05.086 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T12:26:05.149 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T12:26:05.158 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T12:26:05.158 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:26:05.162 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T12:26:05.195 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:26:05.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:26:05.384 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:26:05.452 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:26:05.537 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:26:05.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:26:05.651 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:26:05.689 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T12:26:05.752 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:26:05.832 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T12:26:05.877 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:26:05.877 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T12:26:05.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-04-25T12:26:05.948 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-04-25T12:26:06.017 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:26:06.070 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:26:06.331 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T12:26:06.338 INFO:teuthology.task.internal:Starting timer... 2024-04-25T12:26:06.338 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T12:26:06.348 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T12:26:06.356 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T12:26:06.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-04-25T12:26:06.487 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-04-25T12:26:06.492 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-04-25T12:26:06.569 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-04-25T12:26:06.571 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T12:26:06.571 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-04-25T12:26:06.626 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-04-25T12:26:06.626 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-04-25T12:26:06.649 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-04-25T12:26:06.650 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2024-04-25T12:26:06.650 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:26:06.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:26:06.717 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:26:06.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:26:06.743 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T12:26:06.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:26:06.930 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-04-25T12:26:06.979 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T12:26:06.989 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T12:26:06.995 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T12:26:07.132 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T12:26:07.148 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T12:26:07.150 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T12:31:07.305 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2024-04-25T12:31:07.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:31:07.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:31:07.408 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-25T12:31:07.547 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T12:31:07.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-04-25T12:31:07.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T12:31:07.620 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-04-25T12:31:07.697 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-04-25T12:31:07.697 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T12:31:07.711 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T12:31:07.711 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:31:07.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:31:07.718 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T12:31:07.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:31:07.765 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:31:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:31:07.794 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T12:31:07.801 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-04-25T12:31:07.812 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-04-25T12:31:07.825 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:31:07.832 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T12:31:07.837 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:31:07.890 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-04-25T12:31:07.894 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:31:07.897 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:31:07.897 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-04-25T12:31:07.898 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:07.898 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:07.898 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:07.898 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:07.919 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-04-25T12:31:07.959 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T12:31:07.996 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T12:31:08.129 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:31:08.136 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:31:08.136 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-25T12:31:08.136 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:08.137 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:08.137 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:08.137 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:31:08.138 INFO:teuthology.run_tasks:Running task install... 2024-04-25T12:31:08.146 DEBUG:teuthology.task.install:project ceph 2024-04-25T12:31:08.147 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T12:31:08.147 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T12:31:08.147 INFO:teuthology.task.install:Using flavor: default 2024-04-25T12:31:08.152 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T12:31:08.152 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T12:31:08.153 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T12:31:08.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T12:31:08.155 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T12:31:08.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T12:31:08.299 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T12:31:08.299 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T12:31:08.375 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T12:31:08.375 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T12:31:08.407 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:31:08.408 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:31:08.408 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:31:08.433 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:31:08.434 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:31:08.499 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:31:08.540 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T12:31:08.540 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:31:08.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T12:31:08.573 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:31:08.616 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T12:31:08.616 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T12:31:08.637 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-04-25T12:31:08.639 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-04-25T12:31:08.736 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T12:31:08.833 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T12:31:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:48 files removed 2024-04-25T12:31:08.889 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:31:08.899 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-04-25T12:31:08.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-04-25T12:31:09.198 INFO:teuthology.orchestra.run.smithi123.stdout:48 files removed 2024-04-25T12:31:09.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T12:31:09.638 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 170 kB/s | 84 kB 00:00 2024-04-25T12:31:10.064 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-25T12:31:10.147 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 161 kB/s | 84 kB 00:00 2024-04-25T12:31:10.381 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2024-04-25T12:31:10.689 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-25T12:31:10.992 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T12:31:11.289 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-04-25T12:31:12.292 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-25T12:31:12.851 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-25T12:31:14.377 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T12:31:17.602 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-04-25T12:31:18.912 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T12:31:20.200 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-25T12:31:20.757 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T12:31:22.501 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 21 MB 00:02 2024-04-25T12:31:22.942 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-25T12:31:27.584 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T12:31:28.825 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-25T12:31:29.710 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:31:29.711 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:31:29.721 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:31:29.722 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:31:29.723 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:31:29.724 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:31:29.726 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:31:29.728 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:31:29.729 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:31:29.789 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T12:31:29.805 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T12:31:29.806 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T12:31:29.807 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T12:31:29.808 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:31:29.809 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:31:29.810 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:31:29.811 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:31:29.812 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:31:29.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:31:29.814 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:31:29.815 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-04-25T12:31:29.816 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-04-25T12:31:29.816 INFO:teuthology.orchestra.run.smithi153.stdout:Install 148 Packages 2024-04-25T12:31:29.816 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 4 Packages 2024-04-25T12:31:29.816 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:31:29.816 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 214 M 2024-04-25T12:31:29.816 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-04-25T12:31:30.427 INFO:teuthology.orchestra.run.smithi153.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-04-25T12:31:30.611 INFO:teuthology.orchestra.run.smithi153.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 4.6 MB/s | 863 kB 00:00 2024-04-25T12:31:30.669 INFO:teuthology.orchestra.run.smithi153.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 2.5 MB/s | 147 kB 00:00 2024-04-25T12:31:30.936 INFO:teuthology.orchestra.run.smithi153.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 8.5 MB/s | 2.3 MB 00:00 2024-04-25T12:31:31.169 INFO:teuthology.orchestra.run.smithi153.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-04-25T12:31:31.245 INFO:teuthology.orchestra.run.smithi153.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 5.5 MB/s | 5.4 MB 00:00 2024-04-25T12:31:31.276 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:31:31.277 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T12:31:31.296 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T12:31:31.298 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T12:31:31.299 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T12:31:31.300 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T12:31:31.302 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T12:31:31.306 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T12:31:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T12:31:31.379 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T12:31:31.401 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T12:31:31.402 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T12:31:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-04-25T12:31:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T12:31:31.405 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T12:31:31.406 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T12:31:31.407 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T12:31:31.408 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T12:31:31.409 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T12:31:31.410 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T12:31:31.411 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T12:31:31.412 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 4 Packages 2024-04-25T12:31:31.413 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:31:31.414 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 214 M 2024-04-25T12:31:31.414 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-25T12:31:31.712 INFO:teuthology.orchestra.run.smithi153.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 9.0 MB/s | 4.9 MB 00:00 2024-04-25T12:31:32.670 INFO:teuthology.orchestra.run.smithi153.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 8.1 MB/s | 20 MB 00:02 2024-04-25T12:31:32.795 INFO:teuthology.orchestra.run.smithi153.stdout:(9/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 9.3 MB/s | 10 MB 00:01 2024-04-25T12:31:32.820 INFO:teuthology.orchestra.run.smithi153.stdout:(10/152): ceph-selinux-19.0.0-3291.g994e2f20.el 168 kB/s | 25 kB 00:00 2024-04-25T12:31:32.904 INFO:teuthology.orchestra.run.smithi153.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.6 MB/s | 224 kB 00:00 2024-04-25T12:31:32.987 INFO:teuthology.orchestra.run.smithi153.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 324 kB/s | 27 kB 00:00 2024-04-25T12:31:33.071 INFO:teuthology.orchestra.run.smithi153.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T12:31:33.282 INFO:teuthology.orchestra.run.smithi153.stdout:(14/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 8.7 MB/s | 18 MB 00:02 2024-04-25T12:31:33.315 INFO:teuthology.orchestra.run.smithi153.stdout:(15/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.0 MB/s | 756 kB 00:00 2024-04-25T12:31:33.365 INFO:teuthology.orchestra.run.smithi153.stdout:(16/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.9 MB/s | 158 kB 00:00 2024-04-25T12:31:33.391 INFO:teuthology.orchestra.run.smithi153.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.6 MB/s | 127 kB 00:00 2024-04-25T12:31:33.458 INFO:teuthology.orchestra.run.smithi153.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 5.3 MB/s | 497 kB 00:00 2024-04-25T12:31:33.541 INFO:teuthology.orchestra.run.smithi153.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 542 kB/s | 45 kB 00:00 2024-04-25T12:31:33.642 INFO:teuthology.orchestra.run.smithi153.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.3 MB/s | 135 kB 00:00 2024-04-25T12:31:33.742 INFO:teuthology.orchestra.run.smithi153.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.6 MB/s | 161 kB 00:00 2024-04-25T12:31:33.859 INFO:teuthology.orchestra.run.smithi153.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 2.7 MB/s | 321 kB 00:00 2024-04-25T12:31:33.926 INFO:teuthology.orchestra.run.smithi153.stdout:(23/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 9.6 MB/s | 5.2 MB 00:00 2024-04-25T12:31:33.951 INFO:teuthology.orchestra.run.smithi153.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T12:31:34.002 INFO:teuthology.orchestra.run.smithi153.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 100 kB 00:00 2024-04-25T12:31:34.035 INFO:teuthology.orchestra.run.smithi153.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 86 kB 00:00 2024-04-25T12:31:34.178 INFO:teuthology.orchestra.run.smithi153.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.2 MB/s | 169 kB 00:00 2024-04-25T12:31:34.269 INFO:teuthology.orchestra.run.smithi153.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 374 kB/s | 34 kB 00:00 2024-04-25T12:31:34.378 INFO:teuthology.orchestra.run.smithi153.stdout:(29/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 8.1 MB/s | 3.0 MB 00:00 2024-04-25T12:31:34.403 INFO:teuthology.orchestra.run.smithi153.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.0 MB/s | 142 kB 00:00 2024-04-25T12:31:34.845 INFO:teuthology.orchestra.run.smithi153.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 7.6 MB/s | 3.5 MB 00:00 2024-04-25T12:31:35.337 INFO:teuthology.orchestra.run.smithi153.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 7.9 MB/s | 7.4 MB 00:00 2024-04-25T12:31:35.396 INFO:teuthology.orchestra.run.smithi153.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 452 kB/s | 249 kB 00:00 2024-04-25T12:31:35.412 INFO:teuthology.orchestra.run.smithi153.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 657 kB/s | 49 kB 00:00 2024-04-25T12:31:35.454 INFO:teuthology.orchestra.run.smithi153.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 282 kB/s | 16 kB 00:00 2024-04-25T12:31:35.480 INFO:teuthology.orchestra.run.smithi153.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 4.0 MB/s | 273 kB 00:00 2024-04-25T12:31:35.818 INFO:teuthology.orchestra.run.smithi123.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-25T12:31:35.872 INFO:teuthology.orchestra.run.smithi153.stdout:(37/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 14 MB/s | 43 MB 00:03 2024-04-25T12:31:35.897 INFO:teuthology.orchestra.run.smithi153.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 44 kB/s | 18 kB 00:00 2024-04-25T12:31:35.930 INFO:teuthology.orchestra.run.smithi153.stdout:(39/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 1.6 MB/s | 762 kB 00:00 2024-04-25T12:31:36.034 INFO:teuthology.orchestra.run.smithi123.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 3.9 MB/s | 863 kB 00:00 2024-04-25T12:31:36.056 INFO:teuthology.orchestra.run.smithi153.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-04-25T12:31:36.127 INFO:teuthology.orchestra.run.smithi123.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.6 MB/s | 147 kB 00:00 2024-04-25T12:31:36.131 INFO:teuthology.orchestra.run.smithi153.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-25T12:31:36.156 INFO:teuthology.orchestra.run.smithi153.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-25T12:31:36.198 INFO:teuthology.orchestra.run.smithi153.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-25T12:31:36.290 INFO:teuthology.orchestra.run.smithi153.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 800 kB/s | 106 kB 00:00 2024-04-25T12:31:36.332 INFO:teuthology.orchestra.run.smithi153.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-25T12:31:36.357 INFO:teuthology.orchestra.run.smithi153.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 855 kB/s | 135 kB 00:00 2024-04-25T12:31:36.383 INFO:teuthology.orchestra.run.smithi153.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-25T12:31:36.408 INFO:teuthology.orchestra.run.smithi153.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T12:31:36.410 INFO:teuthology.orchestra.run.smithi123.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 6.3 MB/s | 5.4 MB 00:00 2024-04-25T12:31:36.492 INFO:teuthology.orchestra.run.smithi153.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-04-25T12:31:36.502 INFO:teuthology.orchestra.run.smithi123.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 6.0 MB/s | 2.3 MB 00:00 2024-04-25T12:31:36.533 INFO:teuthology.orchestra.run.smithi153.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 6.2 MB/s | 943 kB 00:00 2024-04-25T12:31:36.585 INFO:teuthology.orchestra.run.smithi153.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 606 kB/s | 107 kB 00:00 2024-04-25T12:31:36.602 INFO:teuthology.orchestra.run.smithi123.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 7.9 MB/s | 1.5 MB 00:00 2024-04-25T12:31:36.667 INFO:teuthology.orchestra.run.smithi153.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 123 kB/s | 21 kB 00:00 2024-04-25T12:31:36.692 INFO:teuthology.orchestra.run.smithi153.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-04-25T12:31:36.718 INFO:teuthology.orchestra.run.smithi153.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 659 kB/s | 33 kB 00:00 2024-04-25T12:31:36.868 INFO:teuthology.orchestra.run.smithi153.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-25T12:31:36.893 INFO:teuthology.orchestra.run.smithi153.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 91 kB/s | 18 kB 00:00 2024-04-25T12:31:36.918 INFO:teuthology.orchestra.run.smithi153.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 56 kB/s | 11 kB 00:00 2024-04-25T12:31:36.943 INFO:teuthology.orchestra.run.smithi153.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T12:31:36.977 INFO:teuthology.orchestra.run.smithi153.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 782 kB/s | 45 kB 00:00 2024-04-25T12:31:37.002 INFO:teuthology.orchestra.run.smithi153.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T12:31:37.036 INFO:teuthology.orchestra.run.smithi153.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-25T12:31:37.061 INFO:teuthology.orchestra.run.smithi153.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 309 kB/s | 25 kB 00:00 2024-04-25T12:31:37.086 INFO:teuthology.orchestra.run.smithi153.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-04-25T12:31:37.111 INFO:teuthology.orchestra.run.smithi153.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T12:31:37.137 INFO:teuthology.orchestra.run.smithi153.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-25T12:31:37.162 INFO:teuthology.orchestra.run.smithi153.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T12:31:37.219 INFO:teuthology.orchestra.run.smithi123.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 6.8 MB/s | 4.9 MB 00:00 2024-04-25T12:31:37.337 INFO:teuthology.orchestra.run.smithi153.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-25T12:31:37.379 INFO:teuthology.orchestra.run.smithi153.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.4 MB/s | 1.6 MB 00:00 2024-04-25T12:31:37.446 INFO:teuthology.orchestra.run.smithi153.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-25T12:31:37.480 INFO:teuthology.orchestra.run.smithi153.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-25T12:31:37.505 INFO:teuthology.orchestra.run.smithi153.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T12:31:37.655 INFO:teuthology.orchestra.run.smithi153.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-04-25T12:31:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 9.3 MB/s | 20 MB 00:02 2024-04-25T12:31:37.689 INFO:teuthology.orchestra.run.smithi153.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-04-25T12:31:37.714 INFO:teuthology.orchestra.run.smithi153.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 422 kB/s | 87 kB 00:00 2024-04-25T12:31:37.728 INFO:teuthology.orchestra.run.smithi123.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 384 kB/s | 25 kB 00:00 2024-04-25T12:31:37.822 INFO:teuthology.orchestra.run.smithi153.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-04-25T12:31:37.848 INFO:teuthology.orchestra.run.smithi153.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-25T12:31:37.873 INFO:teuthology.orchestra.run.smithi153.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T12:31:37.898 INFO:teuthology.orchestra.run.smithi153.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 344 kB/s | 26 kB 00:00 2024-04-25T12:31:37.923 INFO:teuthology.orchestra.run.smithi153.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T12:31:37.948 INFO:teuthology.orchestra.run.smithi153.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T12:31:37.974 INFO:teuthology.orchestra.run.smithi153.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 464 kB/s | 35 kB 00:00 2024-04-25T12:31:37.999 INFO:teuthology.orchestra.run.smithi153.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T12:31:38.091 INFO:teuthology.orchestra.run.smithi153.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-25T12:31:38.116 INFO:teuthology.orchestra.run.smithi153.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-25T12:31:38.142 INFO:teuthology.orchestra.run.smithi153.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 554 kB/s | 77 kB 00:00 2024-04-25T12:31:38.167 INFO:teuthology.orchestra.run.smithi153.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 566 kB/s | 42 kB 00:00 2024-04-25T12:31:38.192 INFO:teuthology.orchestra.run.smithi153.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T12:31:38.217 INFO:teuthology.orchestra.run.smithi153.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T12:31:38.242 INFO:teuthology.orchestra.run.smithi153.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-25T12:31:38.267 INFO:teuthology.orchestra.run.smithi153.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T12:31:38.293 INFO:teuthology.orchestra.run.smithi153.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 717 kB/s | 54 kB 00:00 2024-04-25T12:31:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 9.6 MB/s | 18 MB 00:01 2024-04-25T12:31:38.476 INFO:teuthology.orchestra.run.smithi153.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 231 kB/s | 42 kB 00:00 2024-04-25T12:31:38.501 INFO:teuthology.orchestra.run.smithi153.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 215 kB/s | 44 kB 00:00 2024-04-25T12:31:38.582 INFO:teuthology.orchestra.run.smithi123.stdout:(11/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 7.4 MB/s | 10 MB 00:01 2024-04-25T12:31:38.620 INFO:teuthology.orchestra.run.smithi123.stdout:(12/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.3 MB/s | 224 kB 00:00 2024-04-25T12:31:38.645 INFO:teuthology.orchestra.run.smithi123.stdout:(13/152): cephfs-shell-19.0.0-3291.g994e2f20.el 437 kB/s | 27 kB 00:00 2024-04-25T12:31:38.660 INFO:teuthology.orchestra.run.smithi153.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-04-25T12:31:38.671 INFO:teuthology.orchestra.run.smithi123.stdout:(14/152): libcephfs-devel-19.0.0-3291.g994e2f20 641 kB/s | 32 kB 00:00 2024-04-25T12:31:38.685 INFO:teuthology.orchestra.run.smithi153.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-04-25T12:31:38.711 INFO:teuthology.orchestra.run.smithi153.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 172 kB/s | 36 kB 00:00 2024-04-25T12:31:38.736 INFO:teuthology.orchestra.run.smithi153.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T12:31:38.738 INFO:teuthology.orchestra.run.smithi123.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.3 MB/s | 158 kB 00:00 2024-04-25T12:31:38.761 INFO:teuthology.orchestra.run.smithi153.stdout:(98/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-04-25T12:31:38.779 INFO:teuthology.orchestra.run.smithi123.stdout:(16/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 5.5 MB/s | 756 kB 00:00 2024-04-25T12:31:38.786 INFO:teuthology.orchestra.run.smithi153.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 773 kB/s | 58 kB 00:00 2024-04-25T12:31:38.812 INFO:teuthology.orchestra.run.smithi153.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T12:31:38.813 INFO:teuthology.orchestra.run.smithi123.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T12:31:38.837 INFO:teuthology.orchestra.run.smithi153.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T12:31:38.855 INFO:teuthology.orchestra.run.smithi123.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 6.5 MB/s | 497 kB 00:00 2024-04-25T12:31:38.880 INFO:teuthology.orchestra.run.smithi153.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 477 kB/s | 45 kB 00:00 2024-04-25T12:31:38.904 INFO:teuthology.orchestra.run.smithi153.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-04-25T12:31:38.938 INFO:teuthology.orchestra.run.smithi153.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 15 MB/s | 863 kB 00:00 2024-04-25T12:31:38.943 INFO:teuthology.orchestra.run.smithi123.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 518 kB/s | 45 kB 00:00 2024-04-25T12:31:39.039 INFO:teuthology.orchestra.run.smithi123.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.4 MB/s | 135 kB 00:00 2024-04-25T12:31:39.131 INFO:teuthology.orchestra.run.smithi123.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.7 MB/s | 161 kB 00:00 2024-04-25T12:31:39.213 INFO:teuthology.orchestra.run.smithi153.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-25T12:31:39.271 INFO:teuthology.orchestra.run.smithi153.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 58 kB/s | 19 kB 00:00 2024-04-25T12:31:39.315 INFO:teuthology.orchestra.run.smithi123.stdout:(22/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 10 MB/s | 5.2 MB 00:00 2024-04-25T12:31:39.338 INFO:teuthology.orchestra.run.smithi153.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 714 kB/s | 309 kB 00:00 2024-04-25T12:31:39.356 INFO:teuthology.orchestra.run.smithi123.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 1.4 MB/s | 321 kB 00:00 2024-04-25T12:31:39.390 INFO:teuthology.orchestra.run.smithi123.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.9 MB/s | 298 kB 00:00 2024-04-25T12:31:39.423 INFO:teuthology.orchestra.run.smithi123.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 100 kB 00:00 2024-04-25T12:31:39.449 INFO:teuthology.orchestra.run.smithi123.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.4 MB/s | 86 kB 00:00 2024-04-25T12:31:39.549 INFO:teuthology.orchestra.run.smithi123.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T12:31:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 317 kB/s | 34 kB 00:00 2024-04-25T12:31:39.784 INFO:teuthology.orchestra.run.smithi123.stdout:(29/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 8.5 MB/s | 3.0 MB 00:00 2024-04-25T12:31:39.817 INFO:teuthology.orchestra.run.smithi123.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 902 kB/s | 142 kB 00:00 2024-04-25T12:31:39.847 INFO:teuthology.orchestra.run.smithi153.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 45 kB/s | 26 kB 00:00 2024-04-25T12:31:39.872 INFO:teuthology.orchestra.run.smithi153.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 91 kB/s | 49 kB 00:00 2024-04-25T12:31:40.072 INFO:teuthology.orchestra.run.smithi153.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-04-25T12:31:40.097 INFO:teuthology.orchestra.run.smithi153.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 303 kB/s | 67 kB 00:00 2024-04-25T12:31:40.122 INFO:teuthology.orchestra.run.smithi153.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 610 kB/s | 151 kB 00:00 2024-04-25T12:31:40.265 INFO:teuthology.orchestra.run.smithi153.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T12:31:40.298 INFO:teuthology.orchestra.run.smithi153.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-04-25T12:31:40.332 INFO:teuthology.orchestra.run.smithi153.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 208 kB/s | 43 kB 00:00 2024-04-25T12:31:40.357 INFO:teuthology.orchestra.run.smithi153.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 352 kB/s | 32 kB 00:00 2024-04-25T12:31:40.382 INFO:teuthology.orchestra.run.smithi153.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T12:31:40.392 INFO:teuthology.orchestra.run.smithi123.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 5.8 MB/s | 3.5 MB 00:00 2024-04-25T12:31:40.441 INFO:teuthology.orchestra.run.smithi153.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T12:31:40.475 INFO:teuthology.orchestra.run.smithi153.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T12:31:40.500 INFO:teuthology.orchestra.run.smithi153.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T12:31:40.509 INFO:teuthology.orchestra.run.smithi123.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.1 MB/s | 249 kB 00:00 2024-04-25T12:31:40.525 INFO:teuthology.orchestra.run.smithi153.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-25T12:31:40.558 INFO:teuthology.orchestra.run.smithi153.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T12:31:40.642 INFO:teuthology.orchestra.run.smithi153.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T12:31:40.668 INFO:teuthology.orchestra.run.smithi153.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-25T12:31:40.668 INFO:teuthology.orchestra.run.smithi123.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-04-25T12:31:40.692 INFO:teuthology.orchestra.run.smithi153.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-04-25T12:31:40.723 INFO:teuthology.orchestra.run.smithi123.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 232 kB/s | 49 kB 00:00 2024-04-25T12:31:40.735 INFO:teuthology.orchestra.run.smithi153.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-25T12:31:40.752 INFO:teuthology.orchestra.run.smithi123.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 201 kB/s | 16 kB 00:00 2024-04-25T12:31:40.760 INFO:teuthology.orchestra.run.smithi153.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-25T12:31:40.785 INFO:teuthology.orchestra.run.smithi153.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-25T12:31:40.785 INFO:teuthology.orchestra.run.smithi123.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 4.4 MB/s | 273 kB 00:00 2024-04-25T12:31:40.844 INFO:teuthology.orchestra.run.smithi123.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 8.2 MB/s | 762 kB 00:00 2024-04-25T12:31:40.869 INFO:teuthology.orchestra.run.smithi123.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 220 kB/s | 18 kB 00:00 2024-04-25T12:31:40.893 INFO:teuthology.orchestra.run.smithi153.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-04-25T12:31:40.969 INFO:teuthology.orchestra.run.smithi153.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-25T12:31:40.994 INFO:teuthology.orchestra.run.smithi153.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2024-04-25T12:31:41.069 INFO:teuthology.orchestra.run.smithi153.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 449 kB/s | 79 kB 00:00 2024-04-25T12:31:41.103 INFO:teuthology.orchestra.run.smithi153.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 432 kB/s | 58 kB 00:00 2024-04-25T12:31:41.145 INFO:teuthology.orchestra.run.smithi153.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-25T12:31:41.195 INFO:teuthology.orchestra.run.smithi153.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-25T12:31:41.237 INFO:teuthology.orchestra.run.smithi153.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 671 kB/s | 90 kB 00:00 2024-04-25T12:31:41.262 INFO:teuthology.orchestra.run.smithi153.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-25T12:31:41.287 INFO:teuthology.orchestra.run.smithi153.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-25T12:31:41.379 INFO:teuthology.orchestra.run.smithi153.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-25T12:31:41.403 INFO:teuthology.orchestra.run.smithi123.stdout:(39/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 12 MB/s | 43 MB 00:03 2024-04-25T12:31:41.421 INFO:teuthology.orchestra.run.smithi153.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 442 kB/s | 59 kB 00:00 2024-04-25T12:31:41.453 INFO:teuthology.orchestra.run.smithi123.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 132 kB/s | 80 kB 00:00 2024-04-25T12:31:41.455 INFO:teuthology.orchestra.run.smithi153.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 214 kB/s | 36 kB 00:00 2024-04-25T12:31:41.486 INFO:teuthology.orchestra.run.smithi123.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 195 kB 00:00 2024-04-25T12:31:41.520 INFO:teuthology.orchestra.run.smithi123.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.9 MB/s | 253 kB 00:00 2024-04-25T12:31:41.522 INFO:teuthology.orchestra.run.smithi153.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 358 kB/s | 51 kB 00:00 2024-04-25T12:31:41.547 INFO:teuthology.orchestra.run.smithi153.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-25T12:31:41.572 INFO:teuthology.orchestra.run.smithi153.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-04-25T12:31:41.579 INFO:teuthology.orchestra.run.smithi123.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.2 MB 00:00 2024-04-25T12:31:41.629 INFO:teuthology.orchestra.run.smithi123.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-25T12:31:41.654 INFO:teuthology.orchestra.run.smithi123.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 908 kB/s | 106 kB 00:00 2024-04-25T12:31:41.681 INFO:teuthology.orchestra.run.smithi123.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-25T12:31:41.713 INFO:teuthology.orchestra.run.smithi123.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 2.2 MB/s | 125 kB 00:00 2024-04-25T12:31:41.738 INFO:teuthology.orchestra.run.smithi123.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.6 MB/s | 215 kB 00:00 2024-04-25T12:31:41.747 INFO:teuthology.orchestra.run.smithi153.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-25T12:31:41.772 INFO:teuthology.orchestra.run.smithi153.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 100 kB/s | 20 kB 00:00 2024-04-25T12:31:41.789 INFO:teuthology.orchestra.run.smithi123.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 6.4 MB/s | 689 kB 00:00 2024-04-25T12:31:41.798 INFO:teuthology.orchestra.run.smithi153.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 851 kB/s | 192 kB 00:00 2024-04-25T12:31:41.822 INFO:teuthology.orchestra.run.smithi123.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-25T12:31:41.872 INFO:teuthology.orchestra.run.smithi123.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 7.1 MB/s | 943 kB 00:00 2024-04-25T12:31:41.891 INFO:teuthology.orchestra.run.smithi123.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 222 kB/s | 21 kB 00:00 2024-04-25T12:31:41.922 INFO:teuthology.orchestra.run.smithi123.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.2 MB/s | 313 kB 00:00 2024-04-25T12:31:41.948 INFO:teuthology.orchestra.run.smithi153.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-25T12:31:41.956 INFO:teuthology.orchestra.run.smithi123.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 518 kB/s | 33 kB 00:00 2024-04-25T12:31:42.024 INFO:teuthology.orchestra.run.smithi153.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T12:31:42.040 INFO:teuthology.orchestra.run.smithi123.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-04-25T12:31:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 150 kB/s | 18 kB 00:00 2024-04-25T12:31:42.082 INFO:teuthology.orchestra.run.smithi123.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 97 kB/s | 11 kB 00:00 2024-04-25T12:31:42.115 INFO:teuthology.orchestra.run.smithi123.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T12:31:42.132 INFO:teuthology.orchestra.run.smithi153.stdout:(150/152): librados2-19.0.0-3291.g994e2f20.el9. 9.3 MB/s | 3.3 MB 00:00 2024-04-25T12:31:42.166 INFO:teuthology.orchestra.run.smithi123.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 421 kB/s | 45 kB 00:00 2024-04-25T12:31:42.174 INFO:teuthology.orchestra.run.smithi153.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T12:31:42.216 INFO:teuthology.orchestra.run.smithi123.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.9 MB/s | 662 kB 00:00 2024-04-25T12:31:42.241 INFO:teuthology.orchestra.run.smithi123.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 372 kB/s | 25 kB 00:00 2024-04-25T12:31:42.275 INFO:teuthology.orchestra.run.smithi123.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 5.6 MB/s | 582 kB 00:00 2024-04-25T12:31:42.300 INFO:teuthology.orchestra.run.smithi123.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 588 kB/s | 47 kB 00:00 2024-04-25T12:31:42.334 INFO:teuthology.orchestra.run.smithi123.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 292 kB 00:00 2024-04-25T12:31:42.367 INFO:teuthology.orchestra.run.smithi123.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 188 kB 00:00 2024-04-25T12:31:42.392 INFO:teuthology.orchestra.run.smithi123.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 389 kB/s | 35 kB 00:00 2024-04-25T12:31:42.449 INFO:teuthology.orchestra.run.smithi153.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 4.7 MB/s | 3.1 MB 00:00 2024-04-25T12:31:42.458 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:31:42.458 INFO:teuthology.orchestra.run.smithi153.stdout:Total 17 MB/s | 214 MB 00:12 2024-04-25T12:31:42.543 INFO:teuthology.orchestra.run.smithi123.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-04-25T12:31:42.718 INFO:teuthology.orchestra.run.smithi123.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-04-25T12:31:42.802 INFO:teuthology.orchestra.run.smithi123.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T12:31:42.864 INFO:teuthology.orchestra.run.smithi123.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 2.8 MB/s | 911 kB 00:00 2024-04-25T12:31:42.894 INFO:teuthology.orchestra.run.smithi123.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 785 kB/s | 133 kB 00:00 2024-04-25T12:31:43.036 INFO:teuthology.orchestra.run.smithi123.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-25T12:31:43.061 INFO:teuthology.orchestra.run.smithi123.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 577 kB/s | 87 kB 00:00 2024-04-25T12:31:43.119 INFO:teuthology.orchestra.run.smithi123.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-04-25T12:31:43.278 INFO:teuthology.orchestra.run.smithi123.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-04-25T12:31:43.303 INFO:teuthology.orchestra.run.smithi123.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T12:31:43.328 INFO:teuthology.orchestra.run.smithi123.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-04-25T12:31:43.371 INFO:teuthology.orchestra.run.smithi123.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 634 kB/s | 26 kB 00:00 2024-04-25T12:31:43.396 INFO:teuthology.orchestra.run.smithi123.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 293 kB/s | 19 kB 00:00 2024-04-25T12:31:43.421 INFO:teuthology.orchestra.run.smithi123.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 172 kB 00:00 2024-04-25T12:31:43.463 INFO:teuthology.orchestra.run.smithi123.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 532 kB/s | 35 kB 00:00 2024-04-25T12:31:43.545 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-04-25T12:31:43.605 INFO:teuthology.orchestra.run.smithi123.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-25T12:31:43.629 INFO:teuthology.orchestra.run.smithi123.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-04-25T12:31:43.655 INFO:teuthology.orchestra.run.smithi123.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-25T12:31:43.688 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-04-25T12:31:43.688 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-04-25T12:31:43.697 INFO:teuthology.orchestra.run.smithi123.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-25T12:31:43.722 INFO:teuthology.orchestra.run.smithi123.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 637 kB/s | 42 kB 00:00 2024-04-25T12:31:43.747 INFO:teuthology.orchestra.run.smithi123.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-25T12:31:43.772 INFO:teuthology.orchestra.run.smithi123.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 159 kB 00:00 2024-04-25T12:31:43.797 INFO:teuthology.orchestra.run.smithi123.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-25T12:31:43.823 INFO:teuthology.orchestra.run.smithi123.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 696 kB/s | 51 kB 00:00 2024-04-25T12:31:43.857 INFO:teuthology.orchestra.run.smithi123.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 923 kB/s | 54 kB 00:00 2024-04-25T12:31:44.032 INFO:teuthology.orchestra.run.smithi123.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 205 kB/s | 42 kB 00:00 2024-04-25T12:31:44.065 INFO:teuthology.orchestra.run.smithi123.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 218 kB/s | 44 kB 00:00 2024-04-25T12:31:44.257 INFO:teuthology.orchestra.run.smithi123.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T12:31:44.316 INFO:teuthology.orchestra.run.smithi123.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 804 kB/s | 221 kB 00:00 2024-04-25T12:31:44.341 INFO:teuthology.orchestra.run.smithi123.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 132 kB/s | 36 kB 00:00 2024-04-25T12:31:44.375 INFO:teuthology.orchestra.run.smithi123.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 833 kB/s | 47 kB 00:00 2024-04-25T12:31:44.400 INFO:teuthology.orchestra.run.smithi123.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-25T12:31:44.425 INFO:teuthology.orchestra.run.smithi123.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 724 kB/s | 58 kB 00:00 2024-04-25T12:31:44.459 INFO:teuthology.orchestra.run.smithi123.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-04-25T12:31:44.484 INFO:teuthology.orchestra.run.smithi123.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 778 kB/s | 64 kB 00:00 2024-04-25T12:31:44.501 INFO:teuthology.orchestra.run.smithi123.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-04-25T12:31:44.526 INFO:teuthology.orchestra.run.smithi123.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 537 kB/s | 22 kB 00:00 2024-04-25T12:31:44.568 INFO:teuthology.orchestra.run.smithi123.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T12:31:44.844 INFO:teuthology.orchestra.run.smithi123.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-04-25T12:31:45.128 INFO:teuthology.orchestra.run.smithi123.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 515 kB/s | 309 kB 00:00 2024-04-25T12:31:45.161 INFO:teuthology.orchestra.run.smithi123.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 834 kB/s | 548 kB 00:00 2024-04-25T12:31:45.395 INFO:teuthology.orchestra.run.smithi123.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 98 kB/s | 26 kB 00:00 2024-04-25T12:31:45.417 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-04-25T12:31:45.418 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-04-25T12:31:45.520 INFO:teuthology.orchestra.run.smithi123.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 136 kB/s | 49 kB 00:00 2024-04-25T12:31:45.745 INFO:teuthology.orchestra.run.smithi123.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 193 kB/s | 67 kB 00:00 2024-04-25T12:31:45.981 INFO:teuthology.orchestra.run.smithi123.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-04-25T12:31:46.004 INFO:teuthology.orchestra.run.smithi123.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 312 kB/s | 151 kB 00:00 2024-04-25T12:31:46.354 INFO:teuthology.orchestra.run.smithi123.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-04-25T12:31:46.430 INFO:teuthology.orchestra.run.smithi123.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T12:31:46.613 INFO:teuthology.orchestra.run.smithi123.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 125 kB/s | 32 kB 00:00 2024-04-25T12:31:46.672 INFO:teuthology.orchestra.run.smithi123.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 793 kB/s | 548 kB 00:00 2024-04-25T12:31:46.705 INFO:teuthology.orchestra.run.smithi123.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 54 kB/s | 15 kB 00:00 2024-04-25T12:31:47.064 INFO:teuthology.orchestra.run.smithi123.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 382 kB/s | 172 kB 00:00 2024-04-25T12:31:47.189 INFO:teuthology.orchestra.run.smithi123.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 694 kB/s | 358 kB 00:00 2024-04-25T12:31:47.289 INFO:teuthology.orchestra.run.smithi123.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 520 kB/s | 303 kB 00:00 2024-04-25T12:31:47.323 INFO:teuthology.orchestra.run.smithi123.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 137 kB/s | 35 kB 00:00 2024-04-25T12:31:47.715 INFO:teuthology.orchestra.run.smithi123.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 392 kB/s | 206 kB 00:00 2024-04-25T12:31:47.748 INFO:teuthology.orchestra.run.smithi123.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-25T12:31:47.915 INFO:teuthology.orchestra.run.smithi123.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 54 kB/s | 11 kB 00:00 2024-04-25T12:31:48.016 INFO:teuthology.orchestra.run.smithi123.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-25T12:31:48.091 INFO:teuthology.orchestra.run.smithi123.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-04-25T12:31:48.200 INFO:teuthology.orchestra.run.smithi123.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 82 kB/s | 23 kB 00:00 2024-04-25T12:31:48.283 INFO:teuthology.orchestra.run.smithi123.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-25T12:31:48.361 INFO:teuthology.orchestra.run.smithi123.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 73 kB/s | 20 kB 00:00 2024-04-25T12:31:48.636 INFO:teuthology.orchestra.run.smithi123.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-04-25T12:31:48.703 INFO:teuthology.orchestra.run.smithi123.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-04-25T12:31:48.903 INFO:teuthology.orchestra.run.smithi123.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-25T12:31:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-04-25T12:31:49.170 INFO:teuthology.orchestra.run.smithi123.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 62 kB/s | 16 kB 00:00 2024-04-25T12:31:49.237 INFO:teuthology.orchestra.run.smithi123.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 511 kB/s | 272 kB 00:00 2024-04-25T12:31:49.429 INFO:teuthology.orchestra.run.smithi123.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 204 kB/s | 90 kB 00:00 2024-04-25T12:31:49.454 INFO:teuthology.orchestra.run.smithi123.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 100 kB/s | 28 kB 00:00 2024-04-25T12:31:49.504 INFO:teuthology.orchestra.run.smithi123.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-25T12:31:49.984 INFO:teuthology.orchestra.run.smithi123.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 75 kB/s | 36 kB 00:00 2024-04-25T12:31:50.014 INFO:teuthology.orchestra.run.smithi123.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 106 kB/s | 59 kB 00:00 2024-04-25T12:31:50.047 INFO:teuthology.orchestra.run.smithi123.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 306 kB/s | 188 kB 00:00 2024-04-25T12:31:50.348 INFO:teuthology.orchestra.run.smithi123.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 141 kB/s | 51 kB 00:00 2024-04-25T12:31:50.472 INFO:teuthology.orchestra.run.smithi123.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 211 kB/s | 90 kB 00:00 2024-04-25T12:31:50.523 INFO:teuthology.orchestra.run.smithi123.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 452 kB/s | 230 kB 00:00 2024-04-25T12:31:50.740 INFO:teuthology.orchestra.run.smithi123.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-25T12:31:50.965 INFO:teuthology.orchestra.run.smithi123.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 693 kB/s | 427 kB 00:00 2024-04-25T12:31:50.990 INFO:teuthology.orchestra.run.smithi123.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 411 kB/s | 192 kB 00:00 2024-04-25T12:31:51.349 INFO:teuthology.orchestra.run.smithi123.stdout:(148/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-04-25T12:31:51.457 INFO:teuthology.orchestra.run.smithi123.stdout:(149/152): librados2-19.0.0-3291.g994e2f20.el9. 6.8 MB/s | 3.3 MB 00:00 2024-04-25T12:31:51.488 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-04-25T12:31:51.516 INFO:teuthology.orchestra.run.smithi123.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-04-25T12:31:51.561 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:31:51.599 INFO:teuthology.orchestra.run.smithi123.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T12:31:51.607 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:31:51.661 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:31:51.666 INFO:teuthology.orchestra.run.smithi123.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-04-25T12:31:51.678 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-25T12:31:51.678 INFO:teuthology.orchestra.run.smithi123.stdout:Total 11 MB/s | 214 MB 00:20 2024-04-25T12:31:51.703 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:31:51.729 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:31:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:31:52.108 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:31:52.229 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T12:31:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T12:31:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T12:31:52.472 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T12:31:52.502 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T12:31:52.648 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:31:52.698 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:31:52.726 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T12:31:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T12:31:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T12:31:52.898 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T12:31:52.920 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:31:52.942 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:31:53.146 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:31:53.178 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:31:53.285 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:31:53.374 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:31:53.410 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:31:53.445 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:31:53.536 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:31:53.594 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:31:53.638 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:31:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:31:53.698 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:31:53.768 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:31:53.820 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:31:53.851 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:31:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:31:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:31:54.022 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T12:31:54.065 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T12:31:54.094 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T12:31:54.163 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:31:54.225 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:31:54.298 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:31:54.430 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:31:54.466 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-25T12:31:54.486 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:31:54.705 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-25T12:31:54.706 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-25T12:31:55.005 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:31:55.098 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:31:55.127 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:31:55.170 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:31:55.207 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T12:31:55.249 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:31:55.410 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:31:55.460 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:31:55.514 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:31:55.579 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:31:55.611 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:31:55.647 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:31:55.694 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:31:55.730 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:31:55.773 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:31:55.816 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:31:55.952 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:31:55.996 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:31:56.018 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:31:56.059 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:31:56.154 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:31:56.184 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:31:56.224 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:31:56.250 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:31:56.274 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:31:56.349 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:31:56.477 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:31:56.506 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:31:56.526 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:31:56.560 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:31:56.599 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:31:56.630 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:31:56.661 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:31:56.695 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:31:56.729 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:31:56.770 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:31:56.804 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:31:56.890 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:31:56.934 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:31:56.966 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:31:57.012 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:31:57.110 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:31:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:31:57.497 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-25T12:31:57.497 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-25T12:31:58.001 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:31:58.048 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:31:58.081 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:31:58.179 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:31:58.230 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:31:58.317 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:31:58.358 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:31:58.528 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:31:58.564 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:31:58.659 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:31:59.301 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:31:59.413 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:32:00.638 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:32:00.717 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:32:00.745 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:32:00.796 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:32:01.117 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:32:01.167 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T12:32:01.257 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T12:32:01.288 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T12:32:01.324 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:32:01.812 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:32:01.844 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T12:32:01.898 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T12:32:01.921 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T12:32:04.174 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T12:32:04.217 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T12:32:04.229 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-25T12:32:04.415 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T12:32:04.503 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T12:32:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T12:32:04.660 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T12:32:04.716 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T12:32:04.782 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T12:32:04.820 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T12:32:04.857 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T12:32:04.911 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T12:32:04.911 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:32:04.912 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:32:04.912 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:04.946 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T12:32:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T12:32:05.428 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T12:32:05.613 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T12:32:05.657 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T12:32:05.785 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T12:32:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T12:32:06.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T12:32:06.220 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T12:32:06.244 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T12:32:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T12:32:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T12:32:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T12:32:06.513 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:32:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T12:32:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:32:06.930 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T12:32:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T12:32:07.253 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T12:32:07.297 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T12:32:07.346 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T12:32:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T12:32:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T12:32:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T12:32:07.613 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T12:32:07.653 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T12:32:07.770 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T12:32:07.852 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T12:32:07.891 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T12:32:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T12:32:08.094 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T12:32:08.133 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T12:32:08.186 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T12:32:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T12:32:08.376 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T12:32:08.438 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T12:32:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T12:32:08.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T12:32:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T12:32:09.499 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T12:32:09.610 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T12:32:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T12:32:09.748 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T12:32:09.809 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T12:32:09.876 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T12:32:10.164 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T12:32:10.242 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T12:32:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T12:32:10.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T12:32:10.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T12:32:10.504 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T12:32:10.571 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T12:32:10.612 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T12:32:10.673 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T12:32:10.740 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T12:32:10.948 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T12:32:11.002 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T12:32:11.038 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T12:32:11.086 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T12:32:11.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T12:32:11.280 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T12:32:11.327 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T12:32:11.362 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T12:32:11.394 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T12:32:11.581 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T12:32:11.754 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T12:32:11.820 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T12:32:11.866 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T12:32:11.913 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T12:32:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T12:32:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T12:32:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T12:32:12.116 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T12:32:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T12:32:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T12:32:12.309 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T12:32:12.453 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T12:32:12.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T12:32:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T12:32:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T12:32:12.792 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T12:32:12.861 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T12:32:13.846 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T12:32:13.902 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T12:32:13.955 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T12:32:14.087 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T12:32:14.163 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T12:32:14.274 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T12:32:14.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T12:32:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T12:32:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T12:32:14.751 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T12:32:15.607 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T12:32:15.756 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-04-25T12:32:16.321 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-04-25T12:32:16.322 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:16.499 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T12:32:16.514 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T12:32:16.514 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:16.515 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:32:16.515 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:17.380 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T12:32:17.472 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T12:32:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T12:32:17.588 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T12:32:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T12:32:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T12:32:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T12:32:18.135 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T12:32:18.135 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:18.135 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:32:18.136 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:32:18.136 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:32:18.136 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:18.200 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T12:32:18.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T12:32:18.335 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T12:32:18.361 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T12:32:18.383 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T12:32:18.383 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:18.383 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:32:18.383 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:32:18.383 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:32:18.384 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:18.844 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T12:32:18.867 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T12:32:18.867 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:18.867 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:32:18.867 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:32:18.867 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:32:18.868 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:18.897 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T12:32:18.930 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T12:32:18.966 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:32:19.014 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:32:19.071 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:32:19.079 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T12:32:19.106 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T12:32:19.254 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:32:19.312 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:32:19.389 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:32:19.477 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:32:19.617 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:32:19.653 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:32:19.685 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:32:19.713 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:32:20.140 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T12:32:20.170 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T12:32:20.202 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T12:32:20.263 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T12:32:20.301 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T12:32:21.341 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T12:32:21.377 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T12:32:22.008 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T12:32:22.044 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T12:32:22.343 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T12:32:22.370 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T12:32:22.473 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T12:32:22.644 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T12:32:22.672 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T12:32:22.695 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T12:32:22.695 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:22.695 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:32:22.695 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:32:22.695 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:32:22.695 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:22.711 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T12:32:22.739 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T12:32:22.750 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T12:32:22.772 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T12:32:22.873 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T12:32:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T12:32:22.989 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T12:32:22.989 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T12:32:22.989 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T12:32:22.990 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:23.090 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T12:32:23.876 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:32:23.899 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T12:32:23.922 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T12:32:23.922 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:23.922 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:32:23.922 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:32:23.922 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:32:23.922 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:23.967 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T12:32:23.989 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T12:32:23.989 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:23.989 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:32:23.989 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:24.034 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T12:32:24.056 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T12:32:24.056 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:24.056 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:32:24.056 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:32:24.056 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:32:24.056 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:24.368 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T12:32:24.391 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T12:32:24.391 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:24.391 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:32:24.391 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:32:24.391 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:32:24.391 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T12:32:28.713 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T12:32:28.754 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T12:32:28.785 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:32:28.813 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:32:28.995 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T12:32:29.034 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:32:29.064 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T12:32:29.636 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:32:29.677 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:32:29.697 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:32:30.530 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T12:32:30.615 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:32:30.656 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:32:30.738 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:32:30.769 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:32:30.769 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:32:30.861 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:32:30.894 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:32:30.895 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:32:30.967 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:32:31.043 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-04-25T12:32:36.340 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:36.611 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T12:32:36.661 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T12:32:36.662 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:36.662 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T12:32:36.662 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:36.693 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:32:36.693 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T12:32:36.693 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T12:32:36.693 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T12:32:36.693 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T12:32:36.694 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T12:32:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T12:32:36.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:32:36.708 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:32:36.709 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:32:36.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:32:36.711 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:32:36.712 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:32:36.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:32:36.714 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:32:36.715 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:32:36.716 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:32:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T12:32:38.497 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T12:32:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T12:32:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:32:38.497 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T12:32:38.497 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:38.755 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T12:32:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T12:32:38.823 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:38.824 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T12:32:38.824 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:32:38.824 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T12:32:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:38.851 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:32:38.851 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:38.851 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-04-25T12:32:38.851 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.852 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.853 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:32:38.854 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.855 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:32:38.856 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:32:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:32:38.858 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:32:38.859 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.860 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:32:38.861 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:32:38.862 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:32:38.863 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:32:38.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:32:38.863 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-04-25T12:32:39.249 DEBUG:teuthology.parallel:result is None 2024-04-25T12:32:39.339 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T12:32:39.407 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T12:32:39.407 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:39.407 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T12:32:39.407 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:32:39.407 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T12:32:39.408 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:39.573 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T12:32:39.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T12:32:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T12:32:39.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T12:32:40.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T12:32:40.155 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T12:32:40.272 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:32:40.463 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:32:40.511 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T12:32:40.551 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T12:32:40.583 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T12:32:41.242 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T12:32:41.278 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T12:32:41.330 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T12:32:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T12:32:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T12:32:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T12:32:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T12:32:43.896 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T12:32:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T12:32:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T12:32:44.271 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T12:32:44.309 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T12:32:44.378 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T12:32:44.401 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:44.401 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T12:32:44.401 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:32:44.401 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T12:32:44.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:44.443 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T12:32:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T12:32:44.618 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T12:32:44.688 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T12:32:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T12:32:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T12:32:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T12:32:45.864 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:45.864 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T12:32:45.864 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:32:45.864 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T12:32:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:45.912 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T12:32:45.983 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T12:32:45.983 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:45.983 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T12:32:45.983 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:46.047 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T12:32:46.118 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T12:32:46.118 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:46.118 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T12:32:46.118 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:32:46.119 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T12:32:46.119 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:46.505 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T12:32:46.584 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T12:32:46.584 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T12:32:46.584 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T12:32:46.585 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:32:46.585 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T12:32:46.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:32:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T12:32:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T12:32:51.441 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T12:32:51.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T12:32:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T12:32:51.905 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T12:32:51.987 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T12:32:52.019 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T12:32:53.105 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:32:53.158 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:32:53.214 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T12:32:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T12:32:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T12:32:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T12:32:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T12:32:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T12:32:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:32:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T12:32:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:32:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:32:55.302 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T12:32:55.433 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:32:55.768 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:32:55.768 INFO:teuthology.orchestra.run.smithi123.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T12:32:55.768 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T12:33:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T12:33:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T12:33:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T12:33:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T12:33:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T12:33:01.639 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T12:33:01.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T12:33:01.641 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T12:33:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T12:33:01.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T12:33:01.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T12:33:01.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T12:33:01.646 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T12:33:01.647 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T12:33:01.648 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T12:33:04.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.151 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.152 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T12:33:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T12:33:04.154 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T12:33:04.155 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T12:33:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T12:33:04.157 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T12:33:04.158 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T12:33:04.159 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T12:33:04.160 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T12:33:04.161 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T12:33:04.162 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T12:33:04.162 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T12:33:04.162 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T12:33:04.162 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:04.162 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T12:33:04.743 DEBUG:teuthology.parallel:result is None 2024-04-25T12:33:04.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T12:33:04.886 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:33:04.943 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T12:33:04.943 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T12:33:04.944 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T12:33:04.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T12:33:05.081 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T12:33:05.107 INFO:teuthology.orchestra.run.smithi153.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T12:33:05.107 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T12:33:05.107 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T12:33:05.109 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T12:33:05.109 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:05.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:33:05.181 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:05.181 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T12:33:05.211 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T12:33:05.212 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:05.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:33:05.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:33:05.400 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:05.400 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T12:33:05.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T12:33:05.497 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T12:33:05.498 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:05.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:33:05.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:33:05.679 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:05.679 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T12:33:05.702 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T12:33:05.767 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T12:33:05.767 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:05.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:33:05.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:33:05.939 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:05.939 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T12:33:05.962 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T12:33:06.027 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T12:33:06.110 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T12:33:06.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-04-25T12:33:06.115 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /var/log/ceph 2024-04-25T12:33:06.184 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T12:33:06.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T12:33:06.188 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T12:33:06.247 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T12:33:06.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T12:33:06.251 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T12:33:06.318 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T12:33:06.318 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T12:33:06.318 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T12:33:06.318 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:06.318 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T12:33:06.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T12:33:06.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T12:33:06.617 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:06.618 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T12:33:06.632 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T12:33:06.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T12:33:06.770 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T12:33:06.770 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T12:33:06.770 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673146', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZq9ZGjFaEDy096n+o5IXK+gnVnfHxqzRTfyqL5jdmWMpctx/3t/ZBr6psedrmqaoYwJVf0dhA4uqxbif9q1VE=', 'smithi153.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+17CMczDPuJ1mO2BaF92RtCZmOljpAgviJyKwmuEL0S81N3k/EC7U6WgWhqcmm8HC4aNMINzLTtgTHA0G6e6I='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'thrashosds': None}, {'workunit': {'clients': {'all': ['fs/misc/trivial_sync.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T12:33:06.771 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T12:33:06.775 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T12:33:06.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T12:33:06.856 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T12:33:06.919 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:06.919 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:33:06.957 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:33:06.957 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 626 Links: 1 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:32:56.214561255 +0000 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:30:40.539276021 +0000 2024-04-25T12:33:07.038 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:30:40.539276021 +0000 2024-04-25T12:33:07.039 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:30:40.539276021 +0000 2024-04-25T12:33:07.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:33:07.141 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:33:07.141 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:33:07.141 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000396369 s, 1.3 MB/s 2024-04-25T12:33:07.144 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:33:07.185 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 644 Links: 1 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:32:56.214561255 +0000 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:30:41.152261329 +0000 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:30:41.152261329 +0000 2024-04-25T12:33:07.268 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:30:41.152261329 +0000 2024-04-25T12:33:07.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:33:07.368 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:33:07.369 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:33:07.369 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000297431 s, 1.7 MB/s 2024-04-25T12:33:07.372 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:33:07.417 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-04-25T12:33:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:33:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:07.496 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 664 Links: 1 2024-04-25T12:33:07.496 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:07.496 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:07.497 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:32:56.215561232 +0000 2024-04-25T12:33:07.497 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:30:41.770246518 +0000 2024-04-25T12:33:07.497 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:30:41.770246518 +0000 2024-04-25T12:33:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:30:41.770246518 +0000 2024-04-25T12:33:07.497 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:33:07.602 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:33:07.602 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:33:07.602 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000319233 s, 1.6 MB/s 2024-04-25T12:33:07.606 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:33:07.657 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 684 Links: 1 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 12:32:56.216561208 +0000 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:30:42.381231875 +0000 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:30:42.381231875 +0000 2024-04-25T12:33:07.700 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:30:42.381231875 +0000 2024-04-25T12:33:07.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:33:07.804 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T12:33:07.804 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T12:33:07.804 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000277296 s, 1.8 MB/s 2024-04-25T12:33:07.807 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:33:07.846 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:07.847 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:07.847 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T12:33:07.864 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T12:33:07.864 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-04-25T12:33:07.920 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T12:33:07.920 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:07.920 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T12:33:07.921 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:07.921 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:07.921 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:32:31.403110982 +0000 2024-04-25T12:33:07.921 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:29:34.264635701 +0000 2024-04-25T12:33:07.921 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:29:34.264635701 +0000 2024-04-25T12:33:07.921 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:29:34.264635701 +0000 2024-04-25T12:33:07.921 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T12:33:07.986 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:33:07.986 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:33:07.986 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000222103 s, 2.3 MB/s 2024-04-25T12:33:07.987 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T12:33:08.045 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-04-25T12:33:08.101 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:32:31.403110982 +0000 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:29:34.601629400 +0000 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:29:34.601629400 +0000 2024-04-25T12:33:08.102 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:29:34.601629400 +0000 2024-04-25T12:33:08.102 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T12:33:08.165 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:33:08.165 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:33:08.165 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000203235 s, 2.5 MB/s 2024-04-25T12:33:08.167 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T12:33:08.223 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-04-25T12:33:08.277 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:32:31.404110964 +0000 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:29:34.959622706 +0000 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:29:34.959622706 +0000 2024-04-25T12:33:08.278 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:29:34.959622706 +0000 2024-04-25T12:33:08.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T12:33:08.340 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:33:08.340 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:33:08.340 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000192611 s, 2.7 MB/s 2024-04-25T12:33:08.341 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T12:33:08.397 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-04-25T12:33:08.452 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T12:33:08.452 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T12:33:08.453 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T12:33:08.453 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:33:08.453 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T12:33:08.453 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-04-25 12:32:31.404110964 +0000 2024-04-25T12:33:08.453 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-04-25 12:29:35.304616256 +0000 2024-04-25T12:33:08.453 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-04-25 12:29:35.304616256 +0000 2024-04-25T12:33:08.453 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-04-25 12:29:35.304616256 +0000 2024-04-25T12:33:08.453 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T12:33:08.515 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-04-25T12:33:08.515 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-04-25T12:33:08.515 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000174982 s, 2.9 MB/s 2024-04-25T12:33:08.516 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T12:33:08.571 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:08.571 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T12:33:08.571 INFO:tasks.ceph:Generating config... 2024-04-25T12:33:08.572 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T12:33:08.572 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-25T12:33:08.572 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T12:33:08.572 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T12:33:08.572 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T12:33:08.573 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T12:33:08.573 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T12:33:08.573 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T12:33:08.574 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T12:33:08.574 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T12:33:08.574 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-25T12:33:08.574 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T12:33:08.575 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T12:33:08.575 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T12:33:08.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T12:33:08.744 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T12:33:08.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T12:33:08.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T12:33:08.985 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.123'), ('mon.b', '172.21.15.153'), ('mon.c', '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]')] 2024-04-25T12:33:08.985 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.123,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T12:33:08.986 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:08.986 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T12:33:09.023 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.123 --add b 172.21.15.153 --addv c '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:09.213 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: generated fsid 2239057a-dc25-4ee3-8341-8c80e403f296 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = quincy 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 0 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:fsid 2239057a-dc25-4ee3-8341-8c80e403f296 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:last_changed 2024-04-25T12:33:09.214530+0000 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-04-25T12:33:09.214530+0000 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:min_mon_release 17 (quincy) 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:election_strategy: 1 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:2: [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] mon.c 2024-04-25T12:33:09.214 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T12:33:09.223 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T12:33:09.257 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2239057a-dc25-4ee3-8341-8c80e403f296... 2024-04-25T12:33:09.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T12:33:09.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2024-04-25T12:33:09.345 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2024-04-25T12:33:09.346 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2024-04-25T12:33:09.347 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.123,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on skipped message = False 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = 2239057a-dc25-4ee3-8341-8c80e403f296 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.348 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug trim objects = true 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.349 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore compression mode = aggressive 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 20 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 20 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2024-04-25T12:33:09.350 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 10 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: osd blocked scrub grace period = 3600 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: osd max markdown count = 1000 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2024-04-25T12:33:09.351 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2024-04-25T12:33:09.352 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2024-04-25T12:33:09.353 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: mon down mkfs grace = 300 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: client_shutdown_timeout = 120 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2024-04-25T12:33:09.354 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.e] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.f] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2024-04-25T12:33:09.355 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds balancer = 20 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds bal fragment size max = 10000 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds bal merge size = 5 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds bal split bits = 3 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds bal split size = 100 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds oft prefetch dirfrags = True 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds_dir_prefetch = True 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T12:33:09.356 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2024-04-25T12:33:09.357 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2024-04-25T12:33:09.357 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2024-04-25T12:33:09.402 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-04-25T12:33:09.402 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-04-25T12:33:09.402 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-04-25T12:33:09.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-04-25T12:33:09.404 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-04-25T12:33:09.405 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.123,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on skipped message = False 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 2239057a-dc25-4ee3-8341-8c80e403f296 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.406 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-04-25T12:33:09.407 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore compression mode = aggressive 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2024-04-25T12:33:09.408 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: osd blocked scrub grace period = 3600 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: osd max markdown count = 1000 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T12:33:09.409 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-04-25T12:33:09.410 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T12:33:09.411 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: mon down mkfs grace = 300 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2024-04-25T12:33:09.412 INFO:teuthology.orchestra.run.smithi123.stdout: client_shutdown_timeout = 120 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.e] 2024-04-25T12:33:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.f] 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal fragment size max = 10000 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal merge size = 5 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal split bits = 3 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal split size = 100 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds oft prefetch dirfrags = True 2024-04-25T12:33:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2024-04-25T12:33:09.415 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2024-04-25T12:33:09.415 INFO:teuthology.orchestra.run.smithi123.stdout: mds_dir_prefetch = True 2024-04-25T12:33:09.415 INFO:teuthology.orchestra.run.smithi123.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T12:33:09.415 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-25T12:33:09.415 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-25T12:33:09.415 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-25T12:33:09.431 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T12:33:09.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T12:33:09.612 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T12:33:09.613 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:09.613 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T12:33:09.649 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:09.649 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T12:33:09.729 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:33:09.729 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:09.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T12:33:09.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T12:33:09.851 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:09.851 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:09.931 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:33:09.931 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:09.931 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T12:33:09.931 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T12:33:09.969 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:09.969 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:10.026 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T12:33:10.027 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T12:33:10.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T12:33:10.207 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T12:33:10.223 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T12:33:10.295 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T12:33:10.299 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T12:33:10.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T12:33:10.495 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T12:33:10.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T12:33:10.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T12:33:10.752 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T12:33:10.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T12:33:10.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T12:33:11.003 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T12:33:11.015 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-25T12:33:11.078 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T12:33:11.146 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T12:33:11.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T12:33:11.174 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T12:33:11.275 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T12:33:11.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T12:33:11.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T12:33:11.399 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T12:33:11.402 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-25T12:33:11.425 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T12:33:11.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T12:33:11.592 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T12:33:11.629 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T12:33:11.629 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T12:33:11.629 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T12:33:11.698 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:11.699 INFO:tasks.ceph:role: osd.0 2024-04-25T12:33:11.699 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:33:11.699 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T12:33:11.781 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:11.781 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:11.781 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:11.781 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:11.781 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:11.782 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:11.782 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:11.782 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:11.782 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:11.782 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:12.188 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T12:33:12.194 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T12:33:12.194 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T12:33:12.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T12:33:12.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T12:33:12.497 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:12.497 INFO:tasks.ceph:role: osd.1 2024-04-25T12:33:12.497 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:33:12.497 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:12.573 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:12.953 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T12:33:12.958 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T12:33:12.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T12:33:13.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T12:33:13.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T12:33:13.195 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:13.195 INFO:tasks.ceph:role: osd.2 2024-04-25T12:33:13.195 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:33:13.195 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T12:33:13.262 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:13.262 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:13.262 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:13.262 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:13.262 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:13.262 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:13.262 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:13.263 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:13.263 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:13.263 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:13.659 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T12:33:13.665 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T12:33:13.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T12:33:13.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T12:33:13.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T12:33:13.951 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:13.951 INFO:tasks.ceph:role: osd.3 2024-04-25T12:33:13.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:33:13.952 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T12:33:14.027 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:14.027 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:14.027 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:14.027 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:14.027 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:14.028 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:14.028 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:14.028 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:14.028 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:14.028 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:14.441 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T12:33:14.448 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T12:33:14.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T12:33:14.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T12:33:14.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:14.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:14.918+0000 7f6f3e213640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T12:33:14.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:14.918+0000 7f6f3e213640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T12:33:14.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:14.918+0000 7f6f3e213640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T12:33:14.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:14.918+0000 7f6f3e213640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T12:33:14.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:14.918+0000 7f6f3e213640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T12:33:14.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:14.922+0000 7f6f3e213640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T12:33:18.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T12:33:18.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:18.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:18.423+0000 7f761ccf0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T12:33:18.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:18.424+0000 7f761ccf0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T12:33:18.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:18.424+0000 7f761ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T12:33:18.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:18.424+0000 7f761ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T12:33:18.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:18.424+0000 7f761ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T12:33:18.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:18.426+0000 7f761ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T12:33:21.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T12:33:21.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:21.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:21.921+0000 7f7ebe91f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T12:33:21.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:21.921+0000 7f7ebe91f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T12:33:21.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:21.921+0000 7f7ebe91f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T12:33:21.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:21.921+0000 7f7ebe91f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T12:33:21.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:21.921+0000 7f7ebe91f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T12:33:21.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:21.924+0000 7f7ebe91f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T12:33:24.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T12:33:24.978 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:25.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:25.172+0000 7fa2935da640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T12:33:25.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:25.173+0000 7fa2935da640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T12:33:25.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:25.173+0000 7fa2935da640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T12:33:25.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:25.173+0000 7fa2935da640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T12:33:25.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:25.173+0000 7fa2935da640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T12:33:25.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:25.176+0000 7fa2935da640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T12:33:28.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T12:33:28.485 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T12:33:28.515 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:28.515 INFO:tasks.ceph:role: osd.4 2024-04-25T12:33:28.515 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:33:28.515 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T12:33:28.589 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:28.590 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:28.590 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:28.590 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:28.590 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:29.109 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-04-25T12:33:29.111 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-04-25T12:33:29.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T12:33:29.236 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T12:33:29.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T12:33:29.338 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:29.339 INFO:tasks.ceph:role: osd.5 2024-04-25T12:33:29.339 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:33:29.339 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T12:33:29.413 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:29.413 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:29.413 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:29.413 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:29.414 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:29.414 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:29.414 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:29.414 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:29.414 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:29.414 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:29.898 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-04-25T12:33:29.900 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-04-25T12:33:29.900 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T12:33:29.995 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T12:33:30.022 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T12:33:30.087 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:30.087 INFO:tasks.ceph:role: osd.6 2024-04-25T12:33:30.087 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:33:30.087 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T12:33:30.157 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:30.157 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:30.157 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:30.158 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:30.158 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:30.158 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:30.158 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:30.158 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:30.158 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:30.158 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:30.668 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-04-25T12:33:30.670 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-04-25T12:33:30.670 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T12:33:30.799 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T12:33:30.826 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T12:33:30.889 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T12:33:30.889 INFO:tasks.ceph:role: osd.7 2024-04-25T12:33:30.889 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:33:30.890 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T12:33:30.961 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T12:33:31.481 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-04-25T12:33:31.483 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-04-25T12:33:31.483 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T12:33:31.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T12:33:31.647 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:31.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:31.745+0000 7fbcd3533640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T12:33:31.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:31.745+0000 7fbcd3533640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T12:33:31.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:31.745+0000 7fbcd3533640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T12:33:31.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:31.745+0000 7fbcd3533640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T12:33:31.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:31.745+0000 7fbcd3533640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T12:33:31.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:31.746+0000 7fbcd3533640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T12:33:34.591 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T12:33:34.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:34.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:34.720+0000 7ff016c9d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T12:33:34.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:34.720+0000 7ff016c9d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T12:33:34.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:34.720+0000 7ff016c9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T12:33:34.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:34.720+0000 7ff016c9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T12:33:34.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:34.720+0000 7ff016c9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T12:33:34.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:34.721+0000 7ff016c9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T12:33:36.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:33:36.781 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:33:37.572 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T12:33:37.596 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:37.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:37.690+0000 7ff8b2a9d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T12:33:37.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:37.690+0000 7ff8b2a9d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T12:33:37.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:37.690+0000 7ff8b2a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T12:33:37.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:37.690+0000 7ff8b2a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T12:33:37.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:37.690+0000 7ff8b2a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T12:33:37.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:37.691+0000 7ff8b2a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T12:33:40.552 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T12:33:40.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:40.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:40.684+0000 7fca9762c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T12:33:40.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:40.684+0000 7fca9762c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T12:33:40.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:40.684+0000 7fca9762c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T12:33:40.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:40.684+0000 7fca9762c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T12:33:40.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:40.684+0000 7fca9762c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T12:33:40.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:40.684+0000 7fca9762c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T12:33:43.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T12:33:43.571 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T12:33:43.571 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:43.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T12:33:43.663 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:43.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T12:33:43.746 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:43.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T12:33:43.825 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:43.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-25T12:33:43.954 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:43.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T12:33:44.039 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:44.040 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T12:33:44.112 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:44.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T12:33:44.192 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:44.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T12:33:44.322 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.322 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T12:33:44.350 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.350 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T12:33:44.417 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.418 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T12:33:44.483 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.484 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-25T12:33:44.549 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T12:33:44.613 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.613 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T12:33:44.677 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.677 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T12:33:44.740 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:44.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T12:33:44.804 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:44.804 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T12:33:44.861 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T12:33:44.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T12:33:44.866 DEBUG:teuthology.orchestra.run.smithi153:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.y] 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCGTSpm3N9oDBAAa6WRrRu+CIZL9xR1ndfKPA== 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCGTSpmPR2eHRAADaTWaQiYrsEIRYwFnweFwA== 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCGTSpmw7/jLBAA03RakjBucWqUMqCWYL+8sg== 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.e] 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCHTSpmB8xIABAAaz99RR81b5IQopLUicUZvg== 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.0] 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCKTSpmVb6/NhAAtZg8S6IT+lBqXhDeI56LYA== 2024-04-25T12:33:44.888 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.1] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCOTSpmK/RJGRAAkp2XdW0P2/tzWdhjHf7TAw== 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.2] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCRTSpmBg3+NhAA9e8brw9N8+ReNP/su27OXQ== 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.3] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCVTSpmW0ZkChAAHNcOsYylA74bn5mGDgmTmA== 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.x] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCGTSpmflGzERAAZ72PHpvZ8MDz7q3U2m9JDA== 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCHTSpmSfTJCBAAMK4Erg38qzXYJ312UPPcDg== 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCHTSpm60F8EBAAwGAmz6mXtfw3Z1LL1OyB4Q== 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.f] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCHTSpmpbLZFxAAptQsREytIvddMifnrN3bpQ== 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.4] 2024-04-25T12:33:44.889 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCbTSpmnX52LBAAOmpLd3p5oDshxYONP8DPMw== 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.5] 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCeTSpmin8KKxAA8YXC4HIGdbaUmMsDU5yQFw== 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.6] 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQChTSpmwPVCKRAAY52pLpxNBdLkUOpv1/9YwQ== 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.7] 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCkTSpmKEPiKBAAVfqllJsV/SOqx43Z3NL4/Q== 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2024-04-25T12:33:44.890 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCHTSpmb2JbIxAAD+hEm4xDeNYMGzhfieUxVw== 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGTSpm3N9oDBAAa6WRrRu+CIZL9xR1ndfKPA== 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGTSpmPR2eHRAADaTWaQiYrsEIRYwFnweFwA== 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGTSpmw7/jLBAA03RakjBucWqUMqCWYL+8sg== 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.e] 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHTSpmB8xIABAAaz99RR81b5IQopLUicUZvg== 2024-04-25T12:33:44.938 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCKTSpmVb6/NhAAtZg8S6IT+lBqXhDeI56LYA== 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCOTSpmK/RJGRAAkp2XdW0P2/tzWdhjHf7TAw== 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCRTSpmBg3+NhAA9e8brw9N8+ReNP/su27OXQ== 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCVTSpmW0ZkChAAHNcOsYylA74bn5mGDgmTmA== 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGTSpmflGzERAAZ72PHpvZ8MDz7q3U2m9JDA== 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHTSpmSfTJCBAAMK4Erg38qzXYJ312UPPcDg== 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-25T12:33:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHTSpm60F8EBAAwGAmz6mXtfw3Z1LL1OyB4Q== 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.f] 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHTSpmpbLZFxAAptQsREytIvddMifnrN3bpQ== 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCbTSpmnX52LBAAOmpLd3p5oDshxYONP8DPMw== 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCeTSpmin8KKxAA8YXC4HIGdbaUmMsDU5yQFw== 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQChTSpmwPVCKRAAY52pLpxNBdLkUOpv1/9YwQ== 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCkTSpmKEPiKBAAVfqllJsV/SOqx43Z3NL4/Q== 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-25T12:33:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHTSpmb2JbIxAAD+hEm4xDeNYMGzhfieUxVw== 2024-04-25T12:33:44.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T12:33:44.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T12:33:45.194 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:45.201 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:45.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:45.411 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:45.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:45.622 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:45.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:45.831 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:46.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:46.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:46.235 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:46.242 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:46.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:46.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:46.640 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T12:33:46.645 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T12:33:46.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:46.846 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:47.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:47.053 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:47.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:47.260 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T12:33:47.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:47.469 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:47.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:47.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:47.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:47.882 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:48.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:48.096 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T12:33:48.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T12:33:48.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T12:33:48.504 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T12:33:48.504 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T12:33:48.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T12:33:48.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T12:33:48.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T12:33:48.918 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T12:33:49.027 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T12:33:49.050 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T12:33:49.114 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T12:33:49.217 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T12:33:49.240 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:49.248 DEBUG:teuthology.orchestra.run.smithi153:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T12:33:49.297 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T12:33:49.297 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T12:33:49.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T12:33:49.304 INFO:tasks.ceph.mon.a:Started 2024-04-25T12:33:49.304 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T12:33:49.304 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T12:33:49.337 INFO:tasks.ceph.mon.b:Started 2024-04-25T12:33:49.337 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T12:33:49.337 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T12:33:49.339 INFO:tasks.ceph.mon.c:Started 2024-04-25T12:33:49.339 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T12:33:49.339 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T12:33:49.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T12:33:49.344 INFO:tasks.ceph.mgr.y:Started 2024-04-25T12:33:49.344 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T12:33:49.344 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T12:33:49.346 INFO:tasks.ceph.mgr.x:Started 2024-04-25T12:33:49.346 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T12:33:49.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T12:33:49.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.587+0000 7f5267600640 1 Processor -- start 2024-04-25T12:33:49.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.588+0000 7f5267600640 1 -- start start 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.589+0000 7f5267600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5267600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52600c54b0 con 0x7f52600c3880 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5267600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52600c5750 con 0x7f5260070620 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5267600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f52600c5a30 con 0x7f52600c3c90 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5264c00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5260070620 0x7f5260156a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:57154/0 (socket says 172.21.15.123:57154) 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 learned_addr learned my addr 172.21.15.123:0/2947474842 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:49.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.590+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:33:49.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.791+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:49.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.792+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T12:33:49.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.792+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:49.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.792+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:49.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:49.792+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:33:50.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.193+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:50.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.194+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T12:33:50.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.194+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:50.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.194+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:50.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.194+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:33:50.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.995+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.996+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T12:33:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.996+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.996+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:50.996+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T12:33:52.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 msgr2=0x7f526015c830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:33:52.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f52600c3c90 0x7f526015c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:52.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 >> v1:172.21.15.153:6789/0 conn(0x7f5260070620 legacy=0x7f5260156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:52.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 >> v1:172.21.15.123:6789/0 conn(0x7f52600c3880 legacy=0x7f52600c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:52.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:52.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:52.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f52600c5a30 con 0x7f5240006220 2024-04-25T12:33:52.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5240008dd0 con 0x7f5240003710 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.590+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:33:52.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.591+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:33:52.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:52.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:52.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T12:33:52.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:52.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T12:33:52.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:52.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:52.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:52.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.792+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:33:52.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:52.793+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:53.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:53.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:33:53.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.194+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f525fe00640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T12:33:53.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f5264c00640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T12:33:53.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T12:33:53.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:53.996+0000 7f5264c00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T12:33:54.747 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:54.748+0000 7fcf3d321040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:33:54.904 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:54.904+0000 7fcf3d321040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:33:54.956 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:54.956+0000 7fcf3d321040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:33:55.194 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:55.194+0000 7fcf3d321040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:33:55.238 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:55.238+0000 7fcf3d321040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:33:55.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 msgr2=0x7f5240005bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:33:55.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f525e000640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003710 0x7f5240005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 msgr2=0x7f5240008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:33:55.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f525e000640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.153:3300/0 conn(0x7f5240006220 0x7f5240008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 reap_dead start 2024-04-25T12:33:55.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f525e000640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003ba0 0x7f5240006650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:55.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f525e000640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f5240006b90 0x7f5240007050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.590+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52600c9390 con 0x7f5260070620 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.591+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f52600c5a30 con 0x7f5240006b90 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.591+0000 7f525e000640 1 -- 172.21.15.123:0/2947474842 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5240008dd0 con 0x7f5240003ba0 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.591+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> v2:172.21.15.123:3300/0 conn(0x7f5240006b90 0x7f5240007050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.591+0000 7f525f400640 1 -- 172.21.15.123:0/2947474842 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1597952916 0 0) 0x7f52600c9390 con 0x7f5260070620 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.591+0000 7f525f400640 1 -- 172.21.15.123:0/2947474842 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5248003520 con 0x7f5260070620 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.591+0000 7f525fe00640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003ba0 0x7f5240006650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.591+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003ba0 msgr2=0x7f5240006650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.592+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003ba0 0x7f5240006650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.592+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 >> v1:172.21.15.153:6789/0 conn(0x7f5260070620 legacy=0x7f524000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.592+0000 7f5265600640 1 -- 172.21.15.123:0/2947474842 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52400088a0 con 0x7f5240006b90 2024-04-25T12:33:55.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.593+0000 7f5265600640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f5240007050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52600c5150 tx=0x7f525802af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0d83581a1fa828 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:55.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.593+0000 7f525f400640 1 -- 172.21.15.123:0/2947474842 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258031070 con 0x7f5240006b90 2024-04-25T12:33:55.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.593+0000 7f525f400640 1 -- 172.21.15.123:0/2947474842 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525802ba50 con 0x7f5240006b90 2024-04-25T12:33:55.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.593+0000 7f525f400640 1 -- 172.21.15.123:0/2947474842 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525802be20 con 0x7f5240006b90 2024-04-25T12:33:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.594+0000 7f5267600640 1 -- 172.21.15.123:0/2947474842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 msgr2=0x7f5240007050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.594+0000 7f5267600640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f5240007050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52600c5150 tx=0x7f525802af40 comp rx=0 tx=0).stop 2024-04-25T12:33:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.595+0000 7f5267600640 1 -- 172.21.15.123:0/2947474842 shutdown_connections 2024-04-25T12:33:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.595+0000 7f5267600640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f5240007050 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.595+0000 7f5267600640 1 --2- 172.21.15.123:0/2947474842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5240003ba0 0x7f5240006650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.595+0000 7f5267600640 1 -- 172.21.15.123:0/2947474842 >> 172.21.15.123:0/2947474842 conn(0x7f5260071b90 msgr2=0x7f5260071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:55.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.595+0000 7f5267600640 1 -- 172.21.15.123:0/2947474842 shutdown_connections 2024-04-25T12:33:55.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.595+0000 7f5267600640 1 -- 172.21.15.123:0/2947474842 wait complete. 2024-04-25T12:33:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.598+0000 7f5267600640 1 Processor -- start 2024-04-25T12:33:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.599+0000 7f5267600640 1 -- start start 2024-04-25T12:33:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.600+0000 7f5267600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5240003ba0 0x7f52601f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.600+0000 7f5267600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f52601f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.600+0000 7f5267600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5260070620 0x7f5260150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.600+0000 7f5267600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5260070e90 con 0x7f5240006b90 2024-04-25T12:33:55.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.600+0000 7f5267600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5260070b90 con 0x7f5240003ba0 2024-04-25T12:33:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.600+0000 7f5267600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5260071010 con 0x7f5260070620 2024-04-25T12:33:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f525fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f52601f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5264c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5240003ba0 0x7f52601f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5265600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5260070620 0x7f5260150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f525fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f52601f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54376/0 (socket says 172.21.15.123:54376) 2024-04-25T12:33:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5264c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5240003ba0 0x7f52601f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41644/0 (socket says 172.21.15.123:41644) 2024-04-25T12:33:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f525fe00640 1 -- 172.21.15.123:0/680934471 learned_addr learned my addr 172.21.15.123:0/680934471 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:33:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5264c00640 1 -- 172.21.15.123:0/680934471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5260070620 msgr2=0x7f5260150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5264c00640 1 --2- 172.21.15.123:0/680934471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5260070620 0x7f5260150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5264c00640 1 -- 172.21.15.123:0/680934471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 msgr2=0x7f52601f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5264c00640 1 --2- 172.21.15.123:0/680934471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f52601f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.601+0000 7f5264c00640 1 -- 172.21.15.123:0/680934471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5260157880 con 0x7f5240003ba0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f5264c00640 1 --2- 172.21.15.123:0/680934471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5240003ba0 0x7f52601f2f10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5254007910 tx=0x7f5254001630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f525cc00640 1 -- 172.21.15.123:0/680934471 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525401b020 con 0x7f5240003ba0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f525cc00640 1 -- 172.21.15.123:0/680934471 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5254007940 con 0x7f5240003ba0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f525cc00640 1 -- 172.21.15.123:0/680934471 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540178b0 con 0x7f5240003ba0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52601f2430 con 0x7f5240003ba0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52601f29f0 con 0x7f5240003ba0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f525cc00640 1 -- 172.21.15.123:0/680934471 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f525401e020 con 0x7f5240003ba0 2024-04-25T12:33:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.602+0000 7f525cc00640 1 -- 172.21.15.123:0/680934471 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5254016410 con 0x7f5240003ba0 2024-04-25T12:33:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.604+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5260076700 con 0x7f5240003ba0 2024-04-25T12:33:55.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.607+0000 7f525cc00640 1 -- 172.21.15.123:0/680934471 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5254025030 con 0x7f5240003ba0 2024-04-25T12:33:55.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.682+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5260071210 con 0x7f5240003ba0 2024-04-25T12:33:55.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.689+0000 7f525cc00640 1 -- 172.21.15.123:0/680934471 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f5254016660 con 0x7f5240003ba0 2024-04-25T12:33:55.689 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2024-04-25T12:33:55.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5240003ba0 msgr2=0x7f52601f2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:55.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 --2- 172.21.15.123:0/680934471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5240003ba0 0x7f52601f2f10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5254007910 tx=0x7f5254001630 comp rx=0 tx=0).stop 2024-04-25T12:33:55.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 shutdown_connections 2024-04-25T12:33:55.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 --2- 172.21.15.123:0/680934471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5260070620 0x7f5260150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 --2- 172.21.15.123:0/680934471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240006b90 0x7f52601f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 --2- 172.21.15.123:0/680934471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5240003ba0 0x7f52601f2f10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:55.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 >> 172.21.15.123:0/680934471 conn(0x7f5260071b90 msgr2=0x7f5260156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:55.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 shutdown_connections 2024-04-25T12:33:55.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:55.691+0000 7f5267600640 1 -- 172.21.15.123:0/680934471 wait complete. 2024-04-25T12:33:55.717 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T12:33:55.717 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T12:33:55.718 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:55.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T12:33:55.725 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:55.725+0000 7fa6d9037040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:33:55.788 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:55.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T12:33:55.903 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:55.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T12:33:55.913 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:55.914+0000 7fa6d9037040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:33:55.939 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:55.939+0000 7fcf3d321040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:33:55.981 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:55.982+0000 7fa6d9037040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:33:55.987 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:33:55.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T12:33:56.027 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:56.027+0000 7fcf3d321040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:33:56.250 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:56.069+0000 7fcf3d321040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:33:56.250 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:56.231+0000 7fcf3d321040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:33:56.278 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:56.278+0000 7fcf3d321040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:33:56.293 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:56.294+0000 7fa6d9037040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:33:56.324 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:56.324 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T12:33:56.330 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:56.330+0000 7fcf3d321040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:33:56.349 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:56.350 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T12:33:56.350 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:56.351+0000 7fa6d9037040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:33:56.413 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:56.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T12:33:56.448 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:56.448+0000 7fcf3d321040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:33:56.471 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-04-25T12:33:56.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T12:33:56.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 2d9ee216-abfa-45db-a0ac-ef834981be6f 0 2024-04-25T12:33:56.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.687+0000 7f6d09696640 1 Processor -- start 2024-04-25T12:33:56.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.688+0000 7f6d09696640 1 -- start start 2024-04-25T12:33:56.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d09696640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6d041607d0 0x7f6d04070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d09696640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156940 0x7f6d040c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d09696640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d040c54d0 con 0x7f6d04156d50 2024-04-25T12:33:56.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d09696640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6d040c57b0 con 0x7f6d041607d0 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d09696640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6d04071010 con 0x7f6d04156940 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d027fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156940 0x7f6d040c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d027fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156940 0x7f6d040c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36816/0 (socket says 172.21.15.153:36816) 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d027fc640 1 -- 172.21.15.153:0/845086788 learned_addr learned my addr 172.21.15.153:0/845086788 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d027fc640 1 -- 172.21.15.153:0/845086788 >> v2:172.21.15.123:3300/0 conn(0x7f6d041607d0 msgr2=0x7f6d04070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d027fc640 1 --2- 172.21.15.153:0/845086788 >> v2:172.21.15.123:3300/0 conn(0x7f6d041607d0 0x7f6d04070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d027fc640 1 -- 172.21.15.153:0/845086788 >> v1:172.21.15.123:6789/0 conn(0x7f6d04156d50 legacy=0x7f6d0415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:56.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.689+0000 7f6d027fc640 1 -- 172.21.15.153:0/845086788 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d040712f0 con 0x7f6d04156940 2024-04-25T12:33:56.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.690+0000 7f6d027fc640 1 --2- 172.21.15.153:0/845086788 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156940 0x7f6d040c8450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d07b42915702cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:56.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.690+0000 7f6d01ffb640 1 -- 172.21.15.153:0/845086788 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4037070 con 0x7f6d04156940 2024-04-25T12:33:56.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.690+0000 7f6d01ffb640 1 -- 172.21.15.153:0/845086788 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf4032440 con 0x7f6d04156940 2024-04-25T12:33:56.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.690+0000 7f6d01ffb640 1 -- 172.21.15.153:0/845086788 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4032810 con 0x7f6d04156940 2024-04-25T12:33:56.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 -- 172.21.15.153:0/845086788 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156940 msgr2=0x7f6d040c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 --2- 172.21.15.153:0/845086788 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156940 0x7f6d040c8450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf402a210 comp rx=0 tx=0).stop 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 -- 172.21.15.153:0/845086788 shutdown_connections 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 --2- 172.21.15.153:0/845086788 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156940 0x7f6d040c8450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 --2- 172.21.15.153:0/845086788 >> v2:172.21.15.123:3300/0 conn(0x7f6d041607d0 0x7f6d04070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 -- 172.21.15.153:0/845086788 >> 172.21.15.153:0/845086788 conn(0x7f6d04071b90 msgr2=0x7f6d04071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 -- 172.21.15.153:0/845086788 shutdown_connections 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.691+0000 7f6d09696640 1 -- 172.21.15.153:0/845086788 wait complete. 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.693+0000 7f6d09696640 1 Processor -- start 2024-04-25T12:33:56.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.693+0000 7f6d09696640 1 -- start start 2024-04-25T12:33:56.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d09696640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d04156940 0x7f6d041f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d09696640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156d50 0x7f6d04150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d09696640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d041607d0 0x7f6d0415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d09696640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0415d390 con 0x7f6d04156940 2024-04-25T12:33:56.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d09696640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04160be0 con 0x7f6d041607d0 2024-04-25T12:33:56.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d09696640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0415d510 con 0x7f6d04156d50 2024-04-25T12:33:56.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d027fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156d50 0x7f6d04150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d037fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d041607d0 0x7f6d0415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d027fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156d50 0x7f6d04150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36830/0 (socket says 172.21.15.153:36830) 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d027fc640 1 -- 172.21.15.153:0/2043285876 learned_addr learned my addr 172.21.15.153:0/2043285876 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d037fe640 1 -- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156d50 msgr2=0x7f6d04150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d037fe640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156d50 0x7f6d04150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d037fe640 1 -- 172.21.15.153:0/2043285876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d04156940 msgr2=0x7f6d041f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d037fe640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d04156940 0x7f6d041f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d037fe640 1 -- 172.21.15.153:0/2043285876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d040ce550 con 0x7f6d041607d0 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.694+0000 7f6d027fc640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156d50 0x7f6d04150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6d037fe640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d041607d0 0x7f6d0415bf90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00be20 tx=0x7f6cfc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6ceb7fe640 1 -- 172.21.15.153:0/2043285876 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc019070 con 0x7f6d041607d0 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6ceb7fe640 1 -- 172.21.15.153:0/2043285876 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00ce10 con 0x7f6d041607d0 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d04158c30 con 0x7f6d041607d0 2024-04-25T12:33:56.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6ceb7fe640 1 -- 172.21.15.153:0/2043285876 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc014680 con 0x7f6d041607d0 2024-04-25T12:33:56.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041591f0 con 0x7f6d041607d0 2024-04-25T12:33:56.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6ceb7fe640 1 -- 172.21.15.153:0/2043285876 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6cfc01c070 con 0x7f6d041607d0 2024-04-25T12:33:56.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.695+0000 7f6ceb7fe640 1 -- 172.21.15.153:0/2043285876 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f6cfc01ebe0 con 0x7f6d041607d0 2024-04-25T12:33:56.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.696+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d04076700 con 0x7f6d041607d0 2024-04-25T12:33:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.698+0000 7f6ceb7fe640 1 -- 172.21.15.153:0/2043285876 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6cfc023020 con 0x7f6d041607d0 2024-04-25T12:33:56.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.771+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2d9ee216-abfa-45db-a0ac-ef834981be6f", "id": 0} v 0) -- 0x7f6d04071fa0 con 0x7f6d041607d0 2024-04-25T12:33:56.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.778+0000 7f6ceb7fe640 1 -- 172.21.15.153:0/2043285876 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2d9ee216-abfa-45db-a0ac-ef834981be6f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f6cfc01e5d0 con 0x7f6d041607d0 2024-04-25T12:33:56.778 INFO:teuthology.orchestra.run.smithi153.stdout:0 2024-04-25T12:33:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d041607d0 msgr2=0x7f6d0415bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d041607d0 0x7f6d0415bf90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00be20 tx=0x7f6cfc00bf10 comp rx=0 tx=0).stop 2024-04-25T12:33:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 shutdown_connections 2024-04-25T12:33:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d041607d0 0x7f6d0415bf90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6d04156d50 0x7f6d04150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 --2- 172.21.15.153:0/2043285876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d04156940 0x7f6d041f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 >> 172.21.15.153:0/2043285876 conn(0x7f6d04071b90 msgr2=0x7f6d041e5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 shutdown_connections 2024-04-25T12:33:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.779+0000 7f6d09696640 1 -- 172.21.15.153:0/2043285876 wait complete. 2024-04-25T12:33:56.790 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new a60a3a6f-376a-4fb2-baf1-e13fc615f6e5 1 2024-04-25T12:33:56.851 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:56.851+0000 7fcf3d321040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:33:56.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.899+0000 7f8bcf870640 1 Processor -- start 2024-04-25T12:33:56.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.899+0000 7f8bcf870640 1 -- start start 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcf870640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8bc806c570 0x7f8bc80b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcf870640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc806db20 con 0x7f8bc815a600 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcf870640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc80c9ee0 con 0x7f8bc8154400 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcf870640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc80c8b00 con 0x7f8bc806c570 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcd60c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8bc806c570 0x7f8bc80b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcce0b640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8bc8154400 0x7f8bc80c7630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:42910/0 (socket says 172.21.15.153:42910) 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcce0b640 1 -- 172.21.15.153:0/3762219380 learned_addr learned my addr 172.21.15.153:0/3762219380 (peer_addr_for_me v1:172.21.15.153:0/0) 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcd60c640 1 -- 172.21.15.153:0/3762219380 >> v1:172.21.15.153:6789/0 conn(0x7f8bc8154400 legacy=0x7f8bc80c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcd60c640 1 -- 172.21.15.153:0/3762219380 >> v1:172.21.15.123:6789/0 conn(0x7f8bc815a600 legacy=0x7f8bc815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:56.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.901+0000 7f8bcd60c640 1 -- 172.21.15.153:0/3762219380 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc80c8de0 con 0x7f8bc806c570 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.902+0000 7f8bcd60c640 1 --2- 172.21.15.153:0/3762219380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80b9ca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80ca800 tx=0x7f8bb802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e16d43da16275c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.902+0000 7f8bbffff640 1 -- 172.21.15.153:0/3762219380 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8037070 con 0x7f8bc806c570 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.902+0000 7f8bbffff640 1 -- 172.21.15.153:0/3762219380 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb802ae20 con 0x7f8bc806c570 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.902+0000 7f8bbffff640 1 -- 172.21.15.153:0/3762219380 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8032660 con 0x7f8bc806c570 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.903+0000 7f8bcf870640 1 -- 172.21.15.153:0/3762219380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 msgr2=0x7f8bc80b9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.903+0000 7f8bcf870640 1 --2- 172.21.15.153:0/3762219380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80b9ca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80ca800 tx=0x7f8bb802a110 comp rx=0 tx=0).stop 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.903+0000 7f8bcf870640 1 -- 172.21.15.153:0/3762219380 shutdown_connections 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.903+0000 7f8bcf870640 1 --2- 172.21.15.153:0/3762219380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80b9ca0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.903+0000 7f8bcf870640 1 -- 172.21.15.153:0/3762219380 >> 172.21.15.153:0/3762219380 conn(0x7f8bc8071f10 msgr2=0x7f8bc80caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.903+0000 7f8bcf870640 1 -- 172.21.15.153:0/3762219380 shutdown_connections 2024-04-25T12:33:56.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.903+0000 7f8bcf870640 1 -- 172.21.15.153:0/3762219380 wait complete. 2024-04-25T12:33:56.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.905+0000 7f8bcf870640 1 Processor -- start 2024-04-25T12:33:56.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.905+0000 7f8bcf870640 1 -- start start 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcf870640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcf870640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc8154400 0x7f8bc81e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcf870640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc815a600 0x7f8bc80c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcf870640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8154990 con 0x7f8bc815a600 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcf870640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8154810 con 0x7f8bc806c570 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcf870640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc806caf0 con 0x7f8bc8154400 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcd60c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcd60c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51986/0 (socket says 172.21.15.153:51986) 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcd60c640 1 -- 172.21.15.153:0/1510997400 learned_addr learned my addr 172.21.15.153:0/1510997400 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:56.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcce0b640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc8154400 0x7f8bc81e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcce0b640 1 -- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 msgr2=0x7f8bc80c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcce0b640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcce0b640 1 -- 172.21.15.153:0/1510997400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc815a600 msgr2=0x7f8bc80c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcce0b640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc815a600 0x7f8bc80c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcce0b640 1 -- 172.21.15.153:0/1510997400 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81e5490 con 0x7f8bc8154400 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcd60c640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.906+0000 7f8bcce0b640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc8154400 0x7f8bc81e4f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4000f60 tx=0x7f8bc400c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.907+0000 7f8bbdffb640 1 -- 172.21.15.153:0/1510997400 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4017070 con 0x7f8bc8154400 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.907+0000 7f8bbdffb640 1 -- 172.21.15.153:0/1510997400 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc4005ce0 con 0x7f8bc8154400 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.907+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc80cc840 con 0x7f8bc8154400 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.907+0000 7f8bbdffb640 1 -- 172.21.15.153:0/1510997400 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4007310 con 0x7f8bc8154400 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.907+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc80cd7f0 con 0x7f8bc8154400 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.907+0000 7f8bbdffb640 1 -- 172.21.15.153:0/1510997400 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8bc401a070 con 0x7f8bc8154400 2024-04-25T12:33:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.907+0000 7f8bbdffb640 1 -- 172.21.15.153:0/1510997400 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8bc401c7f0 con 0x7f8bc8154400 2024-04-25T12:33:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.908+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc8076700 con 0x7f8bc8154400 2024-04-25T12:33:56.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.912+0000 7f8bbdffb640 1 -- 172.21.15.153:0/1510997400 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8bc4012de0 con 0x7f8bc8154400 2024-04-25T12:33:56.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.985+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a60a3a6f-376a-4fb2-baf1-e13fc615f6e5", "id": 1} v 0) -- 0x7f8bc806ce70 con 0x7f8bc8154400 2024-04-25T12:33:56.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.993+0000 7f8bbdffb640 1 -- 172.21.15.153:0/1510997400 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a60a3a6f-376a-4fb2-baf1-e13fc615f6e5", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8bc40074b0 con 0x7f8bc8154400 2024-04-25T12:33:56.994 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.994+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc8154400 msgr2=0x7f8bc81e4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.994+0000 7f8bcf870640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc8154400 0x7f8bc81e4f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4000f60 tx=0x7f8bc400c760 comp rx=0 tx=0).stop 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.994+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 shutdown_connections 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.994+0000 7f8bcf870640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc815a600 0x7f8bc80c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.994+0000 7f8bcf870640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8bc8154400 0x7f8bc81e4f50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.994+0000 7f8bcf870640 1 --2- 172.21.15.153:0/1510997400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc806c570 0x7f8bc80c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.994+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 >> 172.21.15.153:0/1510997400 conn(0x7f8bc8071f10 msgr2=0x7f8bc8159920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.995+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 shutdown_connections 2024-04-25T12:33:56.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:56.995+0000 7f8bcf870640 1 -- 172.21.15.153:0/1510997400 wait complete. 2024-04-25T12:33:57.005 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new cfb7cbbd-82cd-46fe-b89b-15775df1fe93 2 2024-04-25T12:33:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.113+0000 7f6741a30640 1 Processor -- start 2024-04-25T12:33:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.113+0000 7f6741a30640 1 -- start start 2024-04-25T12:33:57.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f6741a30640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f673c0c5a60 0x7f673c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f6741a30640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c15da40 0x7f673c15de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f6741a30640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c0ccd10 con 0x7f673c071a10 2024-04-25T12:33:57.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f6741a30640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f673c0ca770 con 0x7f673c0c5a60 2024-04-25T12:33:57.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f6741a30640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f673c0ca990 con 0x7f673c15da40 2024-04-25T12:33:57.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c15da40 0x7f673c15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c15da40 0x7f673c15de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36846/0 (socket says 172.21.15.153:36846) 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f673b7fe640 1 -- 172.21.15.153:0/4206822721 learned_addr learned my addr 172.21.15.153:0/4206822721 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f673b7fe640 1 -- 172.21.15.153:0/4206822721 >> v2:172.21.15.123:3300/0 conn(0x7f673c0c5a60 msgr2=0x7f673c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f673b7fe640 1 --2- 172.21.15.153:0/4206822721 >> v2:172.21.15.123:3300/0 conn(0x7f673c0c5a60 0x7f673c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f673b7fe640 1 -- 172.21.15.153:0/4206822721 >> v1:172.21.15.123:6789/0 conn(0x7f673c071a10 legacy=0x7f673c0c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.115+0000 7f673b7fe640 1 -- 172.21.15.153:0/4206822721 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c0cbc70 con 0x7f673c15da40 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.116+0000 7f673b7fe640 1 --2- 172.21.15.153:0/4206822721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c15da40 0x7f673c15de50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f673c0cd310 tx=0x7f673402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd78631fdfbd68c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.116+0000 7f6739ffb640 1 -- 172.21.15.153:0/4206822721 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734036070 con 0x7f673c15da40 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.116+0000 7f6739ffb640 1 -- 172.21.15.153:0/4206822721 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6734031600 con 0x7f673c15da40 2024-04-25T12:33:57.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.116+0000 7f6739ffb640 1 -- 172.21.15.153:0/4206822721 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67340319d0 con 0x7f673c15da40 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 -- 172.21.15.153:0/4206822721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c15da40 msgr2=0x7f673c15de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 --2- 172.21.15.153:0/4206822721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c15da40 0x7f673c15de50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f673c0cd310 tx=0x7f673402a6c0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 -- 172.21.15.153:0/4206822721 shutdown_connections 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 --2- 172.21.15.153:0/4206822721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c15da40 0x7f673c15de50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 --2- 172.21.15.153:0/4206822721 >> v2:172.21.15.123:3300/0 conn(0x7f673c0c5a60 0x7f673c0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 -- 172.21.15.153:0/4206822721 >> 172.21.15.153:0/4206822721 conn(0x7f673c072d70 msgr2=0x7f673c073180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 -- 172.21.15.153:0/4206822721 shutdown_connections 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.117+0000 7f6741a30640 1 -- 172.21.15.153:0/4206822721 wait complete. 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.119+0000 7f6741a30640 1 Processor -- start 2024-04-25T12:33:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.119+0000 7f6741a30640 1 -- start start 2024-04-25T12:33:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f6741a30640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c071a10 0x7f673c155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f6741a30640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c0c5a60 0x7f673c1f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f6741a30640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f673c15da40 0x7f673c1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f6741a30640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f673c0cc510 con 0x7f673c15da40 2024-04-25T12:33:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f6741a30640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f673c0cc230 con 0x7f673c0c5a60 2024-04-25T12:33:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f6741a30640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f673c0cc7f0 con 0x7f673c071a10 2024-04-25T12:33:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f673affd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c071a10 0x7f673c155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f673a7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c0c5a60 0x7f673c1f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f673affd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c071a10 0x7f673c155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36858/0 (socket says 172.21.15.153:36858) 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f673affd640 1 -- 172.21.15.153:0/921069101 learned_addr learned my addr 172.21.15.153:0/921069101 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.120+0000 7f673a7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c0c5a60 0x7f673c1f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51992/0 (socket says 172.21.15.153:51992) 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673b7fe640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f673c15da40 0x7f673c1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673affd640 1 -- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c0c5a60 msgr2=0x7f673c1f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673affd640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c0c5a60 0x7f673c1f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673affd640 1 -- 172.21.15.153:0/921069101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f673c15da40 msgr2=0x7f673c1f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673affd640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f673c15da40 0x7f673c1f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673affd640 1 -- 172.21.15.153:0/921069101 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c155a40 con 0x7f673c071a10 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673a7fc640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c0c5a60 0x7f673c1f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673b7fe640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f673c15da40 0x7f673c1f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f673affd640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c071a10 0x7f673c155500 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f673000b470 tx=0x7f673000b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f6723fff640 1 -- 172.21.15.153:0/921069101 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730015020 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f6723fff640 1 -- 172.21.15.153:0/921069101 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67300118b0 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f6723fff640 1 -- 172.21.15.153:0/921069101 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730011ca0 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c0c7a40 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c0c8000 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f6723fff640 1 -- 172.21.15.153:0/921069101 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6730018020 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.121+0000 7f6723fff640 1 -- 172.21.15.153:0/921069101 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6730010bc0 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.122+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c076700 con 0x7f673c071a10 2024-04-25T12:33:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.124+0000 7f6723fff640 1 -- 172.21.15.153:0/921069101 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f673001f020 con 0x7f673c071a10 2024-04-25T12:33:57.131 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:57.131+0000 7fcf3d321040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:33:57.183 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:57.184+0000 7fcf3d321040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:33:57.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.195+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cfb7cbbd-82cd-46fe-b89b-15775df1fe93", "id": 2} v 0) -- 0x7f673c0725c0 con 0x7f673c071a10 2024-04-25T12:33:57.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.203+0000 7f6723fff640 1 -- 172.21.15.153:0/921069101 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfb7cbbd-82cd-46fe-b89b-15775df1fe93", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f673000d040 con 0x7f673c071a10 2024-04-25T12:33:57.203 INFO:teuthology.orchestra.run.smithi153.stdout:2 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c071a10 msgr2=0x7f673c155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c071a10 0x7f673c155500 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f673000b470 tx=0x7f673000b930 comp rx=0 tx=0).stop 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 shutdown_connections 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f673c15da40 0x7f673c1f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c0c5a60 0x7f673c1f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 --2- 172.21.15.153:0/921069101 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f673c071a10 0x7f673c155500 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 >> 172.21.15.153:0/921069101 conn(0x7f673c072d70 msgr2=0x7f673c14d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 shutdown_connections 2024-04-25T12:33:57.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.204+0000 7f6741a30640 1 -- 172.21.15.153:0/921069101 wait complete. 2024-04-25T12:33:57.214 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 9e8ccd97-ab68-4f03-bbb3-5820eccf4568 3 2024-04-25T12:33:57.307 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:57.307+0000 7fcf3d321040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:33:57.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.322+0000 7f05f6ab6640 1 Processor -- start 2024-04-25T12:33:57.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.322+0000 7f05f6ab6640 1 -- start start 2024-04-25T12:33:57.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f6ab6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f05f0153910 0x7f05f00cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f6ab6640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f05f00c93f0 0x7f05f00c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f6ab6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f00c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f6ab6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f05f0153d20 con 0x7f05f0153910 2024-04-25T12:33:57.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f6ab6640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f05f00c3bc0 con 0x7f05f00c93f0 2024-04-25T12:33:57.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f6ab6640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f05f00c3ea0 con 0x7f05f00c9800 2024-04-25T12:33:57.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05effff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f05f00c93f0 0x7f05f00c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f5053640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f00c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f5053640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f00c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36860/0 (socket says 172.21.15.153:36860) 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05effff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f05f00c93f0 0x7f05f00c41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52000/0 (socket says 172.21.15.153:52000) 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.323+0000 7f05f5053640 1 -- 172.21.15.153:0/2572177994 learned_addr learned my addr 172.21.15.153:0/2572177994 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.324+0000 7f05f5053640 1 -- 172.21.15.153:0/2572177994 >> v2:172.21.15.153:3300/0 conn(0x7f05f00c93f0 msgr2=0x7f05f00c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.324+0000 7f05f5053640 1 --2- 172.21.15.153:0/2572177994 >> v2:172.21.15.153:3300/0 conn(0x7f05f00c93f0 0x7f05f00c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.324+0000 7f05f5053640 1 -- 172.21.15.153:0/2572177994 >> v2:172.21.15.123:3300/0 conn(0x7f05f0153910 msgr2=0x7f05f00cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.324+0000 7f05f5053640 1 --2- 172.21.15.153:0/2572177994 >> v2:172.21.15.123:3300/0 conn(0x7f05f0153910 0x7f05f00cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.324+0000 7f05f5053640 1 -- 172.21.15.153:0/2572177994 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f015ca40 con 0x7f05f00c9800 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.324+0000 7f05effff640 1 --2- 172.21.15.153:0/2572177994 >> v2:172.21.15.153:3300/0 conn(0x7f05f00c93f0 0x7f05f00c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:57.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.324+0000 7f05f5053640 1 --2- 172.21.15.153:0/2572177994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f00c4710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05f00c5c30 tx=0x7f05e802a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353bdb250473084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.325+0000 7f05ef7fe640 1 -- 172.21.15.153:0/2572177994 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8036070 con 0x7f05f00c9800 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.325+0000 7f05ef7fe640 1 -- 172.21.15.153:0/2572177994 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e8031600 con 0x7f05f00c9800 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.325+0000 7f05ef7fe640 1 -- 172.21.15.153:0/2572177994 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e80319d0 con 0x7f05f00c9800 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.325+0000 7f05f6ab6640 1 -- 172.21.15.153:0/2572177994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 msgr2=0x7f05f00c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.325+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/2572177994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f00c4710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05f00c5c30 tx=0x7f05e802a730 comp rx=0 tx=0).stop 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.326+0000 7f05f6ab6640 1 -- 172.21.15.153:0/2572177994 shutdown_connections 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.326+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/2572177994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f00c4710 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.326+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/2572177994 >> v2:172.21.15.153:3300/0 conn(0x7f05f00c93f0 0x7f05f00c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.326+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/2572177994 >> v2:172.21.15.123:3300/0 conn(0x7f05f0153910 0x7f05f00cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.326+0000 7f05f6ab6640 1 -- 172.21.15.153:0/2572177994 >> 172.21.15.153:0/2572177994 conn(0x7f05f0071b90 msgr2=0x7f05f0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.326+0000 7f05f6ab6640 1 -- 172.21.15.153:0/2572177994 shutdown_connections 2024-04-25T12:33:57.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.326+0000 7f05f6ab6640 1 -- 172.21.15.153:0/2572177994 wait complete. 2024-04-25T12:33:57.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.327+0000 7f05f6ab6640 1 Processor -- start 2024-04-25T12:33:57.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.327+0000 7f05f6ab6640 1 -- start start 2024-04-25T12:33:57.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f6ab6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f00c93f0 0x7f05f01f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f6ab6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f6ab6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05f0153910 0x7f05f00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f6ab6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05f015d2e0 con 0x7f05f0153910 2024-04-25T12:33:57.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f6ab6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05f015d000 con 0x7f05f00c93f0 2024-04-25T12:33:57.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f6ab6640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f05f00c3880 con 0x7f05f00c9800 2024-04-25T12:33:57.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f014bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36870/0 (socket says 172.21.15.153:36870) 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 -- 172.21.15.153:0/3505827781 learned_addr learned my addr 172.21.15.153:0/3505827781 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f4852640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f00c93f0 0x7f05f01f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 -- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f00c93f0 msgr2=0x7f05f01f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f00c93f0 0x7f05f01f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 -- 172.21.15.153:0/3505827781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05f0153910 msgr2=0x7f05f00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05f0153910 0x7f05f00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 -- 172.21.15.153:0/3505827781 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f014c0d0 con 0x7f05f00c9800 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05effff640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f014bb90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e960 tx=0x7f05dc00ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05ed7fa640 1 -- 172.21.15.153:0/3505827781 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc017070 con 0x7f05f00c9800 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05ed7fa640 1 -- 172.21.15.153:0/3505827781 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc00c9a0 con 0x7f05f00c9800 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05ed7fa640 1 -- 172.21.15.153:0/3505827781 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00cd70 con 0x7f05f00c9800 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.329+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f01f0a30 con 0x7f05f00c9800 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.330+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01f12b0 con 0x7f05f00c9800 2024-04-25T12:33:57.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.330+0000 7f05ed7fa640 1 -- 172.21.15.153:0/3505827781 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05dc01a070 con 0x7f05f00c9800 2024-04-25T12:33:57.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.330+0000 7f05ed7fa640 1 -- 172.21.15.153:0/3505827781 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f05dc01c8e0 con 0x7f05f00c9800 2024-04-25T12:33:57.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.330+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f0076700 con 0x7f05f00c9800 2024-04-25T12:33:57.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.335+0000 7f05ed7fa640 1 -- 172.21.15.153:0/3505827781 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f05dc01c450 con 0x7f05f00c9800 2024-04-25T12:33:57.389 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:57.389+0000 7fa6d9037040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:33:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.405+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9e8ccd97-ab68-4f03-bbb3-5820eccf4568", "id": 3} v 0) -- 0x7f05f006ffe0 con 0x7f05f00c9800 2024-04-25T12:33:57.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.413+0000 7f05ed7fa640 1 -- 172.21.15.153:0/3505827781 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9e8ccd97-ab68-4f03-bbb3-5820eccf4568", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f05dc009400 con 0x7f05f00c9800 2024-04-25T12:33:57.413 INFO:teuthology.orchestra.run.smithi153.stdout:3 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 msgr2=0x7f05f014bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f014bb90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e960 tx=0x7f05dc00ee20 comp rx=0 tx=0).stop 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 shutdown_connections 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05f0153910 0x7f05f00ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05f00c9800 0x7f05f014bb90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 --2- 172.21.15.153:0/3505827781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f00c93f0 0x7f05f01f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 >> 172.21.15.153:0/3505827781 conn(0x7f05f0071b90 msgr2=0x7f05f0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 shutdown_connections 2024-04-25T12:33:57.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.414+0000 7f05f6ab6640 1 -- 172.21.15.153:0/3505827781 wait complete. 2024-04-25T12:33:57.424 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 7d5bfd6e-173a-413d-abdb-df6d82c941d3 4 2024-04-25T12:33:57.502 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:57.503+0000 7fa6d9037040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:33:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.531+0000 7f33d3972640 1 Processor -- start 2024-04-25T12:33:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.531+0000 7f33d3972640 1 -- start start 2024-04-25T12:33:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.532+0000 7f33d3972640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f33cc06eb50 0x7f33cc156960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.532+0000 7f33d3972640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc0ca310 0x7f33cc0716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.532+0000 7f33d3972640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33cc15ff80 con 0x7f33cc0ca720 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.532+0000 7f33d3972640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f33cc160b00 con 0x7f33cc06eb50 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.532+0000 7f33d3972640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc160260 con 0x7f33cc0ca310 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d0f0d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc0ca310 0x7f33cc0716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d0f0d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc0ca310 0x7f33cc0716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36876/0 (socket says 172.21.15.153:36876) 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d0f0d640 1 -- 172.21.15.153:0/361027529 learned_addr learned my addr 172.21.15.153:0/361027529 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d1f0f640 1 --1- 172.21.15.153:0/361027529 >> v1:172.21.15.153:6789/0 conn(0x7f33cc0ca720 0x7f33cc0cb030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:42914/0 (socket says 172.21.15.153:42914) 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33bffff640 1 -- 172.21.15.153:0/361027529 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797072598 0 0) 0x7f33cc15ff80 con 0x7f33cc0ca720 2024-04-25T12:33:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d0f0d640 1 -- 172.21.15.153:0/361027529 >> v2:172.21.15.123:3300/0 conn(0x7f33cc06eb50 msgr2=0x7f33cc156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d0f0d640 1 --2- 172.21.15.153:0/361027529 >> v2:172.21.15.123:3300/0 conn(0x7f33cc06eb50 0x7f33cc156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d0f0d640 1 -- 172.21.15.153:0/361027529 >> v1:172.21.15.153:6789/0 conn(0x7f33cc0ca720 legacy=0x7f33cc0cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.533+0000 7f33d0f0d640 1 -- 172.21.15.153:0/361027529 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc160540 con 0x7f33cc0ca310 2024-04-25T12:33:57.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.534+0000 7f33d0f0d640 1 --2- 172.21.15.153:0/361027529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc0ca310 0x7f33cc0716c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fce05affe7b62d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.534+0000 7f33bffff640 1 -- 172.21.15.153:0/361027529 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8037070 con 0x7f33cc0ca310 2024-04-25T12:33:57.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.534+0000 7f33bffff640 1 -- 172.21.15.153:0/361027529 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c8032440 con 0x7f33cc0ca310 2024-04-25T12:33:57.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.534+0000 7f33bffff640 1 -- 172.21.15.153:0/361027529 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8032810 con 0x7f33cc0ca310 2024-04-25T12:33:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.534+0000 7f33d3972640 1 -- 172.21.15.153:0/361027529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc0ca310 msgr2=0x7f33cc0716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.534+0000 7f33d3972640 1 --2- 172.21.15.153:0/361027529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc0ca310 0x7f33cc0716c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c802a270 comp rx=0 tx=0).stop 2024-04-25T12:33:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.535+0000 7f33d3972640 1 -- 172.21.15.153:0/361027529 shutdown_connections 2024-04-25T12:33:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.535+0000 7f33d3972640 1 --2- 172.21.15.153:0/361027529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc0ca310 0x7f33cc0716c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.535+0000 7f33d3972640 1 --2- 172.21.15.153:0/361027529 >> v2:172.21.15.123:3300/0 conn(0x7f33cc06eb50 0x7f33cc156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.535+0000 7f33d3972640 1 -- 172.21.15.153:0/361027529 >> 172.21.15.153:0/361027529 conn(0x7f33cc072d70 msgr2=0x7f33cc073180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.535+0000 7f33d3972640 1 -- 172.21.15.153:0/361027529 shutdown_connections 2024-04-25T12:33:57.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.535+0000 7f33d3972640 1 -- 172.21.15.153:0/361027529 wait complete. 2024-04-25T12:33:57.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.536+0000 7f33d3972640 1 Processor -- start 2024-04-25T12:33:57.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.536+0000 7f33d3972640 1 -- start start 2024-04-25T12:33:57.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.537+0000 7f33d3972640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc0ca310 0x7f33cc14c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.537+0000 7f33d3972640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc0ca720 0x7f33cc1483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d3972640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc1f54d0 0x7f33cc0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d3972640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc0cb740 con 0x7f33cc0ca310 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d3972640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc0711a0 con 0x7f33cc0ca720 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d3972640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc0cb8c0 con 0x7f33cc1f54d0 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d0f0d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc0ca720 0x7f33cc1483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d1f0f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc1f54d0 0x7f33cc0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d0f0d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc0ca720 0x7f33cc1483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52026/0 (socket says 172.21.15.153:52026) 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d0f0d640 1 -- 172.21.15.153:0/3373425805 learned_addr learned my addr 172.21.15.153:0/3373425805 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d1f0f640 1 -- 172.21.15.153:0/3373425805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc0ca720 msgr2=0x7f33cc1483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d1f0f640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc0ca720 0x7f33cc1483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d1f0f640 1 -- 172.21.15.153:0/3373425805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc0ca310 msgr2=0x7f33cc14c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d170e640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc0ca310 0x7f33cc14c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d1f0f640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc0ca310 0x7f33cc14c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d1f0f640 1 -- 172.21.15.153:0/3373425805 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc14ce60 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d1f0f640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc1f54d0 0x7f33cc0ce6f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f33c400c780 tx=0x7f33c400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33bdffb640 1 -- 172.21.15.153:0/3373425805 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4018070 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33bdffb640 1 -- 172.21.15.153:0/3373425805 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c4004d60 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33bdffb640 1 -- 172.21.15.153:0/3373425805 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c400a880 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.538+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc157370 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.539+0000 7f33bdffb640 1 -- 172.21.15.153:0/3373425805 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f33c401b070 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.539+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1f2710 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.539+0000 7f33bdffb640 1 -- 172.21.15.153:0/3373425805 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f33c40070d0 con 0x7f33cc1f54d0 2024-04-25T12:33:57.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.539+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc06f810 con 0x7f33cc1f54d0 2024-04-25T12:33:57.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.543+0000 7f33bdffb640 1 -- 172.21.15.153:0/3373425805 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f33c40072d0 con 0x7f33cc1f54d0 2024-04-25T12:33:57.558 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:57.558+0000 7fa6d9037040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:33:57.602 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:57.602+0000 7fcf3d321040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:33:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.614+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7d5bfd6e-173a-413d-abdb-df6d82c941d3", "id": 4} v 0) -- 0x7f33cc076700 con 0x7f33cc1f54d0 2024-04-25T12:33:57.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.621+0000 7f33bdffb640 1 -- 172.21.15.153:0/3373425805 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d5bfd6e-173a-413d-abdb-df6d82c941d3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f33c40077a0 con 0x7f33cc1f54d0 2024-04-25T12:33:57.621 INFO:teuthology.orchestra.run.smithi153.stdout:4 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.622+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc1f54d0 msgr2=0x7f33cc0ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.622+0000 7f33d3972640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc1f54d0 0x7f33cc0ce6f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f33c400c780 tx=0x7f33c400cc40 comp rx=0 tx=0).stop 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.623+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 shutdown_connections 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.623+0000 7f33d3972640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33cc1f54d0 0x7f33cc0ce6f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.623+0000 7f33d3972640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc0ca720 0x7f33cc1483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.623+0000 7f33d3972640 1 --2- 172.21.15.153:0/3373425805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc0ca310 0x7f33cc14c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.623+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 >> 172.21.15.153:0/3373425805 conn(0x7f33cc072d70 msgr2=0x7f33cc073180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.623+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 shutdown_connections 2024-04-25T12:33:57.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.623+0000 7f33d3972640 1 -- 172.21.15.153:0/3373425805 wait complete. 2024-04-25T12:33:57.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 9141fa91-c547-4711-910a-9ef123017fb5 5 2024-04-25T12:33:57.654 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:57.655+0000 7fcf3d321040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:33:57.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.741+0000 7f4ef09e3640 1 Processor -- start 2024-04-25T12:33:57.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.741+0000 7f4ef09e3640 1 -- start start 2024-04-25T12:33:57.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4ef09e3640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4eec0c93f0 0x7f4eec0c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4ef09e3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec0c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4ef09e3640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec0c3bc0 con 0x7f4eec153910 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4ef09e3640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec0cb3a0 con 0x7f4eec0c93f0 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4ef09e3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec153d20 con 0x7f4eec0c9800 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4eead9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec0c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4eead9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec0c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36890/0 (socket says 172.21.15.153:36890) 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4eead9d640 1 -- 172.21.15.153:0/1265366472 learned_addr learned my addr 172.21.15.153:0/1265366472 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4eead9d640 1 -- 172.21.15.153:0/1265366472 >> v2:172.21.15.123:3300/0 conn(0x7f4eec0c93f0 msgr2=0x7f4eec0c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4eead9d640 1 --2- 172.21.15.153:0/1265366472 >> v2:172.21.15.123:3300/0 conn(0x7f4eec0c93f0 0x7f4eec0c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4eead9d640 1 -- 172.21.15.153:0/1265366472 >> v1:172.21.15.123:6789/0 conn(0x7f4eec153910 legacy=0x7f4eec0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.742+0000 7f4eead9d640 1 -- 172.21.15.153:0/1265366472 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0c3ea0 con 0x7f4eec0c9800 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.743+0000 7f4eead9d640 1 --2- 172.21.15.153:0/1265366472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec0c4710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009e50 tx=0x7f4ee402a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a01cabecf49838 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.743+0000 7f4ee9d9b640 1 -- 172.21.15.153:0/1265366472 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4036070 con 0x7f4eec0c9800 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.743+0000 7f4ee9d9b640 1 -- 172.21.15.153:0/1265366472 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee4031600 con 0x7f4eec0c9800 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.743+0000 7f4ee9d9b640 1 -- 172.21.15.153:0/1265366472 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40319d0 con 0x7f4eec0c9800 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 -- 172.21.15.153:0/1265366472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 msgr2=0x7f4eec0c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 --2- 172.21.15.153:0/1265366472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec0c4710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009e50 tx=0x7f4ee402a750 comp rx=0 tx=0).stop 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 -- 172.21.15.153:0/1265366472 shutdown_connections 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 --2- 172.21.15.153:0/1265366472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec0c4710 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 --2- 172.21.15.153:0/1265366472 >> v2:172.21.15.123:3300/0 conn(0x7f4eec0c93f0 0x7f4eec0c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 -- 172.21.15.153:0/1265366472 >> 172.21.15.153:0/1265366472 conn(0x7f4eec071b90 msgr2=0x7f4eec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 -- 172.21.15.153:0/1265366472 shutdown_connections 2024-04-25T12:33:57.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.744+0000 7f4ef09e3640 1 -- 172.21.15.153:0/1265366472 wait complete. 2024-04-25T12:33:57.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.746+0000 7f4ef09e3640 1 Processor -- start 2024-04-25T12:33:57.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.746+0000 7f4ef09e3640 1 -- start start 2024-04-25T12:33:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ef09e3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eec0c93f0 0x7f4eec145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ef09e3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ef09e3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eec153910 0x7f4eec06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ef09e3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec15d000 con 0x7f4eec153910 2024-04-25T12:33:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ef09e3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec15cd20 con 0x7f4eec0c93f0 2024-04-25T12:33:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ef09e3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec15d2e0 con 0x7f4eec0c9800 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4eea59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eec0c93f0 0x7f4eec145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec1f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36896/0 (socket says 172.21.15.153:36896) 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4ee3fff640 1 -- 172.21.15.153:0/985037751 learned_addr learned my addr 172.21.15.153:0/985037751 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.747+0000 7f4eea59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eec0c93f0 0x7f4eec145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52036/0 (socket says 172.21.15.153:52036) 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4eead9d640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eec153910 0x7f4eec06c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee3fff640 1 -- 172.21.15.153:0/985037751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eec0c93f0 msgr2=0x7f4eec145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee3fff640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eec0c93f0 0x7f4eec145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee3fff640 1 -- 172.21.15.153:0/985037751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eec153910 msgr2=0x7f4eec06c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee3fff640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eec153910 0x7f4eec06c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee3fff640 1 -- 172.21.15.153:0/985037751 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec146120 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee3fff640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec1f42e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b410 tx=0x7f4ed400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee2ffd640 1 -- 172.21.15.153:0/985037751 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4015020 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee2ffd640 1 -- 172.21.15.153:0/985037751 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed4011850 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec1f4b40 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee2ffd640 1 -- 172.21.15.153:0/985037751 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4011c40 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec144e20 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee2ffd640 1 -- 172.21.15.153:0/985037751 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ed4018020 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.748+0000 7f4ee2ffd640 1 -- 172.21.15.153:0/985037751 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f4ed4011de0 con 0x7f4eec0c9800 2024-04-25T12:33:57.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.749+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec076700 con 0x7f4eec0c9800 2024-04-25T12:33:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.752+0000 7f4ee2ffd640 1 -- 172.21.15.153:0/985037751 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4ed401f020 con 0x7f4eec0c9800 2024-04-25T12:33:57.766 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:57.767+0000 7fa6d9037040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:33:57.817 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:57.817+0000 7fcf3d321040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:33:57.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.823+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9141fa91-c547-4711-910a-9ef123017fb5", "id": 5} v 0) -- 0x7f4eec0c5a10 con 0x7f4eec0c9800 2024-04-25T12:33:57.829 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:57.830+0000 7fa6d9037040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:33:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.832+0000 7f4ee2ffd640 1 -- 172.21.15.153:0/985037751 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9141fa91-c547-4711-910a-9ef123017fb5", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f4ed401a860 con 0x7f4eec0c9800 2024-04-25T12:33:57.832 INFO:teuthology.orchestra.run.smithi153.stdout:5 2024-04-25T12:33:57.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.833+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 msgr2=0x7f4eec1f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.833+0000 7f4ef09e3640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec1f42e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b410 tx=0x7f4ed400b8d0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.833+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 shutdown_connections 2024-04-25T12:33:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.833+0000 7f4ef09e3640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eec153910 0x7f4eec06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.833+0000 7f4ef09e3640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eec0c9800 0x7f4eec1f42e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.833+0000 7f4ef09e3640 1 --2- 172.21.15.153:0/985037751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eec0c93f0 0x7f4eec145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.833+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 >> 172.21.15.153:0/985037751 conn(0x7f4eec071b90 msgr2=0x7f4eec0705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.834+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 shutdown_connections 2024-04-25T12:33:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.834+0000 7f4ef09e3640 1 -- 172.21.15.153:0/985037751 wait complete. 2024-04-25T12:33:57.844 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 66fbc935-086e-4b03-aa03-96e0bc8b1d27 6 2024-04-25T12:33:57.890 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:57.891+0000 7fa6d9037040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:33:57.960 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:57.960+0000 7fcf3d321040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:33:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.992+0000 7effe5f84640 1 Processor -- start 2024-04-25T12:33:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.992+0000 7effe5f84640 1 -- start start 2024-04-25T12:33:57.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.993+0000 7effe5f84640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.993+0000 7effe5f84640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7effe0073800 0x7effe0073c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:57.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effe5f84640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe01606f0 con 0x7effe006d5f0 2024-04-25T12:33:57.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effe5f84640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7effe00c5b60 con 0x7effe0073800 2024-04-25T12:33:57.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effe5f84640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7effe01609d0 con 0x7effe006db20 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdffff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7effe0073800 0x7effe0073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdf7fe640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7effe006d5f0 0x7effe0156850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:42920/0 (socket says 172.21.15.153:42920) 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effd7fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdf7fe640 1 -- 172.21.15.153:0/4269900665 learned_addr learned my addr 172.21.15.153:0/4269900665 (peer_addr_for_me v1:172.21.15.153:0/0) 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdeffd640 1 -- 172.21.15.153:0/4269900665 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2970013625 0 0) 0x7effe01606f0 con 0x7effe006d5f0 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdeffd640 1 -- 172.21.15.153:0/4269900665 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effcc003520 con 0x7effe006d5f0 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdffff640 1 -- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 msgr2=0x7effe00ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdffff640 1 --2- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdffff640 1 -- 172.21.15.153:0/4269900665 >> v1:172.21.15.153:6789/0 conn(0x7effe006d5f0 legacy=0x7effe0156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effdffff640 1 -- 172.21.15.153:0/4269900665 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe00c55a0 con 0x7effe0073800 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.994+0000 7effd7fff640 1 --2- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:57.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.995+0000 7effdffff640 1 --2- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 0x7effe0073c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7effd8009d90 tx=0x7effd802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cd28c4852a76ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:57.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.995+0000 7effdeffd640 1 -- 172.21.15.153:0/4269900665 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8007740 con 0x7effe0073800 2024-04-25T12:33:57.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.995+0000 7effdeffd640 1 -- 172.21.15.153:0/4269900665 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd80078e0 con 0x7effe0073800 2024-04-25T12:33:57.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.995+0000 7effdeffd640 1 -- 172.21.15.153:0/4269900665 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8007cb0 con 0x7effe0073800 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 -- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 msgr2=0x7effe0073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 --2- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 0x7effe0073c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7effd8009d90 tx=0x7effd802a6f0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 -- 172.21.15.153:0/4269900665 shutdown_connections 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 --2- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 0x7effe0073c10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 --2- 172.21.15.153:0/4269900665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 -- 172.21.15.153:0/4269900665 >> 172.21.15.153:0/4269900665 conn(0x7effe0071f10 msgr2=0x7effe00caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 -- 172.21.15.153:0/4269900665 shutdown_connections 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.996+0000 7effe5f84640 1 -- 172.21.15.153:0/4269900665 wait complete. 2024-04-25T12:33:57.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.998+0000 7effe5f84640 1 Processor -- start 2024-04-25T12:33:57.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.998+0000 7effe5f84640 1 -- start start 2024-04-25T12:33:58.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effe5f84640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe006d5f0 0x7effe0152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effe5f84640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effe5f84640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 0x7effe00cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effe5f84640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effe015cbb0 con 0x7effe006d5f0 2024-04-25T12:33:58.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effe5f84640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effe015ca30 con 0x7effe0073800 2024-04-25T12:33:58.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effe5f84640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7effe015cd30 con 0x7effe006db20 2024-04-25T12:33:58.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effdffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 0x7effe00cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:36908/0 (socket says 172.21.15.153:36908) 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 -- 172.21.15.153:0/824702025 learned_addr learned my addr 172.21.15.153:0/824702025 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 -- 172.21.15.153:0/824702025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 msgr2=0x7effe00cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 --2- 172.21.15.153:0/824702025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 0x7effe00cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 -- 172.21.15.153:0/824702025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe006d5f0 msgr2=0x7effe0152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 --2- 172.21.15.153:0/824702025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe006d5f0 0x7effe0152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 -- 172.21.15.153:0/824702025 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe0153050 con 0x7effe006db20 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:57.999+0000 7effd7fff640 1 --2- 172.21.15.153:0/824702025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00cbd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7effc800e960 tx=0x7effc800ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.000+0000 7effdcff9640 1 -- 172.21.15.153:0/824702025 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8017070 con 0x7effe006db20 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.000+0000 7effdcff9640 1 -- 172.21.15.153:0/824702025 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc800ca00 con 0x7effe006db20 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.000+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe015fd70 con 0x7effe006db20 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.000+0000 7effdcff9640 1 -- 172.21.15.153:0/824702025 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc800cdd0 con 0x7effe006db20 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.000+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7effe00ce450 con 0x7effe006db20 2024-04-25T12:33:58.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.000+0000 7effdcff9640 1 -- 172.21.15.153:0/824702025 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7effc801a070 con 0x7effe006db20 2024-04-25T12:33:58.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.000+0000 7effdcff9640 1 -- 172.21.15.153:0/824702025 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7effc801ca40 con 0x7effe006db20 2024-04-25T12:33:58.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.001+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe0076700 con 0x7effe006db20 2024-04-25T12:33:58.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.003+0000 7effdcff9640 1 -- 172.21.15.153:0/824702025 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7effc801c450 con 0x7effe006db20 2024-04-25T12:33:58.013 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:58.013+0000 7fcf3d321040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:33:58.017 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:58.018+0000 7fa6d9037040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:33:58.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.074+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "66fbc935-086e-4b03-aa03-96e0bc8b1d27", "id": 6} v 0) -- 0x7effe00cde90 con 0x7effe006db20 2024-04-25T12:33:58.084 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:58.084+0000 7fcf3d321040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:33:58.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.084+0000 7effdcff9640 1 -- 172.21.15.153:0/824702025 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "66fbc935-086e-4b03-aa03-96e0bc8b1d27", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7effc801c630 con 0x7effe006db20 2024-04-25T12:33:58.084 INFO:teuthology.orchestra.run.smithi153.stdout:6 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 msgr2=0x7effe00cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 --2- 172.21.15.153:0/824702025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00cbd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7effc800e960 tx=0x7effc800ee20 comp rx=0 tx=0).stop 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 shutdown_connections 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 --2- 172.21.15.153:0/824702025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe0073800 0x7effe00cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 --2- 172.21.15.153:0/824702025 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7effe006db20 0x7effe00cbd20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 --2- 172.21.15.153:0/824702025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe006d5f0 0x7effe0152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 >> 172.21.15.153:0/824702025 conn(0x7effe0071f10 msgr2=0x7effe0153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 shutdown_connections 2024-04-25T12:33:58.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.086+0000 7effe5f84640 1 -- 172.21.15.153:0/824702025 wait complete. 2024-04-25T12:33:58.099 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new de59a619-b84a-4623-ac89-447178815d65 7 2024-04-25T12:33:58.133 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:58.134+0000 7fcf3d321040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:33:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.207+0000 7ff84c66e640 1 Processor -- start 2024-04-25T12:33:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.207+0000 7ff84c66e640 1 -- start start 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.208+0000 7ff84c66e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440c5df0 0x7ff84415b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.208+0000 7ff84c66e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84406d1d0 con 0x7ff8440711e0 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.208+0000 7ff84c66e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8440708b0 con 0x7ff8440ca270 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.208+0000 7ff84c66e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff84406cf80 con 0x7ff8440c5df0 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.209+0000 7ff84a40a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440c5df0 0x7ff84415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.209+0000 7ff84ac0b640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff8440ca270 0x7ff8440c7690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:42936/0 (socket says 172.21.15.153:42936) 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.209+0000 7ff84ac0b640 1 -- 172.21.15.153:0/2468792900 learned_addr learned my addr 172.21.15.153:0/2468792900 (peer_addr_for_me v1:172.21.15.153:0/0) 2024-04-25T12:33:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.209+0000 7ff849408640 1 -- 172.21.15.153:0/2468792900 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2732025532 0 0) 0x7ff8440708b0 con 0x7ff8440ca270 2024-04-25T12:33:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.209+0000 7ff84a40a640 1 -- 172.21.15.153:0/2468792900 >> v1:172.21.15.153:6789/0 conn(0x7ff8440ca270 legacy=0x7ff8440c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.209+0000 7ff84a40a640 1 -- 172.21.15.153:0/2468792900 >> v1:172.21.15.123:6789/0 conn(0x7ff8440711e0 legacy=0x7ff84415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.209+0000 7ff84a40a640 1 -- 172.21.15.153:0/2468792900 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff844070b90 con 0x7ff8440c5df0 2024-04-25T12:33:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.210+0000 7ff84a40a640 1 --2- 172.21.15.153:0/2468792900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440c5df0 0x7ff84415b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8340097d0 tx=0x7ff83402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8cca18601d203aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.210+0000 7ff849408640 1 -- 172.21.15.153:0/2468792900 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834037070 con 0x7ff8440c5df0 2024-04-25T12:33:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.210+0000 7ff849408640 1 -- 172.21.15.153:0/2468792900 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff834032440 con 0x7ff8440c5df0 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.210+0000 7ff849408640 1 -- 172.21.15.153:0/2468792900 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834032810 con 0x7ff8440c5df0 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.210+0000 7ff84c66e640 1 -- 172.21.15.153:0/2468792900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440c5df0 msgr2=0x7ff84415b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.210+0000 7ff84c66e640 1 --2- 172.21.15.153:0/2468792900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440c5df0 0x7ff84415b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8340097d0 tx=0x7ff83402a250 comp rx=0 tx=0).stop 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.211+0000 7ff84c66e640 1 -- 172.21.15.153:0/2468792900 shutdown_connections 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.211+0000 7ff84c66e640 1 --2- 172.21.15.153:0/2468792900 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440c5df0 0x7ff84415b100 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.211+0000 7ff84c66e640 1 -- 172.21.15.153:0/2468792900 >> 172.21.15.153:0/2468792900 conn(0x7ff844071b90 msgr2=0x7ff844071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.211+0000 7ff84c66e640 1 -- 172.21.15.153:0/2468792900 shutdown_connections 2024-04-25T12:33:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.211+0000 7ff84c66e640 1 -- 172.21.15.153:0/2468792900 wait complete. 2024-04-25T12:33:58.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.212+0000 7ff84c66e640 1 Processor -- start 2024-04-25T12:33:58.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.212+0000 7ff84c66e640 1 -- start start 2024-04-25T12:33:58.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.213+0000 7ff84c66e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440711e0 0x7ff8441514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84c66e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8440c5df0 0x7ff8440c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84c66e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440ca270 0x7ff8441472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84c66e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff84415e820 con 0x7ff8440c5df0 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84c66e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff84415a4e0 con 0x7ff8440711e0 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84c66e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff84415e9a0 con 0x7ff8440ca270 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440711e0 0x7ff8441514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84ac0b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440ca270 0x7ff8441472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440711e0 0x7ff8441514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52074/0 (socket says 172.21.15.153:52074) 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 -- 172.21.15.153:0/222647913 learned_addr learned my addr 172.21.15.153:0/222647913 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 -- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440ca270 msgr2=0x7ff8441472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440ca270 0x7ff8441472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 -- 172.21.15.153:0/222647913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8440c5df0 msgr2=0x7ff8440c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8440c5df0 0x7ff8440c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 -- 172.21.15.153:0/222647913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff844151a10 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84ac0b640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440ca270 0x7ff8441472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84a40a640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440711e0 0x7ff8441514d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8340097d0 tx=0x7ff8340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff83affd640 1 -- 172.21.15.153:0/222647913 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834037040 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff83affd640 1 -- 172.21.15.153:0/222647913 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff834042070 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff83affd640 1 -- 172.21.15.153:0/222647913 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834030bd0 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84414c700 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.214+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84414ccc0 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.215+0000 7ff83affd640 1 -- 172.21.15.153:0/222647913 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff834047460 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.215+0000 7ff83affd640 1 -- 172.21.15.153:0/222647913 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ff83403daa0 con 0x7ff8440711e0 2024-04-25T12:33:58.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.215+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff844076700 con 0x7ff8440711e0 2024-04-25T12:33:58.218 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.217+0000 7ff83affd640 1 -- 172.21.15.153:0/222647913 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff834032e20 con 0x7ff8440711e0 2024-04-25T12:33:58.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.288+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "de59a619-b84a-4623-ac89-447178815d65", "id": 7} v 0) -- 0x7ff844150a30 con 0x7ff8440711e0 2024-04-25T12:33:58.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.296+0000 7ff83affd640 1 -- 172.21.15.153:0/222647913 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "de59a619-b84a-4623-ac89-447178815d65", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ff83403d440 con 0x7ff8440711e0 2024-04-25T12:33:58.296 INFO:teuthology.orchestra.run.smithi153.stdout:7 2024-04-25T12:33:58.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440711e0 msgr2=0x7ff8441514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440711e0 0x7ff8441514d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8340097d0 tx=0x7ff8340046c0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 shutdown_connections 2024-04-25T12:33:58.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff8440ca270 0x7ff8441472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8440c5df0 0x7ff8440c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 --2- 172.21.15.153:0/222647913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8440711e0 0x7ff8441514d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 >> 172.21.15.153:0/222647913 conn(0x7ff844071b90 msgr2=0x7ff8441597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:58.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 shutdown_connections 2024-04-25T12:33:58.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-04-25T12:33:58.297+0000 7ff84c66e640 1 -- 172.21.15.153:0/222647913 wait complete. 2024-04-25T12:33:58.307 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T12:33:58.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T12:33:58.316 INFO:tasks.ceph.osd.0:Started 2024-04-25T12:33:58.316 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T12:33:58.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T12:33:58.323 INFO:tasks.ceph.osd.1:Started 2024-04-25T12:33:58.323 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T12:33:58.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T12:33:58.329 INFO:tasks.ceph.osd.2:Started 2024-04-25T12:33:58.329 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T12:33:58.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T12:33:58.334 INFO:tasks.ceph.osd.3:Started 2024-04-25T12:33:58.334 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T12:33:58.334 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T12:33:58.349 INFO:tasks.ceph.osd.4:Started 2024-04-25T12:33:58.350 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T12:33:58.350 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T12:33:58.352 INFO:tasks.ceph.osd.5:Started 2024-04-25T12:33:58.352 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T12:33:58.352 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T12:33:58.354 INFO:tasks.ceph.osd.6:Started 2024-04-25T12:33:58.354 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T12:33:58.354 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T12:33:58.356 INFO:tasks.ceph.osd.7:Started 2024-04-25T12:33:58.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:33:58.534 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:58.535+0000 7fa6d9037040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:33:58.658 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:58.658+0000 7fcf3d321040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:33:58.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.703+0000 7f2af3000640 1 Processor -- start 2024-04-25T12:33:58.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.703+0000 7f2af3000640 1 -- start start 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2af3000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2aec0c3880 0x7f2aec156a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2af3000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0706e0 con 0x7f2aec0ca380 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2af3000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0709c0 con 0x7f2aec0c9e30 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2af3000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec0c5560 con 0x7f2aec0c3880 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aebe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2aec0c3880 0x7f2aec156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aebe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2aec0c3880 0x7f2aec156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54406/0 (socket says 172.21.15.123:54406) 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aebe00640 1 -- 172.21.15.123:0/3511581844 learned_addr learned my addr 172.21.15.123:0/3511581844 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aeb400640 1 --1- 172.21.15.123:0/3511581844 >> v1:172.21.15.123:6789/0 conn(0x7f2aec0ca380 0x7f2aec0c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44282/0 (socket says 172.21.15.123:44282) 2024-04-25T12:33:58.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aeaa00640 1 -- 172.21.15.123:0/3511581844 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331397610 0 0) 0x7f2aec0709c0 con 0x7f2aec0c9e30 2024-04-25T12:33:58.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aeaa00640 1 -- 172.21.15.123:0/3511581844 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad0003520 con 0x7f2aec0c9e30 2024-04-25T12:33:58.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aebe00640 1 -- 172.21.15.123:0/3511581844 >> v1:172.21.15.153:6789/0 conn(0x7f2aec0c9e30 legacy=0x7f2aec15ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aebe00640 1 -- 172.21.15.123:0/3511581844 >> v1:172.21.15.123:6789/0 conn(0x7f2aec0ca380 legacy=0x7f2aec0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.705+0000 7f2aebe00640 1 -- 172.21.15.123:0/3511581844 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec156fb0 con 0x7f2aec0c3880 2024-04-25T12:33:58.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.707+0000 7f2aebe00640 1 --2- 172.21.15.123:0/3511581844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0c3880 0x7f2aec156a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009870 tx=0x7f2adc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0df2a7e33447b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:58.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.707+0000 7f2aeaa00640 1 -- 172.21.15.123:0/3511581844 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc037070 con 0x7f2aec0c3880 2024-04-25T12:33:58.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.707+0000 7f2aeaa00640 1 -- 172.21.15.123:0/3511581844 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc032440 con 0x7f2aec0c3880 2024-04-25T12:33:58.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.707+0000 7f2aeaa00640 1 -- 172.21.15.123:0/3511581844 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc032810 con 0x7f2aec0c3880 2024-04-25T12:33:58.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.708+0000 7f2af3000640 1 -- 172.21.15.123:0/3511581844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0c3880 msgr2=0x7f2aec156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.708+0000 7f2af3000640 1 --2- 172.21.15.123:0/3511581844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0c3880 0x7f2aec156a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009870 tx=0x7f2adc02a2a0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.708+0000 7f2af3000640 1 -- 172.21.15.123:0/3511581844 shutdown_connections 2024-04-25T12:33:58.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.708+0000 7f2af3000640 1 --2- 172.21.15.123:0/3511581844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0c3880 0x7f2aec156a70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.708+0000 7f2af3000640 1 -- 172.21.15.123:0/3511581844 >> 172.21.15.123:0/3511581844 conn(0x7f2aec071b90 msgr2=0x7f2aec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:58.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.708+0000 7f2af3000640 1 -- 172.21.15.123:0/3511581844 shutdown_connections 2024-04-25T12:33:58.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.708+0000 7f2af3000640 1 -- 172.21.15.123:0/3511581844 wait complete. 2024-04-25T12:33:58.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.710+0000 7f2af3000640 1 Processor -- start 2024-04-25T12:33:58.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.710+0000 7f2af3000640 1 -- start start 2024-04-25T12:33:58.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.711+0000 7f2af3000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aec0c3880 0x7f2aec1f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.711+0000 7f2af3000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aec0c9e30 0x7f2aec151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.711+0000 7f2af3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0ca380 0x7f2aec14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.711+0000 7f2af3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1607d0 con 0x7f2aec0ca380 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.711+0000 7f2af3000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec160650 con 0x7f2aec0c3880 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.711+0000 7f2af3000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec154e70 con 0x7f2aec0c9e30 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aeb400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aec0c9e30 0x7f2aec151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aebe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aec0c3880 0x7f2aec1f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aeb400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aec0c9e30 0x7f2aec151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38748/0 (socket says 172.21.15.123:38748) 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aeb400640 1 -- 172.21.15.123:0/2392664803 learned_addr learned my addr 172.21.15.123:0/2392664803 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2af1000640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0ca380 0x7f2aec14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aebe00640 1 -- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aec0c9e30 msgr2=0x7f2aec151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aebe00640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aec0c9e30 0x7f2aec151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aebe00640 1 -- 172.21.15.123:0/2392664803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0ca380 msgr2=0x7f2aec14be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aebe00640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0ca380 0x7f2aec14be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aebe00640 1 -- 172.21.15.123:0/2392664803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec1549a0 con 0x7f2aec0c3880 2024-04-25T12:33:58.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aeb400640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aec0c9e30 0x7f2aec151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.712+0000 7f2aebe00640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aec0c3880 0x7f2aec1f2350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2adc031000 tx=0x7f2adc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.713+0000 7f2ae3e00640 1 -- 172.21.15.123:0/2392664803 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc041070 con 0x7f2aec0c3880 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.713+0000 7f2ae3e00640 1 -- 172.21.15.123:0/2392664803 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc03c870 con 0x7f2aec0c3880 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.713+0000 7f2ae3e00640 1 -- 172.21.15.123:0/2392664803 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc037040 con 0x7f2aec0c3880 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.713+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec154560 con 0x7f2aec0c3880 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.713+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec0ccf80 con 0x7f2aec0c3880 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.713+0000 7f2ae3e00640 1 -- 172.21.15.123:0/2392664803 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2adc044070 con 0x7f2aec0c3880 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.713+0000 7f2ae3e00640 1 -- 172.21.15.123:0/2392664803 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2adc046450 con 0x7f2aec0c3880 2024-04-25T12:33:58.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.714+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec076700 con 0x7f2aec0c3880 2024-04-25T12:33:58.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.716+0000 7f2ae3e00640 1 -- 172.21.15.123:0/2392664803 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2adc03cc60 con 0x7f2aec0c3880 2024-04-25T12:33:58.775 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:58.775+0000 7fcf3d321040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:33:58.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.780+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2aec0711a0 con 0x7f2aec0c3880 2024-04-25T12:33:58.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.781+0000 7f2ae3e00640 1 -- 172.21.15.123:0/2392664803 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2adc03ce40 con 0x7f2aec0c3880 2024-04-25T12:33:58.781 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:33:58.781 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":10,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:33:58.290536+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:33:58.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aec0c3880 msgr2=0x7f2aec1f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:58.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aec0c3880 0x7f2aec1f2350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2adc031000 tx=0x7f2adc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 shutdown_connections 2024-04-25T12:33:58.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aec0ca380 0x7f2aec14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2aec0c9e30 0x7f2aec151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 --2- 172.21.15.123:0/2392664803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2aec0c3880 0x7f2aec1f2350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:58.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 >> 172.21.15.123:0/2392664803 conn(0x7f2aec071b90 msgr2=0x7f2aec0cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:58.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 shutdown_connections 2024-04-25T12:33:58.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:33:58.782+0000 7f2af3000640 1 -- 172.21.15.123:0/2392664803 wait complete. 2024-04-25T12:33:58.805 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T12:33:58.805 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T12:33:58.858 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:58.858+0000 7fa6d9037040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:33:58.921 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:58.921+0000 7fa6d9037040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:33:58.954 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:33:58.954+0000 7ff8bb933640 -1 Falling back to public interface 2024-04-25T12:33:58.958 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:33:58.958+0000 7f408aa13640 -1 Falling back to public interface 2024-04-25T12:33:58.991 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:33:58.991+0000 7f699ec14640 -1 Falling back to public interface 2024-04-25T12:33:58.997 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:33:58.997+0000 7fed0b212640 -1 Falling back to public interface 2024-04-25T12:33:59.053 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:59.053+0000 7fa6d9037040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:33:59.088 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:33:59.088+0000 7f8b7420e640 -1 Falling back to public interface 2024-04-25T12:33:59.143 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:33:59.143+0000 7fbef5726640 -1 Falling back to public interface 2024-04-25T12:33:59.149 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:33:59.149+0000 7f3e0b40f640 -1 Falling back to public interface 2024-04-25T12:33:59.193 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:33:59.193+0000 7f87011dc640 -1 Falling back to public interface 2024-04-25T12:33:59.388 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:59.388+0000 7fa6d9037040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:33:59.446 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:59.447+0000 7fa6d9037040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:33:59.610 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T12:33:59.633 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:59.633+0000 7fa6d9037040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:33:59.797 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:59.797+0000 7fcedd4cd640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T12:33:59.797 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-04-25T12:33:59.797+0000 7fcedd4cd640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T12:33:59.809 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:59.810+0000 7fa6d9037040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:33:59.834 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.834+0000 7ff5f5600640 1 Processor -- start 2024-04-25T12:33:59.834 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.834+0000 7ff5f5600640 1 -- start start 2024-04-25T12:33:59.836 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5f5600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f01609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:59.836 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5f5600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5f00c5da0 0x7ff5f015c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:59.836 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5f5600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f00c6370 con 0x7ff5f0156930 2024-04-25T12:33:59.836 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5f5600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f00c6910 con 0x7ff5f00c5da0 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5f5600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f00c6630 con 0x7ff5f006df80 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5ee000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5f00c5da0 0x7ff5f015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5ef400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff5f0156930 0x7ff5f00cb060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44290/0 (socket says 172.21.15.123:44290) 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5ef400640 1 -- 172.21.15.123:0/694006580 learned_addr learned my addr 172.21.15.123:0/694006580 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.836+0000 7ff5eea00640 1 --2- 172.21.15.123:0/694006580 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f01609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.837+0000 7ff5ee000640 1 -- 172.21.15.123:0/694006580 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 msgr2=0x7ff5f01609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.837+0000 7ff5ee000640 1 --2- 172.21.15.123:0/694006580 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f01609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.837+0000 7ff5ee000640 1 -- 172.21.15.123:0/694006580 >> v1:172.21.15.123:6789/0 conn(0x7ff5f0156930 legacy=0x7ff5f00cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.837+0000 7ff5ed600640 1 -- 172.21.15.123:0/694006580 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209837067 0 0) 0x7ff5f00c6370 con 0x7ff5f0156930 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.837+0000 7ff5ee000640 1 -- 172.21.15.123:0/694006580 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f015ff10 con 0x7ff5f00c5da0 2024-04-25T12:33:59.837 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.837+0000 7ff5eea00640 1 --2- 172.21.15.123:0/694006580 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f01609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:59.838 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.838+0000 7ff5ee000640 1 --2- 172.21.15.123:0/694006580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f00c5da0 0x7ff5f015c880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0097a0 tx=0x7ff5dc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc558f52b915b55e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:59.838 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.838+0000 7ff5ed600640 1 -- 172.21.15.123:0/694006580 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc037070 con 0x7ff5f00c5da0 2024-04-25T12:33:59.838 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.838+0000 7ff5ed600640 1 -- 172.21.15.123:0/694006580 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5dc02ae50 con 0x7ff5f00c5da0 2024-04-25T12:33:59.838 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.838+0000 7ff5ed600640 1 -- 172.21.15.123:0/694006580 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc032660 con 0x7ff5f00c5da0 2024-04-25T12:33:59.839 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 -- 172.21.15.123:0/694006580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f00c5da0 msgr2=0x7ff5f015c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:59.839 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 --2- 172.21.15.123:0/694006580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f00c5da0 0x7ff5f015c880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0097a0 tx=0x7ff5dc02a100 comp rx=0 tx=0).stop 2024-04-25T12:33:59.839 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 -- 172.21.15.123:0/694006580 shutdown_connections 2024-04-25T12:33:59.839 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 --2- 172.21.15.123:0/694006580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f00c5da0 0x7ff5f015c880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:59.840 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 --2- 172.21.15.123:0/694006580 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f01609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:59.840 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 -- 172.21.15.123:0/694006580 >> 172.21.15.123:0/694006580 conn(0x7ff5f0072e60 msgr2=0x7ff5f0073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:33:59.840 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 -- 172.21.15.123:0/694006580 shutdown_connections 2024-04-25T12:33:59.840 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.839+0000 7ff5f5600640 1 -- 172.21.15.123:0/694006580 wait complete. 2024-04-25T12:33:59.841 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.841+0000 7ff5f5600640 1 Processor -- start 2024-04-25T12:33:59.841 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.842+0000 7ff5f5600640 1 -- start start 2024-04-25T12:33:59.843 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5f5600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:59.843 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5f5600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5f00c5da0 0x7ff5f00c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5f5600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f0156930 0x7ff5f01f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5f5600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f006d680 con 0x7ff5f0156930 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5f5600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01604d0 con 0x7ff5f00c5da0 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5f5600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f00cbde0 con 0x7ff5f006df80 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f0156930 0x7ff5f01f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f0156930 0x7ff5f01f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54422/0 (socket says 172.21.15.123:54422) 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5ef400640 1 -- 172.21.15.123:0/4097181031 learned_addr learned my addr 172.21.15.123:0/4097181031 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5eea00640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.843+0000 7ff5ee000640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5f00c5da0 0x7ff5f00c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5ef400640 1 -- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 msgr2=0x7ff5f00c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5ef400640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f00c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:59.844 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5ef400640 1 -- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5f00c5da0 msgr2=0x7ff5f00c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:33:59.845 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5ef400640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5f00c5da0 0x7ff5f00c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:33:59.845 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5ef400640 1 -- 172.21.15.123:0/4097181031 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f01f3a30 con 0x7ff5f0156930 2024-04-25T12:33:59.845 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5ee000640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5f00c5da0 0x7ff5f00c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:33:59.845 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5ef400640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f0156930 0x7ff5f01f34f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c7b0 tx=0x7ff5e800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:59.846 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5e6a00640 1 -- 172.21.15.123:0/4097181031 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8018070 con 0x7ff5f0156930 2024-04-25T12:33:59.846 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5e6a00640 1 -- 172.21.15.123:0/4097181031 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e800a640 con 0x7ff5f0156930 2024-04-25T12:33:59.846 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.844+0000 7ff5e6a00640 1 -- 172.21.15.123:0/4097181031 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800ce60 con 0x7ff5f0156930 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.845+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f015c400 con 0x7ff5f0156930 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.845+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f01f2cf0 con 0x7ff5f0156930 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.846+0000 7ff5e6a00640 1 -- 172.21.15.123:0/4097181031 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 3) ==== 48543+0+0 (secure 0 0 0) 0x7ff5e801b070 con 0x7ff5f0156930 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.846+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f015c920 con 0x7ff5f0156930 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.846+0000 7ff5e6a00640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff5d003c300 0x7ff5d003e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.847+0000 7ff5e6a00640 1 -- 172.21.15.123:0/4097181031 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff5e804ece0 con 0x7ff5f0156930 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.847+0000 7ff5eea00640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff5d003c300 0x7ff5d003e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:33:59.847 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.847+0000 7ff5eea00640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff5d003c300 0x7ff5d003e810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009560 tx=0x7ff5e0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:33:59.852 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:33:59.853+0000 7ff5e6a00640 1 -- 172.21.15.123:0/4097181031 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5f015c920 con 0x7ff5f0156930 2024-04-25T12:33:59.888 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:59.888+0000 7fa6d9037040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:33:59.977 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:33:59.977+0000 7fa6d9037040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:34:00.034 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:34:00.035+0000 7fa6d9037040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:34:00.062 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.063+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5f015a2a0 con 0x7ff5f0156930 2024-04-25T12:34:00.063 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.064+0000 7ff5e6a00640 1 -- 172.21.15.123:0/4097181031 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff5e8011360 con 0x7ff5f0156930 2024-04-25T12:34:00.063 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-25T12:34:00.063 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":10,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:33:58.290536+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:00.067 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.067+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff5d003c300 msgr2=0x7ff5d003e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:00.067 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.067+0000 7ff5f5600640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff5d003c300 0x7ff5d003e810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009560 tx=0x7ff5e0009190 comp rx=0 tx=0).stop 2024-04-25T12:34:00.067 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.067+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f0156930 msgr2=0x7ff5f01f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:00.067 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.067+0000 7ff5f5600640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f0156930 0x7ff5f01f34f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c7b0 tx=0x7ff5e800cc70 comp rx=0 tx=0).stop 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 shutdown_connections 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff5d003c300 0x7ff5d003e810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5f0156930 0x7ff5f01f34f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5f00c5da0 0x7ff5f00c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 --2- 172.21.15.123:0/4097181031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5f006df80 0x7ff5f00c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 >> 172.21.15.123:0/4097181031 conn(0x7ff5f0072e60 msgr2=0x7ff5f0154040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 shutdown_connections 2024-04-25T12:34:00.068 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:00.068+0000 7ff5f5600640 1 -- 172.21.15.123:0/4097181031 wait complete. 2024-04-25T12:34:00.097 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T12:34:00.597 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:34:00.598+0000 7fa6d9037040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:34:00.730 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T12:34:00.731+0000 7fa6d9037040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:34:02.397 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:34:02.397+0000 7f408aa13640 -1 osd.6 0 log_to_monitors true 2024-04-25T12:34:02.437 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:34:02.437+0000 7ff8bb933640 -1 osd.4 0 log_to_monitors true 2024-04-25T12:34:02.481 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:34:02.481+0000 7f699ec14640 -1 osd.7 0 log_to_monitors true 2024-04-25T12:34:02.484 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:34:02.485+0000 7fed0b212640 -1 osd.5 0 log_to_monitors true 2024-04-25T12:34:02.694 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:34:02.694+0000 7f8b7420e640 -1 osd.1 0 log_to_monitors true 2024-04-25T12:34:02.751 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:34:02.752+0000 7fbef5726640 -1 osd.2 0 log_to_monitors true 2024-04-25T12:34:02.831 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:34:02.832+0000 7f3e0b40f640 -1 osd.0 0 log_to_monitors true 2024-04-25T12:34:02.870 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:34:02.871+0000 7f87011dc640 -1 osd.3 0 log_to_monitors true 2024-04-25T12:34:04.905 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:34:04.904+0000 7f408628c640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T12:34:04.919 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:34:04.919+0000 7f407e087640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:04.927 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:34:04.928+0000 7ff8b78c9640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T12:34:04.933 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:34:04.933+0000 7fed06c51640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T12:34:04.942 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:34:04.943+0000 7ff8af6c4640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:04.946 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:34:04.947+0000 7fecfea4c640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:04.951 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:34:04.952+0000 7f699a4a5640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T12:34:04.969 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:34:04.969+0000 7f69922a0640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:05.065 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:34:05.065+0000 7fbef0600640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T12:34:05.068 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:34:05.067+0000 7f8b6ea00640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T12:34:05.074 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:34:05.074+0000 7fbee6600640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:05.082 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:34:05.082+0000 7f8b64a00640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:06.006 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:34:06.006+0000 7f3e06600640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T12:34:06.011 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:34:06.011+0000 7f86fc000640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T12:34:06.017 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:34:06.017+0000 7f86f2000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:06.019 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:34:06.019+0000 7f3dfbc00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:34:06.904 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:06.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:34:06.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:34:07.163 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.162+0000 7f7bf7e00640 1 Processor -- start 2024-04-25T12:34:07.163 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.162+0000 7f7bf7e00640 1 -- start start 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.164+0000 7f7bf7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7bf80c7ef0 0x7f7bf81422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf7e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7bf80c2860 0x7f7bf815c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf7e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bf80c46c0 con 0x7f7bf80c2450 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf7e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf8159cc0 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf7e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf80c2e20 con 0x7f7bf80c2860 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7bf80c7ef0 0x7f7bf81422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7bf80c7ef0 0x7f7bf81422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40184/0 (socket says 172.21.15.123:40184) 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf6a00640 1 -- 172.21.15.123:0/1678572793 learned_addr learned my addr 172.21.15.123:0/1678572793 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf7400640 1 --2- 172.21.15.123:0/1678572793 >> v2:172.21.15.153:3300/0 conn(0x7f7bf80c2860 0x7f7bf815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf6a00640 1 -- 172.21.15.123:0/1678572793 >> v2:172.21.15.153:3300/0 conn(0x7f7bf80c2860 msgr2=0x7f7bf815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf6a00640 1 --2- 172.21.15.123:0/1678572793 >> v2:172.21.15.153:3300/0 conn(0x7f7bf80c2860 0x7f7bf815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.166 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf6a00640 1 -- 172.21.15.123:0/1678572793 >> v1:172.21.15.153:6789/0 conn(0x7f7bf80c2450 legacy=0x7f7bf80ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.167 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.165+0000 7f7bf6a00640 1 -- 172.21.15.123:0/1678572793 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf8159390 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.167 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.166+0000 7f7bf6a00640 1 --2- 172.21.15.123:0/1678572793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf81422a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8159e40 tx=0x7f7be802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7a9028e7360b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:07.167 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf6000640 1 -- 172.21.15.123:0/1678572793 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8037070 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.167 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf6000640 1 -- 172.21.15.123:0/1678572793 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be802ae50 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.167 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf6000640 1 -- 172.21.15.123:0/1678572793 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8032860 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.168 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1678572793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 msgr2=0x7f7bf81422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.168 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1678572793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf81422a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8159e40 tx=0x7f7be802a300 comp rx=0 tx=0).stop 2024-04-25T12:34:07.168 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1678572793 shutdown_connections 2024-04-25T12:34:07.168 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1678572793 >> v2:172.21.15.153:3300/0 conn(0x7f7bf80c2860 0x7f7bf815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.169 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1678572793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf81422a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.169 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.167+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1678572793 >> 172.21.15.123:0/1678572793 conn(0x7f7bf8070b70 msgr2=0x7f7bf8070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:07.169 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.168+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1678572793 shutdown_connections 2024-04-25T12:34:07.169 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.168+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1678572793 wait complete. 2024-04-25T12:34:07.170 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.170+0000 7f7bf7e00640 1 Processor -- start 2024-04-25T12:34:07.170 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.170+0000 7f7bf7e00640 1 -- start start 2024-04-25T12:34:07.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.171+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7bf80c2450 0x7f7bf81f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.171+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf80c2860 0x7f7bf814b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.171+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf8138250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.171+0000 7f7bf7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8159040 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.171+0000 7f7bf7e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8159950 con 0x7f7bf80c2860 2024-04-25T12:34:07.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.171+0000 7f7bf7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf80ca940 con 0x7f7bf80c2450 2024-04-25T12:34:07.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf8138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf8138250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40188/0 (socket says 172.21.15.123:40188) 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 -- 172.21.15.123:0/1607395768 learned_addr learned my addr 172.21.15.123:0/1607395768 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf6a00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7bf80c2450 0x7f7bf81f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7befe00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf80c2860 0x7f7bf814b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 -- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7bf80c2450 msgr2=0x7f7bf81f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7bf80c2450 0x7f7bf81f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 -- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf80c2860 msgr2=0x7f7bf814b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf80c2860 0x7f7bf814b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 -- 172.21.15.123:0/1607395768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf81511a0 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7befe00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf80c2860 0x7f7bf814b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7bf7400640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf8138250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009cb0 tx=0x7f7bf000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:07.174 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.172+0000 7f7beea00640 1 -- 172.21.15.123:0/1607395768 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0018020 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.175 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.173+0000 7f7beea00640 1 -- 172.21.15.123:0/1607395768 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf0014480 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.175 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.173+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf814ba10 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.175 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.173+0000 7f7beea00640 1 -- 172.21.15.123:0/1607395768 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0014870 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.175 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.173+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf814ca90 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.175 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.174+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf8050740 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.175 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.175+0000 7f7beea00640 1 -- 172.21.15.123:0/1607395768 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f7bf001b020 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.177 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.176+0000 7f7beea00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7bd4074420 0x7f7bd4076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.177 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.177+0000 7f7beea00640 1 -- 172.21.15.123:0/1607395768 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7f7bf0095ae0 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.177 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.177+0000 7f7bf6a00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7bd4074420 0x7f7bd4076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.178+0000 7f7bf6a00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7bd4074420 0x7f7bd4076930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7be8034000 tx=0x7f7be802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:07.183 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.182+0000 7f7beea00640 1 -- 172.21.15.123:0/1607395768 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bf0022020 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.380 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.380+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bf806fd80 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.382 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.381+0000 7f7beea00640 1 -- 172.21.15.123:0/1607395768 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+12902 (secure 0 0 0) 0x7f7bf0010040 con 0x7f7bf80c7ef0 2024-04-25T12:34:07.382 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-25T12:34:07.382 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":15,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:07.013466+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7bd4074420 msgr2=0x7f7bd4076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7bd4074420 0x7f7bd4076930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7be8034000 tx=0x7f7be802e000 comp rx=0 tx=0).stop 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 msgr2=0x7f7bf8138250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf8138250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009cb0 tx=0x7f7bf000c490 comp rx=0 tx=0).stop 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 shutdown_connections 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7bd4074420 0x7f7bd4076930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf80c7ef0 0x7f7bf8138250 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf80c2860 0x7f7bf814b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 --2- 172.21.15.123:0/1607395768 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7bf80c2450 0x7f7bf81f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 >> 172.21.15.123:0/1607395768 conn(0x7f7bf8070b70 msgr2=0x7f7bf806e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:07.385 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 shutdown_connections 2024-04-25T12:34:07.386 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T12:34:07.384+0000 7f7bf7e00640 1 -- 172.21.15.123:0/1607395768 wait complete. 2024-04-25T12:34:07.405 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T12:34:07.405 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T12:34:07.405 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T12:34:07.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T12:34:07.412 INFO:tasks.ceph.mds.a:Started 2024-04-25T12:34:07.412 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T12:34:07.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T12:34:07.419 INFO:tasks.ceph.mds.c:Started 2024-04-25T12:34:07.419 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-25T12:34:07.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-25T12:34:07.426 INFO:tasks.ceph.mds.e:Started 2024-04-25T12:34:07.426 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T12:34:07.426 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T12:34:07.429 INFO:tasks.ceph.mds.b:Started 2024-04-25T12:34:07.430 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T12:34:07.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T12:34:07.432 INFO:tasks.ceph.mds.d:Started 2024-04-25T12:34:07.432 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-25T12:34:07.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-25T12:34:07.434 INFO:tasks.ceph.mds.f:Started 2024-04-25T12:34:07.435 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T12:34:07.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:07.527 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2024-04-25T12:34:07.528 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2024-04-25T12:34:07.530 INFO:tasks.ceph.mds.f.smithi153.stdout:starting mds.f at 2024-04-25T12:34:07.638 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2024-04-25T12:34:07.639 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2024-04-25T12:34:07.655 INFO:tasks.ceph.mds.e.smithi123.stdout:starting mds.e at 2024-04-25T12:34:07.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.696+0000 7f23c9000640 1 Processor -- start 2024-04-25T12:34:07.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.696+0000 7f23c9000640 1 -- start start 2024-04-25T12:34:07.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.697+0000 7f23c9000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c9000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23c4074870 0x7f23c415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c9000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f23c40c8c60 0x7f23c40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c9000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f23c40c61f0 con 0x7f23c4074870 2024-04-25T12:34:07.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c9000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f23c40c87d0 con 0x7f23c40c8c60 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c9000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f23c4073ce0 con 0x7f23c40c4d50 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23c4074870 0x7f23c415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23c4074870 0x7f23c415afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40254/0 (socket says 172.21.15.123:40254) 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 -- 172.21.15.123:0/1281905487 learned_addr learned my addr 172.21.15.123:0/1281905487 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1e00640 1 --2- 172.21.15.123:0/1281905487 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c2800640 1 --2- 172.21.15.123:0/1281905487 >> v2:172.21.15.153:3300/0 conn(0x7f23c40c8c60 0x7f23c40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 -- 172.21.15.123:0/1281905487 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 msgr2=0x7f23c4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 --2- 172.21.15.123:0/1281905487 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 -- 172.21.15.123:0/1281905487 >> v2:172.21.15.153:3300/0 conn(0x7f23c40c8c60 msgr2=0x7f23c40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 --2- 172.21.15.123:0/1281905487 >> v2:172.21.15.153:3300/0 conn(0x7f23c40c8c60 0x7f23c40c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c1400640 1 -- 172.21.15.123:0/1281905487 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c4073fc0 con 0x7f23c4074870 2024-04-25T12:34:07.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.698+0000 7f23c2800640 1 --2- 172.21.15.123:0/1281905487 >> v2:172.21.15.153:3300/0 conn(0x7f23c40c8c60 0x7f23c40c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.699+0000 7f23c1400640 1 --2- 172.21.15.123:0/1281905487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c4074870 0x7f23c415afa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23c406eda0 tx=0x7f23b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35a8f4caf8456c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.700+0000 7f23c0a00640 1 -- 172.21.15.123:0/1281905487 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0037070 con 0x7f23c4074870 2024-04-25T12:34:07.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.700+0000 7f23c0a00640 1 -- 172.21.15.123:0/1281905487 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b0032440 con 0x7f23c4074870 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.700+0000 7f23c0a00640 1 -- 172.21.15.123:0/1281905487 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0032810 con 0x7f23c4074870 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.700+0000 7f23c9000640 1 -- 172.21.15.123:0/1281905487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c4074870 msgr2=0x7f23c415afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.700+0000 7f23c9000640 1 --2- 172.21.15.123:0/1281905487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c4074870 0x7f23c415afa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23c406eda0 tx=0x7f23b002a270 comp rx=0 tx=0).stop 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.701+0000 7f23c9000640 1 -- 172.21.15.123:0/1281905487 shutdown_connections 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.701+0000 7f23c9000640 1 --2- 172.21.15.123:0/1281905487 >> v2:172.21.15.153:3300/0 conn(0x7f23c40c8c60 0x7f23c40c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.701+0000 7f23c9000640 1 --2- 172.21.15.123:0/1281905487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c4074870 0x7f23c415afa0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.701+0000 7f23c9000640 1 --2- 172.21.15.123:0/1281905487 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.701+0000 7f23c9000640 1 -- 172.21.15.123:0/1281905487 >> 172.21.15.123:0/1281905487 conn(0x7f23c406c750 msgr2=0x7f23c406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.701+0000 7f23c9000640 1 -- 172.21.15.123:0/1281905487 shutdown_connections 2024-04-25T12:34:07.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.701+0000 7f23c9000640 1 -- 172.21.15.123:0/1281905487 wait complete. 2024-04-25T12:34:07.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.703+0000 7f23c9000640 1 Processor -- start 2024-04-25T12:34:07.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.703+0000 7f23c9000640 1 -- start start 2024-04-25T12:34:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.704+0000 7f23c9000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4074870 0x7f23c40c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.704+0000 7f23c9000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c41f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c40c8c60 0x7f23c4151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23c40cda90 con 0x7f23c40c8c60 2024-04-25T12:34:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c9000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23c40c8460 con 0x7f23c4074870 2024-04-25T12:34:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c9000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f23c40cdc10 con 0x7f23c40c4d50 2024-04-25T12:34:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c40c8c60 0x7f23c4151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c1400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c41f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c40c8c60 0x7f23c4151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40258/0 (socket says 172.21.15.123:40258) 2024-04-25T12:34:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c2800640 1 -- 172.21.15.123:0/2087703636 learned_addr learned my addr 172.21.15.123:0/2087703636 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:07.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c1e00640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4074870 0x7f23c40c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c1400640 1 -- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4074870 msgr2=0x7f23c40c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c1400640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4074870 0x7f23c40c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c1400640 1 -- 172.21.15.123:0/2087703636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c40c8c60 msgr2=0x7f23c4151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c1400640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c40c8c60 0x7f23c4151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c1400640 1 -- 172.21.15.123:0/2087703636 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c41f6410 con 0x7f23c40c4d50 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.705+0000 7f23c2800640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c40c8c60 0x7f23c4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.706+0000 7f23c1400640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c41f5ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ce700 tx=0x7f23b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.706+0000 7f23ba000640 1 -- 172.21.15.123:0/2087703636 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0037040 con 0x7f23c40c4d50 2024-04-25T12:34:07.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.706+0000 7f23ba000640 1 -- 172.21.15.123:0/2087703636 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b0042070 con 0x7f23c40c4d50 2024-04-25T12:34:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.706+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c40c4350 con 0x7f23c40c4d50 2024-04-25T12:34:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.706+0000 7f23ba000640 1 -- 172.21.15.123:0/2087703636 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b003b870 con 0x7f23c40c4d50 2024-04-25T12:34:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.706+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c40c2df0 con 0x7f23c40c4d50 2024-04-25T12:34:07.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.707+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c414f890 con 0x7f23c40c4d50 2024-04-25T12:34:07.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.708+0000 7f23ba000640 1 -- 172.21.15.123:0/2087703636 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f23b003ba10 con 0x7f23c40c4d50 2024-04-25T12:34:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.710+0000 7f23ba000640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f23a4074450 0x7f23a4076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.710+0000 7f23ba000640 1 -- 172.21.15.123:0/2087703636 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7f23b00b5240 con 0x7f23c40c4d50 2024-04-25T12:34:07.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.710+0000 7f23c1e00640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f23a4074450 0x7f23a4076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:07.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.711+0000 7f23c1e00640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f23a4074450 0x7f23a4076960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009aa0 tx=0x7f23b4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:07.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.714+0000 7f23ba000640 1 -- 172.21.15.123:0/2087703636 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23c414f890 con 0x7f23c40c4d50 2024-04-25T12:34:07.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.910+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23c4076700 con 0x7f23c40c4d50 2024-04-25T12:34:07.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.912+0000 7f23ba000640 1 -- 172.21.15.123:0/2087703636 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+12902 (secure 0 0 0) 0x7f23b004c020 con 0x7f23c40c4d50 2024-04-25T12:34:07.912 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:07.913 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":15,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:07.013466+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:07.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f23a4074450 msgr2=0x7f23a4076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f23a4074450 0x7f23a4076960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009aa0 tx=0x7f23b4009190 comp rx=0 tx=0).stop 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 msgr2=0x7f23c41f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c41f5ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ce700 tx=0x7f23b00046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 shutdown_connections 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f23a4074450 0x7f23a4076960 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23c40c8c60 0x7f23c4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23c40c4d50 0x7f23c41f5ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 --2- 172.21.15.123:0/2087703636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4074870 0x7f23c40c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 >> 172.21.15.123:0/2087703636 conn(0x7f23c406c750 msgr2=0x7f23c406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 shutdown_connections 2024-04-25T12:34:07.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:07.915+0000 7f23c9000640 1 -- 172.21.15.123:0/2087703636 wait complete. 2024-04-25T12:34:07.938 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:34:07.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:34:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.205+0000 7f74e9c00640 1 Processor -- start 2024-04-25T12:34:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.206+0000 7f74e9c00640 1 -- start start 2024-04-25T12:34:08.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74e9c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e4073460 0x7f74e415d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74e9c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74e4073c10 con 0x7f74e40b2a10 2024-04-25T12:34:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74e9c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74e4074100 con 0x7f74e40ca060 2024-04-25T12:34:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74e9c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f74e415d7b0 con 0x7f74e4073460 2024-04-25T12:34:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74e3400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f74e40ca060 0x7f74e40ca470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:59742/0 (socket says 172.21.15.123:59742) 2024-04-25T12:34:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74e3400640 1 -- 172.21.15.123:0/1504649136 learned_addr learned my addr 172.21.15.123:0/1504649136 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74dbe00640 1 --1- 172.21.15.123:0/1504649136 >> v1:172.21.15.123:6789/0 conn(0x7f74e40b2a10 0x7f74e40c3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42990/0 (socket says 172.21.15.123:42990) 2024-04-25T12:34:08.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.208+0000 7f74e2a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e4073460 0x7f74e415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.209+0000 7f74e2000640 1 -- 172.21.15.123:0/1504649136 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309087626 0 0) 0x7f74e4074100 con 0x7f74e40ca060 2024-04-25T12:34:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.209+0000 7f74e2000640 1 -- 172.21.15.123:0/1504649136 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c8003520 con 0x7f74e40ca060 2024-04-25T12:34:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.209+0000 7f74e2a00640 1 -- 172.21.15.123:0/1504649136 >> v1:172.21.15.153:6789/0 conn(0x7f74e40ca060 legacy=0x7f74e40ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.209+0000 7f74e2a00640 1 -- 172.21.15.123:0/1504649136 >> v1:172.21.15.123:6789/0 conn(0x7f74e40b2a10 legacy=0x7f74e40c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.209+0000 7f74e2000640 1 -- 172.21.15.123:0/1504649136 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3612859970 0 0) 0x7f74e4073c10 con 0x7f74e40b2a10 2024-04-25T12:34:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.209+0000 7f74e2a00640 1 -- 172.21.15.123:0/1504649136 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e4070a20 con 0x7f74e4073460 2024-04-25T12:34:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.210+0000 7f74e2a00640 1 --2- 172.21.15.123:0/1504649136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e4073460 0x7f74e415d270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097d0 tx=0x7f74d402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8ebbed7f6e94cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.210+0000 7f74e2000640 1 -- 172.21.15.123:0/1504649136 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4037070 con 0x7f74e4073460 2024-04-25T12:34:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.210+0000 7f74e2000640 1 -- 172.21.15.123:0/1504649136 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d4032440 con 0x7f74e4073460 2024-04-25T12:34:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.210+0000 7f74e2000640 1 -- 172.21.15.123:0/1504649136 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4032810 con 0x7f74e4073460 2024-04-25T12:34:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.211+0000 7f74e9c00640 1 -- 172.21.15.123:0/1504649136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e4073460 msgr2=0x7f74e415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.211+0000 7f74e9c00640 1 --2- 172.21.15.123:0/1504649136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e4073460 0x7f74e415d270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097d0 tx=0x7f74d402a210 comp rx=0 tx=0).stop 2024-04-25T12:34:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.211+0000 7f74e9c00640 1 -- 172.21.15.123:0/1504649136 shutdown_connections 2024-04-25T12:34:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.211+0000 7f74e9c00640 1 --2- 172.21.15.123:0/1504649136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e4073460 0x7f74e415d270 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.211+0000 7f74e9c00640 1 -- 172.21.15.123:0/1504649136 >> 172.21.15.123:0/1504649136 conn(0x7f74e406f440 msgr2=0x7f74e406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.211+0000 7f74e9c00640 1 -- 172.21.15.123:0/1504649136 shutdown_connections 2024-04-25T12:34:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.211+0000 7f74e9c00640 1 -- 172.21.15.123:0/1504649136 wait complete. 2024-04-25T12:34:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.213+0000 7f74e9c00640 1 Processor -- start 2024-04-25T12:34:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.213+0000 7f74e9c00640 1 -- start start 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e4073460 0x7f74e415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e9c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e40b2a10 0x7f74e4154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e40ca060 0x7f74e414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4071310 con 0x7f74e40ca060 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e9c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4156fb0 con 0x7f74e4073460 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e9c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f74e4073930 con 0x7f74e40b2a10 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e40ca060 0x7f74e414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74dbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e40b2a10 0x7f74e4154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e4073460 0x7f74e415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e40ca060 0x7f74e414e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40268/0 (socket says 172.21.15.123:40268) 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.215+0000 7f74e3400640 1 -- 172.21.15.123:0/853217749 learned_addr learned my addr 172.21.15.123:0/853217749 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74dbe00640 1 -- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e4073460 msgr2=0x7f74e415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74dbe00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e4073460 0x7f74e415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74dbe00640 1 -- 172.21.15.123:0/853217749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e40ca060 msgr2=0x7f74e414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74dbe00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e40ca060 0x7f74e414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74dbe00640 1 -- 172.21.15.123:0/853217749 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e41f3250 con 0x7f74e40b2a10 2024-04-25T12:34:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74dbe00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e40b2a10 0x7f74e4154720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00c410 tx=0x7f74cc00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74daa00640 1 -- 172.21.15.123:0/853217749 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0074e0 con 0x7f74e40b2a10 2024-04-25T12:34:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74daa00640 1 -- 172.21.15.123:0/853217749 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74cc0047e0 con 0x7f74e40b2a10 2024-04-25T12:34:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74daa00640 1 -- 172.21.15.123:0/853217749 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc004bd0 con 0x7f74e40b2a10 2024-04-25T12:34:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e415c210 con 0x7f74e40b2a10 2024-04-25T12:34:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.216+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e41f2f70 con 0x7f74e40b2a10 2024-04-25T12:34:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.217+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e4076700 con 0x7f74e40b2a10 2024-04-25T12:34:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.219+0000 7f74daa00640 1 -- 172.21.15.123:0/853217749 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f74cc011070 con 0x7f74e40b2a10 2024-04-25T12:34:08.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.220+0000 7f74daa00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f74c0074450 0x7f74c0076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.220+0000 7f74daa00640 1 -- 172.21.15.123:0/853217749 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f74cc094800 con 0x7f74e40b2a10 2024-04-25T12:34:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.220+0000 7f74e2a00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f74c0074450 0x7f74c0076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.221+0000 7f74e2a00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f74c0074450 0x7f74c0076960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f74d4034000 tx=0x7f74d402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:08.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.224+0000 7f74daa00640 1 -- 172.21.15.123:0/853217749 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74cc05ee20 con 0x7f74e40b2a10 2024-04-25T12:34:08.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.422+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74e40c9b90 con 0x7f74e40b2a10 2024-04-25T12:34:08.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.423+0000 7f74daa00640 1 -- 172.21.15.123:0/853217749 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f74cc063120 con 0x7f74e40b2a10 2024-04-25T12:34:08.424 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T12:34:08.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f74c0074450 msgr2=0x7f74c0076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f74c0074450 0x7f74c0076960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f74d4034000 tx=0x7f74d402e000 comp rx=0 tx=0).stop 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e40b2a10 msgr2=0x7f74e4154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e40b2a10 0x7f74e4154720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00c410 tx=0x7f74cc00c8d0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 shutdown_connections 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f74c0074450 0x7f74c0076960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e40ca060 0x7f74e414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74e40b2a10 0x7f74e4154720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 --2- 172.21.15.123:0/853217749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e4073460 0x7f74e415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 >> 172.21.15.123:0/853217749 conn(0x7f74e406f440 msgr2=0x7f74e40b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 shutdown_connections 2024-04-25T12:34:08.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.427+0000 7f74e9c00640 1 -- 172.21.15.123:0/853217749 wait complete. 2024-04-25T12:34:08.451 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:34:08.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.730+0000 7feb03200640 1 Processor -- start 2024-04-25T12:34:08.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.730+0000 7feb03200640 1 -- start start 2024-04-25T12:34:08.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.732+0000 7feb03200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc06dad0 0x7feafc0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.732+0000 7feb03200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feafc0ca1a0 con 0x7feafc0c9c60 2024-04-25T12:34:08.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.732+0000 7feb03200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feafc0ca440 con 0x7feafc159500 2024-04-25T12:34:08.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.732+0000 7feb03200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feafc0c6d10 con 0x7feafc06dad0 2024-04-25T12:34:08.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.732+0000 7feafbe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7feafc0c9c60 0x7feafc073800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43006/0 (socket says 172.21.15.123:43006) 2024-04-25T12:34:08.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.732+0000 7feafbe00640 1 -- 172.21.15.123:0/3649987079 learned_addr learned my addr 172.21.15.123:0/3649987079 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:08.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.732+0000 7feafb400640 1 --2- 172.21.15.123:0/3649987079 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc06dad0 0x7feafc0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.733+0000 7feafaa00640 1 -- 172.21.15.123:0/3649987079 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237462721 0 0) 0x7feafc0ca1a0 con 0x7feafc0c9c60 2024-04-25T12:34:08.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.733+0000 7feafaa00640 1 -- 172.21.15.123:0/3649987079 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feae0003440 con 0x7feafc0c9c60 2024-04-25T12:34:08.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.733+0000 7feafaa00640 1 -- 172.21.15.123:0/3649987079 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383008186 0 0) 0x7feafc0ca440 con 0x7feafc159500 2024-04-25T12:34:08.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.733+0000 7feafb400640 1 -- 172.21.15.123:0/3649987079 >> v1:172.21.15.153:6789/0 conn(0x7feafc159500 legacy=0x7feafc163350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.733+0000 7feafb400640 1 -- 172.21.15.123:0/3649987079 >> v1:172.21.15.123:6789/0 conn(0x7feafc0c9c60 legacy=0x7feafc073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.733+0000 7feafb400640 1 -- 172.21.15.123:0/3649987079 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc0ca720 con 0x7feafc06dad0 2024-04-25T12:34:08.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.734+0000 7feafb400640 1 --2- 172.21.15.123:0/3649987079 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc06dad0 0x7feafc0c8920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feae80097d0 tx=0x7feae802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bcaaab25df065e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:08.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.734+0000 7feafaa00640 1 -- 172.21.15.123:0/3649987079 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8037070 con 0x7feafc06dad0 2024-04-25T12:34:08.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.735+0000 7feafaa00640 1 -- 172.21.15.123:0/3649987079 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae8032440 con 0x7feafc06dad0 2024-04-25T12:34:08.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.735+0000 7feafaa00640 1 -- 172.21.15.123:0/3649987079 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80327f0 con 0x7feafc06dad0 2024-04-25T12:34:08.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.735+0000 7feb03200640 1 -- 172.21.15.123:0/3649987079 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc06dad0 msgr2=0x7feafc0c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.735+0000 7feb03200640 1 --2- 172.21.15.123:0/3649987079 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc06dad0 0x7feafc0c8920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feae80097d0 tx=0x7feae802a210 comp rx=0 tx=0).stop 2024-04-25T12:34:08.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.735+0000 7feb03200640 1 -- 172.21.15.123:0/3649987079 shutdown_connections 2024-04-25T12:34:08.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.735+0000 7feb03200640 1 --2- 172.21.15.123:0/3649987079 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc06dad0 0x7feafc0c8920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.735+0000 7feb03200640 1 -- 172.21.15.123:0/3649987079 >> 172.21.15.123:0/3649987079 conn(0x7feafc071f10 msgr2=0x7feafc0caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:08.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.736+0000 7feb03200640 1 -- 172.21.15.123:0/3649987079 shutdown_connections 2024-04-25T12:34:08.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.736+0000 7feb03200640 1 -- 172.21.15.123:0/3649987079 wait complete. 2024-04-25T12:34:08.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.738+0000 7feb03200640 1 Processor -- start 2024-04-25T12:34:08.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.738+0000 7feb03200640 1 -- start start 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.739+0000 7feb03200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feafc06dad0 0x7feafc0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feb03200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feafc0c9c60 0x7feafc1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feb03200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc159500 0x7feafc154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feb03200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1589d0 con 0x7feafc06dad0 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feb03200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feafc159910 con 0x7feafc0c9c60 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feb03200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feafc073f10 con 0x7feafc159500 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feafc06dad0 0x7feafc0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feafc06dad0 0x7feafc0cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40270/0 (socket says 172.21.15.123:40270) 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 -- 172.21.15.123:0/3797265054 learned_addr learned my addr 172.21.15.123:0/3797265054 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafb400640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feafc0c9c60 0x7feafc1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feb01200640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc159500 0x7feafc154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 -- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc159500 msgr2=0x7feafc154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc159500 0x7feafc154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 -- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feafc0c9c60 msgr2=0x7feafc1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feafc0c9c60 0x7feafc1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.740+0000 7feafbe00640 1 -- 172.21.15.123:0/3797265054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc0cd5e0 con 0x7feafc06dad0 2024-04-25T12:34:08.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.741+0000 7feafbe00640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feafc06dad0 0x7feafc0cbe20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c390 tx=0x7feaec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:08.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.741+0000 7feaf3e00640 1 -- 172.21.15.123:0/3797265054 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec017070 con 0x7feafc06dad0 2024-04-25T12:34:08.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.741+0000 7feaf3e00640 1 -- 172.21.15.123:0/3797265054 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec005ce0 con 0x7feafc06dad0 2024-04-25T12:34:08.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.741+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc0cd8c0 con 0x7feafc06dad0 2024-04-25T12:34:08.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.741+0000 7feaf3e00640 1 -- 172.21.15.123:0/3797265054 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec0072f0 con 0x7feafc06dad0 2024-04-25T12:34:08.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.741+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc0cde80 con 0x7feafc06dad0 2024-04-25T12:34:08.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.743+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feafc076700 con 0x7feafc06dad0 2024-04-25T12:34:08.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.744+0000 7feaf3e00640 1 -- 172.21.15.123:0/3797265054 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7feaec01a070 con 0x7feafc06dad0 2024-04-25T12:34:08.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.745+0000 7feaf3e00640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7feadc074450 0x7feadc076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:08.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.745+0000 7feaf3e00640 1 -- 172.21.15.123:0/3797265054 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7feaec094100 con 0x7feafc06dad0 2024-04-25T12:34:08.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.745+0000 7feafb400640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7feadc074450 0x7feadc076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:08.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.746+0000 7feafb400640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7feadc074450 0x7feadc076960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feae8034000 tx=0x7feae802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:08.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.752+0000 7feaf3e00640 1 -- 172.21.15.123:0/3797265054 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feaec05e5d0 con 0x7feafc06dad0 2024-04-25T12:34:08.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.996+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feafc1562d0 con 0x7feafc06dad0 2024-04-25T12:34:08.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.997+0000 7feaf3e00640 1 -- 172.21.15.123:0/3797265054 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5524 (secure 0 0 0) 0x7feaec0628d0 con 0x7feafc06dad0 2024-04-25T12:34:08.998 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:08.998 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T12:34:08.998 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2024-04-25T12:34:09.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.999+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7feadc074450 msgr2=0x7feadc076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.999+0000 7feb03200640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7feadc074450 0x7feadc076960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feae8034000 tx=0x7feae802e000 comp rx=0 tx=0).stop 2024-04-25T12:34:09.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.999+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feafc06dad0 msgr2=0x7feafc0cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:08.999+0000 7feb03200640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feafc06dad0 0x7feafc0cbe20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c390 tx=0x7feaec00c850 comp rx=0 tx=0).stop 2024-04-25T12:34:09.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 shutdown_connections 2024-04-25T12:34:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7feadc074450 0x7feadc076960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feafc159500 0x7feafc154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feafc0c9c60 0x7feafc1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 --2- 172.21.15.123:0/3797265054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feafc06dad0 0x7feafc0cbe20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 >> 172.21.15.123:0/3797265054 conn(0x7feafc071f10 msgr2=0x7feafc0c5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 shutdown_connections 2024-04-25T12:34:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.000+0000 7feb03200640 1 -- 172.21.15.123:0/3797265054 wait complete. 2024-04-25T12:34:09.023 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:09.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.277+0000 7f6e9b200640 1 Processor -- start 2024-04-25T12:34:09.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.277+0000 7f6e9b200640 1 -- start start 2024-04-25T12:34:09.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e9b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415c060 0x7f6e94167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e9b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e9415e270 con 0x7f6e9415d170 2024-04-25T12:34:09.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e9b200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e940d2480 con 0x7f6e940d1d80 2024-04-25T12:34:09.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e9b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6e940d2760 con 0x7f6e9415c060 2024-04-25T12:34:09.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e99e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f6e940d1d80 0x7f6e9415db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:59762/0 (socket says 172.21.15.123:59762) 2024-04-25T12:34:09.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e99e00640 1 -- 172.21.15.123:0/1062304960 learned_addr learned my addr 172.21.15.123:0/1062304960 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:09.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e9a800640 1 --2- 172.21.15.123:0/1062304960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415c060 0x7f6e94167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.279+0000 7f6e98a00640 1 -- 172.21.15.123:0/1062304960 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461099163 0 0) 0x7f6e940d2480 con 0x7f6e940d1d80 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.280+0000 7f6e98a00640 1 -- 172.21.15.123:0/1062304960 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e7c003520 con 0x7f6e940d1d80 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.280+0000 7f6e98a00640 1 -- 172.21.15.123:0/1062304960 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158246618 0 0) 0x7f6e9415e270 con 0x7f6e9415d170 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.280+0000 7f6e98a00640 1 -- 172.21.15.123:0/1062304960 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e940d2480 con 0x7f6e9415d170 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.280+0000 7f6e9a800640 1 -- 172.21.15.123:0/1062304960 >> v1:172.21.15.153:6789/0 conn(0x7f6e940d1d80 legacy=0x7f6e9415db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.280+0000 7f6e9a800640 1 -- 172.21.15.123:0/1062304960 >> v1:172.21.15.123:6789/0 conn(0x7f6e9415d170 legacy=0x7f6e940ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.280+0000 7f6e9a800640 1 -- 172.21.15.123:0/1062304960 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e940ce260 con 0x7f6e9415c060 2024-04-25T12:34:09.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.281+0000 7f6e9a800640 1 --2- 172.21.15.123:0/1062304960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415c060 0x7f6e94167fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009dc0 tx=0x7f6e9002a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fee3730811c24b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:09.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.281+0000 7f6e98a00640 1 -- 172.21.15.123:0/1062304960 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90007740 con 0x7f6e9415c060 2024-04-25T12:34:09.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.281+0000 7f6e98a00640 1 -- 172.21.15.123:0/1062304960 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e900078e0 con 0x7f6e9415c060 2024-04-25T12:34:09.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.281+0000 7f6e98a00640 1 -- 172.21.15.123:0/1062304960 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90007cb0 con 0x7f6e9415c060 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.282+0000 7f6e9b200640 1 -- 172.21.15.123:0/1062304960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415c060 msgr2=0x7f6e94167fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.282+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1062304960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415c060 0x7f6e94167fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009dc0 tx=0x7f6e9002a760 comp rx=0 tx=0).stop 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.282+0000 7f6e9b200640 1 -- 172.21.15.123:0/1062304960 shutdown_connections 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.282+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1062304960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415c060 0x7f6e94167fa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.282+0000 7f6e9b200640 1 -- 172.21.15.123:0/1062304960 >> 172.21.15.123:0/1062304960 conn(0x7f6e9408da20 msgr2=0x7f6e9408de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.282+0000 7f6e9b200640 1 -- 172.21.15.123:0/1062304960 shutdown_connections 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.282+0000 7f6e9b200640 1 -- 172.21.15.123:0/1062304960 wait complete. 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.285+0000 7f6e9b200640 1 Processor -- start 2024-04-25T12:34:09.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.285+0000 7f6e9b200640 1 -- start start 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.287+0000 7f6e9b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e940d1d80 0x7f6e941f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.287+0000 7f6e9b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9415c060 0x7f6e94153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.287+0000 7f6e9b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415d170 0x7f6e94154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.287+0000 7f6e9b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940ce820 con 0x7f6e9415c060 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.287+0000 7f6e9b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940ceb90 con 0x7f6e940d1d80 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.287+0000 7f6e9b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6e940d2190 con 0x7f6e9415d170 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e940d1d80 0x7f6e941f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e9a800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415d170 0x7f6e94154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9415c060 0x7f6e94153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e9a800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415d170 0x7f6e94154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37314/0 (socket says 172.21.15.123:37314) 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e940d1d80 0x7f6e941f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:44848/0 (socket says 172.21.15.123:44848) 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e9a800640 1 -- 172.21.15.123:0/1075400872 learned_addr learned my addr 172.21.15.123:0/1075400872 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 -- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415d170 msgr2=0x7f6e94154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415d170 0x7f6e94154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 -- 172.21.15.123:0/1075400872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9415c060 msgr2=0x7f6e94153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9415c060 0x7f6e94153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 -- 172.21.15.123:0/1075400872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e941f3020 con 0x7f6e940d1d80 2024-04-25T12:34:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99400640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9415c060 0x7f6e94153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e9a800640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415d170 0x7f6e94154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.288+0000 7f6e99e00640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e940d1d80 0x7f6e941f2ae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c3c0 tx=0x7f6e8800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.289+0000 7f6e8e000640 1 -- 172.21.15.123:0/1075400872 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88017070 con 0x7f6e940d1d80 2024-04-25T12:34:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.289+0000 7f6e8e000640 1 -- 172.21.15.123:0/1075400872 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e88005d00 con 0x7f6e940d1d80 2024-04-25T12:34:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.289+0000 7f6e8e000640 1 -- 172.21.15.123:0/1075400872 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88007310 con 0x7f6e940d1d80 2024-04-25T12:34:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.289+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e941e4f30 con 0x7f6e940d1d80 2024-04-25T12:34:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.289+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e941f2320 con 0x7f6e940d1d80 2024-04-25T12:34:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.290+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e941f2110 con 0x7f6e940d1d80 2024-04-25T12:34:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.291+0000 7f6e8e000640 1 -- 172.21.15.123:0/1075400872 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f6e8801a070 con 0x7f6e940d1d80 2024-04-25T12:34:09.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.292+0000 7f6e8e000640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6e78074450 0x7f6e78076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.292+0000 7f6e8e000640 1 -- 172.21.15.123:0/1075400872 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6e880940c0 con 0x7f6e940d1d80 2024-04-25T12:34:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.292+0000 7f6e99400640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6e78074450 0x7f6e78076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.293+0000 7f6e99400640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6e78074450 0x7f6e78076960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e84007fd0 tx=0x7f6e84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:09.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.297+0000 7f6e8e000640 1 -- 172.21.15.123:0/1075400872 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e8805e590 con 0x7f6e940d1d80 2024-04-25T12:34:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.496+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e94076da0 con 0x7f6e940d1d80 2024-04-25T12:34:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.498+0000 7f6e8e000640 1 -- 172.21.15.123:0/1075400872 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12957 (secure 0 0 0) 0x7f6e88062890 con 0x7f6e940d1d80 2024-04-25T12:34:09.498 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:09.498 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":16,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:08.016160+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6e78074450 msgr2=0x7f6e78076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6e78074450 0x7f6e78076960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e84007fd0 tx=0x7f6e84009040 comp rx=0 tx=0).stop 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e940d1d80 msgr2=0x7f6e941f2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e940d1d80 0x7f6e941f2ae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c3c0 tx=0x7f6e8800c880 comp rx=0 tx=0).stop 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 shutdown_connections 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6e78074450 0x7f6e78076960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e9415d170 0x7f6e94154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9415c060 0x7f6e94153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 --2- 172.21.15.123:0/1075400872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e940d1d80 0x7f6e941f2ae0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 >> 172.21.15.123:0/1075400872 conn(0x7f6e9408da20 msgr2=0x7f6e9408de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:09.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 shutdown_connections 2024-04-25T12:34:09.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.500+0000 7f6e9b200640 1 -- 172.21.15.123:0/1075400872 wait complete. 2024-04-25T12:34:09.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:34:09.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:34:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.777+0000 7ff723e00640 1 Processor -- start 2024-04-25T12:34:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.778+0000 7ff723e00640 1 -- start start 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.779+0000 7ff723e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240cdd60 0x7ff7240b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.779+0000 7ff723e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7240c8c10 0x7ff724156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.779+0000 7ff723e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff72415c760 con 0x7ff72406d9b0 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.779+0000 7ff723e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff724156d40 con 0x7ff7240c8c10 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.779+0000 7ff723e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff72415ca40 con 0x7ff7240cdd60 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7240c8c10 0x7ff724156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240cdd60 0x7ff7240b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7240c8c10 0x7ff724156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:44850/0 (socket says 172.21.15.123:44850) 2024-04-25T12:34:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722000640 1 -- 172.21.15.123:0/990165007 learned_addr learned my addr 172.21.15.123:0/990165007 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722000640 1 -- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240cdd60 msgr2=0x7ff7240b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722000640 1 --2- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240cdd60 0x7ff7240b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722000640 1 -- 172.21.15.123:0/990165007 >> v1:172.21.15.123:6789/0 conn(0x7ff72406d9b0 legacy=0x7ff7240c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722000640 1 -- 172.21.15.123:0/990165007 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72415cd20 con 0x7ff7240c8c10 2024-04-25T12:34:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.780+0000 7ff722a00640 1 --2- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240cdd60 0x7ff7240b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:09.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.781+0000 7ff722000640 1 --2- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7240c8c10 0x7ff724156800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b74d6f2d9c631b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:09.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.781+0000 7ff721600640 1 -- 172.21.15.123:0/990165007 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710037070 con 0x7ff7240c8c10 2024-04-25T12:34:09.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.781+0000 7ff721600640 1 -- 172.21.15.123:0/990165007 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff710032440 con 0x7ff7240c8c10 2024-04-25T12:34:09.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.781+0000 7ff721600640 1 -- 172.21.15.123:0/990165007 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710032810 con 0x7ff7240c8c10 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 -- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7240c8c10 msgr2=0x7ff724156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 --2- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7240c8c10 0x7ff724156800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71002a250 comp rx=0 tx=0).stop 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 -- 172.21.15.123:0/990165007 shutdown_connections 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 --2- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7240c8c10 0x7ff724156800 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 --2- 172.21.15.123:0/990165007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240cdd60 0x7ff7240b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 -- 172.21.15.123:0/990165007 >> 172.21.15.123:0/990165007 conn(0x7ff7240d1810 msgr2=0x7ff7240d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 -- 172.21.15.123:0/990165007 shutdown_connections 2024-04-25T12:34:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.782+0000 7ff723e00640 1 -- 172.21.15.123:0/990165007 wait complete. 2024-04-25T12:34:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.784+0000 7ff723e00640 1 Processor -- start 2024-04-25T12:34:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.784+0000 7ff723e00640 1 -- start start 2024-04-25T12:34:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff723e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff72406d9b0 0x7ff7241f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff723e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240c8c10 0x7ff7241f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff723e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7240cdd60 0x7ff724151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff723e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7240ca5b0 con 0x7ff7240cdd60 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff723e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff72415d2e0 con 0x7ff72406d9b0 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff723e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff7240ca730 con 0x7ff7240c8c10 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff72406d9b0 0x7ff7241f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240c8c10 0x7ff7241f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240c8c10 0x7ff7241f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37330/0 (socket says 172.21.15.123:37330) 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 -- 172.21.15.123:0/2974000736 learned_addr learned my addr 172.21.15.123:0/2974000736 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff72406d9b0 0x7ff7241f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:44854/0 (socket says 172.21.15.123:44854) 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff723400640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7240cdd60 0x7ff724151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 -- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff72406d9b0 msgr2=0x7ff7241f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff72406d9b0 0x7ff7241f2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 -- 172.21.15.123:0/2974000736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7240cdd60 msgr2=0x7ff724151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7240cdd60 0x7ff724151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722000640 1 -- 172.21.15.123:0/2974000736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241f34d0 con 0x7ff7240c8c10 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.786+0000 7ff722a00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff72406d9b0 0x7ff7241f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.787+0000 7ff722000640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240c8c10 0x7ff7241f6450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff710000c00 tx=0x7ff7100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.787+0000 7ff71aa00640 1 -- 172.21.15.123:0/2974000736 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710037040 con 0x7ff7240c8c10 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.787+0000 7ff71aa00640 1 -- 172.21.15.123:0/2974000736 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff710042070 con 0x7ff7240c8c10 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.787+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7241e5e70 con 0x7ff7240c8c10 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.787+0000 7ff71aa00640 1 -- 172.21.15.123:0/2974000736 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710030880 con 0x7ff7240c8c10 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.787+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241f6990 con 0x7ff7240c8c10 2024-04-25T12:34:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.788+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7240cb9d0 con 0x7ff7240c8c10 2024-04-25T12:34:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.790+0000 7ff71aa00640 1 -- 172.21.15.123:0/2974000736 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff710030a20 con 0x7ff7240c8c10 2024-04-25T12:34:09.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.790+0000 7ff71aa00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7040744b0 0x7ff7040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:09.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.791+0000 7ff71aa00640 1 -- 172.21.15.123:0/2974000736 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff7100b5bd0 con 0x7ff7240c8c10 2024-04-25T12:34:09.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.791+0000 7ff722a00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7040744b0 0x7ff7040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:09.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.791+0000 7ff722a00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7040744b0 0x7ff7040769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff714009500 tx=0x7ff714009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.795+0000 7ff71aa00640 1 -- 172.21.15.123:0/2974000736 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7240cb9d0 con 0x7ff7240c8c10 2024-04-25T12:34:09.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.995+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff724076da0 con 0x7ff7240c8c10 2024-04-25T12:34:09.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:09.996+0000 7ff71aa00640 1 -- 172.21.15.123:0/2974000736 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7ff7100800d0 con 0x7ff7240c8c10 2024-04-25T12:34:09.997 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7040744b0 msgr2=0x7ff7040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7040744b0 0x7ff7040769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff714009500 tx=0x7ff714009110 comp rx=0 tx=0).stop 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240c8c10 msgr2=0x7ff7241f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240c8c10 0x7ff7241f6450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff710000c00 tx=0x7ff7100046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 shutdown_connections 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7040744b0 0x7ff7040769c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7240cdd60 0x7ff724151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7240c8c10 0x7ff7241f6450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 --2- 172.21.15.123:0/2974000736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff72406d9b0 0x7ff7241f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 >> 172.21.15.123:0/2974000736 conn(0x7ff7240d1810 msgr2=0x7ff7240c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 shutdown_connections 2024-04-25T12:34:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.000+0000 7ff723e00640 1 -- 172.21.15.123:0/2974000736 wait complete. 2024-04-25T12:34:10.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T12:34:10.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.285+0000 7fa9d6400640 1 Processor -- start 2024-04-25T12:34:10.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.286+0000 7fa9d6400640 1 -- start start 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d6400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d01568c0 0x7fa9d00ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d6400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa9d00cdca0 0x7fa9d00ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d6400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d00cedf0 con 0x7fa9d00c6800 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d6400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d00cebd0 con 0x7fa9d00cdca0 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d6400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d00cf0d0 con 0x7fa9d01568c0 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9cfe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa9d00cdca0 0x7fa9d00ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d5a00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa9d00c6800 0x7fa9d015c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:59768/0 (socket says 172.21.15.123:59768) 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d5a00640 1 -- 172.21.15.123:0/3877417117 learned_addr learned my addr 172.21.15.123:0/3877417117 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:10.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.288+0000 7fa9d5000640 1 --2- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d01568c0 0x7fa9d00ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.289+0000 7fa9cf400640 1 -- 172.21.15.123:0/3877417117 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854624912 0 0) 0x7fa9d00cedf0 con 0x7fa9d00c6800 2024-04-25T12:34:10.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.289+0000 7fa9cf400640 1 -- 172.21.15.123:0/3877417117 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9b4003520 con 0x7fa9d00c6800 2024-04-25T12:34:10.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.289+0000 7fa9cfe00640 1 -- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d01568c0 msgr2=0x7fa9d00ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:10.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.289+0000 7fa9cfe00640 1 --2- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d01568c0 0x7fa9d00ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.289+0000 7fa9cfe00640 1 -- 172.21.15.123:0/3877417117 >> v1:172.21.15.153:6789/0 conn(0x7fa9d00c6800 legacy=0x7fa9d015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.289+0000 7fa9cfe00640 1 -- 172.21.15.123:0/3877417117 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d00c7df0 con 0x7fa9d00cdca0 2024-04-25T12:34:10.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.289+0000 7fa9d5000640 1 --2- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d01568c0 0x7fa9d00ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:10.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.290+0000 7fa9cfe00640 1 --2- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d00ccf20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009d90 tx=0x7fa9c002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121192a6371faa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:10.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.290+0000 7fa9cf400640 1 -- 172.21.15.123:0/3877417117 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007740 con 0x7fa9d00cdca0 2024-04-25T12:34:10.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.290+0000 7fa9cf400640 1 -- 172.21.15.123:0/3877417117 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c00078e0 con 0x7fa9d00cdca0 2024-04-25T12:34:10.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.290+0000 7fa9cf400640 1 -- 172.21.15.123:0/3877417117 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007c90 con 0x7fa9d00cdca0 2024-04-25T12:34:10.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 -- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 msgr2=0x7fa9d00ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d00ccf20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009d90 tx=0x7fa9c002a660 comp rx=0 tx=0).stop 2024-04-25T12:34:10.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 -- 172.21.15.123:0/3877417117 shutdown_connections 2024-04-25T12:34:10.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d00ccf20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3877417117 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d01568c0 0x7fa9d00ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 -- 172.21.15.123:0/3877417117 >> 172.21.15.123:0/3877417117 conn(0x7fa9d006dcd0 msgr2=0x7fa9d006e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:10.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 -- 172.21.15.123:0/3877417117 shutdown_connections 2024-04-25T12:34:10.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.291+0000 7fa9d6400640 1 -- 172.21.15.123:0/3877417117 wait complete. 2024-04-25T12:34:10.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.293+0000 7fa9d6400640 1 Processor -- start 2024-04-25T12:34:10.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.293+0000 7fa9d6400640 1 -- start start 2024-04-25T12:34:10.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d6400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d00c6800 0x7fa9d00c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d6400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d015b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d6400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d0159430 0x7fa9d014c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d6400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d00c83b0 con 0x7fa9d00c6800 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d6400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d00c8720 con 0x7fa9d00cdca0 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d6400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d015cb00 con 0x7fa9d0159430 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d00c6800 0x7fa9d00c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d00c6800 0x7fa9d00c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40300/0 (socket says 172.21.15.123:40300) 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 -- 172.21.15.123:0/3668988444 learned_addr learned my addr 172.21.15.123:0/3668988444 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9cfe00640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5a00640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d0159430 0x7fa9d014c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 -- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d0159430 msgr2=0x7fa9d014c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d0159430 0x7fa9d014c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 -- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 msgr2=0x7fa9d015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d015b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9d5000640 1 -- 172.21.15.123:0/3668988444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d00c9640 con 0x7fa9d00c6800 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.295+0000 7fa9cfe00640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d015b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.296+0000 7fa9d5000640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d00c6800 0x7fa9d00c9100 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800e960 tx=0x7fa9b800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.296+0000 7fa9ccc00640 1 -- 172.21.15.123:0/3668988444 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8017070 con 0x7fa9d00c6800 2024-04-25T12:34:10.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.296+0000 7fa9ccc00640 1 -- 172.21.15.123:0/3668988444 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b800ca00 con 0x7fa9d00c6800 2024-04-25T12:34:10.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.296+0000 7fa9ccc00640 1 -- 172.21.15.123:0/3668988444 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800cdd0 con 0x7fa9d00c6800 2024-04-25T12:34:10.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.296+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d014e8a0 con 0x7fa9d00c6800 2024-04-25T12:34:10.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.296+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d0156e80 con 0x7fa9d00c6800 2024-04-25T12:34:10.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.297+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d0076da0 con 0x7fa9d00c6800 2024-04-25T12:34:10.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.299+0000 7fa9ccc00640 1 -- 172.21.15.123:0/3668988444 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa9b801a070 con 0x7fa9d00c6800 2024-04-25T12:34:10.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.299+0000 7fa9ccc00640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9ac0744b0 0x7fa9ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.300+0000 7fa9ccc00640 1 -- 172.21.15.123:0/3668988444 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa9b8095060 con 0x7fa9d00c6800 2024-04-25T12:34:10.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.300+0000 7fa9cfe00640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9ac0744b0 0x7fa9ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.301+0000 7fa9cfe00640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9ac0744b0 0x7fa9ac0769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0034000 tx=0x7fa9c002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:10.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.305+0000 7fa9ccc00640 1 -- 172.21.15.123:0/3668988444 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9b805f590 con 0x7fa9d00c6800 2024-04-25T12:34:10.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.505+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa9d00b4870 con 0x7fa9d00c6800 2024-04-25T12:34:10.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.506+0000 7fa9ccc00640 1 -- 172.21.15.123:0/3668988444 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7fa9b8063890 con 0x7fa9d00c6800 2024-04-25T12:34:10.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:10.506 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-04-25T12:34:10.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:34:10.507 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-04-25T12:34:10.507 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-04-25T12:34:10.507 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:34:10.507 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9ac0744b0 msgr2=0x7fa9ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9ac0744b0 0x7fa9ac0769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0034000 tx=0x7fa9c002f000 comp rx=0 tx=0).stop 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d00c6800 msgr2=0x7fa9d00c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d00c6800 0x7fa9d00c9100 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800e960 tx=0x7fa9b800ee20 comp rx=0 tx=0).stop 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 shutdown_connections 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9ac0744b0 0x7fa9ac0769c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9d0159430 0x7fa9d014c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9d00cdca0 0x7fa9d015b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 --2- 172.21.15.123:0/3668988444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d00c6800 0x7fa9d00c9100 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 >> 172.21.15.123:0/3668988444 conn(0x7fa9d006dcd0 msgr2=0x7fa9d0154170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 shutdown_connections 2024-04-25T12:34:10.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.508+0000 7fa9d6400640 1 -- 172.21.15.123:0/3668988444 wait complete. 2024-04-25T12:34:10.532 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T12:34:10.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T12:34:10.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.803+0000 7fab76600640 1 Processor -- start 2024-04-25T12:34:10.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.803+0000 7fab76600640 1 -- start start 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.805+0000 7fab76600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fab700c85b0 0x7fab700c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.805+0000 7fab76600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c97d0 0x7fab70163a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.805+0000 7fab76600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab700c9040 con 0x7fab700c93c0 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.805+0000 7fab76600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fab7015c700 con 0x7fab700c85b0 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.805+0000 7fab76600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fab7015c950 con 0x7fab700c97d0 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab75200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fab700c85b0 0x7fab700c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab75c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c97d0 0x7fab70163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab6fe00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fab700c93c0 0x7fab70159110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:59776/0 (socket says 172.21.15.123:59776) 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab75200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fab700c85b0 0x7fab700c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:44894/0 (socket says 172.21.15.123:44894) 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab6fe00640 1 -- 172.21.15.123:0/876910352 learned_addr learned my addr 172.21.15.123:0/876910352 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:10.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab75200640 1 -- 172.21.15.123:0/876910352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c97d0 msgr2=0x7fab70163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:10.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab75200640 1 --2- 172.21.15.123:0/876910352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c97d0 0x7fab70163a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab75200640 1 -- 172.21.15.123:0/876910352 >> v1:172.21.15.153:6789/0 conn(0x7fab700c93c0 legacy=0x7fab70159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab75200640 1 -- 172.21.15.123:0/876910352 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7015cc30 con 0x7fab700c85b0 2024-04-25T12:34:10.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.806+0000 7fab6f400640 1 -- 172.21.15.123:0/876910352 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 79544060 0 0) 0x7fab700c9040 con 0x7fab700c93c0 2024-04-25T12:34:10.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.807+0000 7fab75200640 1 --2- 172.21.15.123:0/876910352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c85b0 0x7fab700c7530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab60009d90 tx=0x7fab6002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563dc280886359b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:10.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.807+0000 7fab6f400640 1 -- 172.21.15.123:0/876910352 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60007740 con 0x7fab700c85b0 2024-04-25T12:34:10.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.807+0000 7fab6f400640 1 -- 172.21.15.123:0/876910352 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab600078e0 con 0x7fab700c85b0 2024-04-25T12:34:10.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.807+0000 7fab6f400640 1 -- 172.21.15.123:0/876910352 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60007cb0 con 0x7fab700c85b0 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 -- 172.21.15.123:0/876910352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c85b0 msgr2=0x7fab700c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 --2- 172.21.15.123:0/876910352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c85b0 0x7fab700c7530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab60009d90 tx=0x7fab6002a6f0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 -- 172.21.15.123:0/876910352 shutdown_connections 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 --2- 172.21.15.123:0/876910352 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c97d0 0x7fab70163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 --2- 172.21.15.123:0/876910352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c85b0 0x7fab700c7530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 -- 172.21.15.123:0/876910352 >> 172.21.15.123:0/876910352 conn(0x7fab700c9f60 msgr2=0x7fab700ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 -- 172.21.15.123:0/876910352 shutdown_connections 2024-04-25T12:34:10.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.808+0000 7fab76600640 1 -- 172.21.15.123:0/876910352 wait complete. 2024-04-25T12:34:10.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.810+0000 7fab76600640 1 Processor -- start 2024-04-25T12:34:10.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.811+0000 7fab76600640 1 -- start start 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab76600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c85b0 0x7fab700cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab76600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c93c0 0x7fab701e4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab76600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab700c97d0 0x7fab7006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab76600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab700c80d0 con 0x7fab700c97d0 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab76600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab7015d1f0 con 0x7fab700c93c0 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab76600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fab700c9be0 con 0x7fab700c85b0 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab75200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c85b0 0x7fab700cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab6fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c93c0 0x7fab701e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab75200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c85b0 0x7fab700cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37364/0 (socket says 172.21.15.123:37364) 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab75c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab700c97d0 0x7fab7006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.812+0000 7fab75200640 1 -- 172.21.15.123:0/1347843215 learned_addr learned my addr 172.21.15.123:0/1347843215 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6fe00640 1 -- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c85b0 msgr2=0x7fab700cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6fe00640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c85b0 0x7fab700cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6fe00640 1 -- 172.21.15.123:0/1347843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab700c97d0 msgr2=0x7fab7006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6fe00640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab700c97d0 0x7fab7006ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6fe00640 1 -- 172.21.15.123:0/1347843215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab701f2ef0 con 0x7fab700c93c0 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab75200640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c85b0 0x7fab700cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab75c00640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab700c97d0 0x7fab7006ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6fe00640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c93c0 0x7fab701e4020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c440 tx=0x7fab5c00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6cc00640 1 -- 172.21.15.123:0/1347843215 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0074e0 con 0x7fab700c93c0 2024-04-25T12:34:10.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6cc00640 1 -- 172.21.15.123:0/1347843215 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c0047e0 con 0x7fab700c93c0 2024-04-25T12:34:10.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab6cc00640 1 -- 172.21.15.123:0/1347843215 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c004bd0 con 0x7fab700c93c0 2024-04-25T12:34:10.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.813+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab701f31b0 con 0x7fab700c93c0 2024-04-25T12:34:10.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.814+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab701f3770 con 0x7fab700c93c0 2024-04-25T12:34:10.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.815+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab700706a0 con 0x7fab700c93c0 2024-04-25T12:34:10.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.816+0000 7fab6cc00640 1 -- 172.21.15.123:0/1347843215 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab5c011070 con 0x7fab700c93c0 2024-04-25T12:34:10.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.817+0000 7fab6cc00640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab500744e0 0x7fab500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:10.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.817+0000 7fab6cc00640 1 -- 172.21.15.123:0/1347843215 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fab5c094a00 con 0x7fab700c93c0 2024-04-25T12:34:10.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.817+0000 7fab75200640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab500744e0 0x7fab500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:10.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.818+0000 7fab75200640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab500744e0 0x7fab500769f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab60008000 tx=0x7fab6002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:10.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:10.822+0000 7fab6cc00640 1 -- 172.21.15.123:0/1347843215 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab700706a0 con 0x7fab700c93c0 2024-04-25T12:34:11.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.010+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fab700caab0 con 0x7fab700c93c0 2024-04-25T12:34:11.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.880+0000 7fab6cc00640 1 -- 172.21.15.123:0/1347843215 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fab700caab0 con 0x7fab700c93c0 2024-04-25T12:34:11.881 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-04-25T12:34:11.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.884+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab500744e0 msgr2=0x7fab500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:11.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.884+0000 7fab76600640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab500744e0 0x7fab500769f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab60008000 tx=0x7fab6002f000 comp rx=0 tx=0).stop 2024-04-25T12:34:11.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.884+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c93c0 msgr2=0x7fab701e4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:11.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.884+0000 7fab76600640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c93c0 0x7fab701e4020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c440 tx=0x7fab5c00c900 comp rx=0 tx=0).stop 2024-04-25T12:34:11.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.885+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 shutdown_connections 2024-04-25T12:34:11.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.885+0000 7fab76600640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab500744e0 0x7fab500769f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:11.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.885+0000 7fab76600640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab700c97d0 0x7fab7006ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:11.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.885+0000 7fab76600640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab700c93c0 0x7fab701e4020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:11.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.885+0000 7fab76600640 1 --2- 172.21.15.123:0/1347843215 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab700c85b0 0x7fab700cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:11.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.885+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 >> 172.21.15.123:0/1347843215 conn(0x7fab700c9f60 msgr2=0x7fab701504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:11.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.886+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 shutdown_connections 2024-04-25T12:34:11.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:11.887+0000 7fab76600640 1 -- 172.21.15.123:0/1347843215 wait complete. 2024-04-25T12:34:11.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T12:34:12.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.170+0000 7f0aca000640 1 Processor -- start 2024-04-25T12:34:12.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.170+0000 7f0aca000640 1 -- start start 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0aca000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0ac40c8c10 0x7f0ac40b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0aca000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0ac4156800 0x7f0ac40c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0aca000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac415c760 con 0x7f0ac40cdd60 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0aca000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac415ca40 con 0x7f0ac4156800 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0aca000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac415cd20 con 0x7f0ac40c8c10 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0ac9600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0ac4156800 0x7f0ac40c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0ac9600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0ac4156800 0x7f0ac40c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40314/0 (socket says 172.21.15.123:40314) 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0ac9600640 1 -- 172.21.15.123:0/3253524115 learned_addr learned my addr 172.21.15.123:0/3253524115 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:12.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.172+0000 7f0ac3e00640 1 --2- 172.21.15.123:0/3253524115 >> v2:172.21.15.153:3300/0 conn(0x7f0ac40c8c10 0x7f0ac40b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:12.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.173+0000 7f0ac9600640 1 -- 172.21.15.123:0/3253524115 >> v2:172.21.15.153:3300/0 conn(0x7f0ac40c8c10 msgr2=0x7f0ac40b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:12.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.173+0000 7f0ac9600640 1 --2- 172.21.15.123:0/3253524115 >> v2:172.21.15.153:3300/0 conn(0x7f0ac40c8c10 0x7f0ac40b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.173+0000 7f0ac9600640 1 -- 172.21.15.123:0/3253524115 >> v1:172.21.15.123:6789/0 conn(0x7f0ac40cdd60 legacy=0x7f0ac406d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:12.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.173+0000 7f0ac9600640 1 -- 172.21.15.123:0/3253524115 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac415d000 con 0x7f0ac4156800 2024-04-25T12:34:12.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.174+0000 7f0ac9600640 1 --2- 172.21.15.123:0/3253524115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4156800 0x7f0ac40c9ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40c8890 tx=0x7f0abc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab8abfe9966464f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:12.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.174+0000 7f0ac3400640 1 -- 172.21.15.123:0/3253524115 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc036070 con 0x7f0ac4156800 2024-04-25T12:34:12.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.174+0000 7f0ac3400640 1 -- 172.21.15.123:0/3253524115 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc031600 con 0x7f0ac4156800 2024-04-25T12:34:12.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.174+0000 7f0ac3400640 1 -- 172.21.15.123:0/3253524115 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0319d0 con 0x7f0ac4156800 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 -- 172.21.15.123:0/3253524115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4156800 msgr2=0x7f0ac40c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 --2- 172.21.15.123:0/3253524115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4156800 0x7f0ac40c9ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40c8890 tx=0x7f0abc02a730 comp rx=0 tx=0).stop 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 -- 172.21.15.123:0/3253524115 shutdown_connections 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 --2- 172.21.15.123:0/3253524115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4156800 0x7f0ac40c9ea0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 --2- 172.21.15.123:0/3253524115 >> v2:172.21.15.153:3300/0 conn(0x7f0ac40c8c10 0x7f0ac40b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 -- 172.21.15.123:0/3253524115 >> 172.21.15.123:0/3253524115 conn(0x7f0ac40d1810 msgr2=0x7f0ac40d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 -- 172.21.15.123:0/3253524115 shutdown_connections 2024-04-25T12:34:12.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.175+0000 7f0aca000640 1 -- 172.21.15.123:0/3253524115 wait complete. 2024-04-25T12:34:12.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.177+0000 7f0aca000640 1 Processor -- start 2024-04-25T12:34:12.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.178+0000 7f0aca000640 1 -- start start 2024-04-25T12:34:12.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0aca000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ac40c8c10 0x7f0ac4159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:12.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0aca000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac40cdd60 0x7f0ac41576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:12.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0aca000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac4156800 0x7f0ac4153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:12.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0aca000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac406e0c0 con 0x7f0ac40cdd60 2024-04-25T12:34:12.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0aca000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4156c10 con 0x7f0ac4156800 2024-04-25T12:34:12.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0aca000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac406e240 con 0x7f0ac40c8c10 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0ac3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac40cdd60 0x7f0ac41576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0ac3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac40cdd60 0x7f0ac41576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40316/0 (socket says 172.21.15.123:40316) 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0ac8c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ac40c8c10 0x7f0ac4159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0ac3e00640 1 -- 172.21.15.123:0/590021957 learned_addr learned my addr 172.21.15.123:0/590021957 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.179+0000 7f0ac9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac4156800 0x7f0ac4153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac3e00640 1 -- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ac40c8c10 msgr2=0x7f0ac4159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac3e00640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ac40c8c10 0x7f0ac4159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac3e00640 1 -- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac4156800 msgr2=0x7f0ac4153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac3e00640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac4156800 0x7f0ac4153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac3e00640 1 -- 172.21.15.123:0/590021957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac4159ed0 con 0x7f0ac40cdd60 2024-04-25T12:34:12.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac3e00640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac40cdd60 0x7f0ac41576b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000e8a0 tx=0x7f0ab000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:12.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac0c00640 1 -- 172.21.15.123:0/590021957 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0017070 con 0x7f0ac40cdd60 2024-04-25T12:34:12.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac0c00640 1 -- 172.21.15.123:0/590021957 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab000ee90 con 0x7f0ac40cdd60 2024-04-25T12:34:12.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac4158930 con 0x7f0ac40cdd60 2024-04-25T12:34:12.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0ac0c00640 1 -- 172.21.15.123:0/590021957 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000cb40 con 0x7f0ac40cdd60 2024-04-25T12:34:12.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.180+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac4158eb0 con 0x7f0ac40cdd60 2024-04-25T12:34:12.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.181+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac4076da0 con 0x7f0ac40cdd60 2024-04-25T12:34:12.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.184+0000 7f0ac0c00640 1 -- 172.21.15.123:0/590021957 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ab001a070 con 0x7f0ac40cdd60 2024-04-25T12:34:12.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.185+0000 7f0ac0c00640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0aa40744b0 0x7f0aa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:12.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.185+0000 7f0ac0c00640 1 -- 172.21.15.123:0/590021957 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f0ab0095600 con 0x7f0ac40cdd60 2024-04-25T12:34:12.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.185+0000 7f0ac8c00640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0aa40744b0 0x7f0aa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:12.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.186+0000 7f0ac8c00640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0aa40744b0 0x7f0aa40769c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80016d0 tx=0x7f0ab8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:12.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.189+0000 7f0ac0c00640 1 -- 172.21.15.123:0/590021957 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ab005fa00 con 0x7f0ac40cdd60 2024-04-25T12:34:12.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.372+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f0ac40cdad0 con 0x7f0ac40cdd60 2024-04-25T12:34:12.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.881+0000 7f0ac0c00640 1 -- 172.21.15.123:0/590021957 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f0ab0063d00 con 0x7f0ac40cdd60 2024-04-25T12:34:12.882 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-04-25T12:34:12.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0aa40744b0 msgr2=0x7f0aa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:12.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0aa40744b0 0x7f0aa40769c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80016d0 tx=0x7f0ab8009040 comp rx=0 tx=0).stop 2024-04-25T12:34:12.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac40cdd60 msgr2=0x7f0ac41576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:12.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac40cdd60 0x7f0ac41576b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000e8a0 tx=0x7f0ab000ed60 comp rx=0 tx=0).stop 2024-04-25T12:34:12.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 shutdown_connections 2024-04-25T12:34:12.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0aa40744b0 0x7f0aa40769c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac4156800 0x7f0ac4153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac40cdd60 0x7f0ac41576b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 --2- 172.21.15.123:0/590021957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0ac40c8c10 0x7f0ac4159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:12.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.887+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 >> 172.21.15.123:0/590021957 conn(0x7f0ac40d1810 msgr2=0x7f0ac40c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.889+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 shutdown_connections 2024-04-25T12:34:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:12.890+0000 7f0aca000640 1 -- 172.21.15.123:0/590021957 wait complete. 2024-04-25T12:34:12.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T12:34:13.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.177+0000 7f14d1400640 1 Processor -- start 2024-04-25T12:34:13.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.178+0000 7f14d1400640 1 -- start start 2024-04-25T12:34:13.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14d1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14cc06f700 0x7f14cc1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:13.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14d1400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14cc15e680 con 0x7f14cc15ac20 2024-04-25T12:34:13.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14d1400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14cc06fc00 con 0x7f14cc06c570 2024-04-25T12:34:13.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14d1400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f14cc06fee0 con 0x7f14cc06f700 2024-04-25T12:34:13.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14cbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14cc06f700 0x7f14cc1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:13.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14c3e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f14cc15ac20 0x7f14cc0c9180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55898/0 (socket says 172.21.15.123:55898) 2024-04-25T12:34:13.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14cbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14cc06f700 0x7f14cc1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41156/0 (socket says 172.21.15.123:41156) 2024-04-25T12:34:13.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14c3e00640 1 -- 172.21.15.123:0/1305058976 learned_addr learned my addr 172.21.15.123:0/1305058976 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:13.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.180+0000 7f14cb400640 1 -- 172.21.15.123:0/1305058976 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671122671 0 0) 0x7f14cc06fc00 con 0x7f14cc06c570 2024-04-25T12:34:13.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.181+0000 7f14cb400640 1 -- 172.21.15.123:0/1305058976 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14b0003520 con 0x7f14cc06c570 2024-04-25T12:34:13.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.181+0000 7f14cbe00640 1 -- 172.21.15.123:0/1305058976 >> v1:172.21.15.153:6789/0 conn(0x7f14cc06c570 legacy=0x7f14cc164b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:13.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.181+0000 7f14cbe00640 1 -- 172.21.15.123:0/1305058976 >> v1:172.21.15.123:6789/0 conn(0x7f14cc15ac20 legacy=0x7f14cc0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:13.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.181+0000 7f14cbe00640 1 -- 172.21.15.123:0/1305058976 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14cc15d8c0 con 0x7f14cc06f700 2024-04-25T12:34:13.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.182+0000 7f14cbe00640 1 --2- 172.21.15.123:0/1305058976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06f700 0x7f14cc1542d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0097a0 tx=0x7f14bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5b6e6467ed3d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:13.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.182+0000 7f14cb400640 1 -- 172.21.15.123:0/1305058976 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc037070 con 0x7f14cc06f700 2024-04-25T12:34:13.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.182+0000 7f14cb400640 1 -- 172.21.15.123:0/1305058976 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc032440 con 0x7f14cc06f700 2024-04-25T12:34:13.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.182+0000 7f14cb400640 1 -- 172.21.15.123:0/1305058976 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0327f0 con 0x7f14cc06f700 2024-04-25T12:34:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.183+0000 7f14d1400640 1 -- 172.21.15.123:0/1305058976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06f700 msgr2=0x7f14cc1542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.183+0000 7f14d1400640 1 --2- 172.21.15.123:0/1305058976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06f700 0x7f14cc1542d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0097a0 tx=0x7f14bc02a230 comp rx=0 tx=0).stop 2024-04-25T12:34:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.183+0000 7f14d1400640 1 -- 172.21.15.123:0/1305058976 shutdown_connections 2024-04-25T12:34:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.183+0000 7f14d1400640 1 --2- 172.21.15.123:0/1305058976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06f700 0x7f14cc1542d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.183+0000 7f14d1400640 1 -- 172.21.15.123:0/1305058976 >> 172.21.15.123:0/1305058976 conn(0x7f14cc06e070 msgr2=0x7f14cc0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.183+0000 7f14d1400640 1 -- 172.21.15.123:0/1305058976 shutdown_connections 2024-04-25T12:34:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.183+0000 7f14d1400640 1 -- 172.21.15.123:0/1305058976 wait complete. 2024-04-25T12:34:13.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.185+0000 7f14d1400640 1 Processor -- start 2024-04-25T12:34:13.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.185+0000 7f14d1400640 1 -- start start 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.186+0000 7f14d1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06c570 0x7f14cc1f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.186+0000 7f14d1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14cc06f700 0x7f14cc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.186+0000 7f14d1400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14cc15ac20 0x7f14cc0715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14d1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc15e160 con 0x7f14cc06c570 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14d1400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc15de80 con 0x7f14cc06f700 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14d1400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f14cc0c9890 con 0x7f14cc15ac20 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06c570 0x7f14cc1f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06c570 0x7f14cc1f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41162/0 (socket says 172.21.15.123:41162) 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 -- 172.21.15.123:0/1047858021 learned_addr learned my addr 172.21.15.123:0/1047858021 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14c3e00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14cc06f700 0x7f14cc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14d0a00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14cc15ac20 0x7f14cc0715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 -- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14cc15ac20 msgr2=0x7f14cc0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14cc15ac20 0x7f14cc0715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 -- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14cc06f700 msgr2=0x7f14cc15c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:13.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14cc06f700 0x7f14cc15c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 -- 172.21.15.123:0/1047858021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14cc145070 con 0x7f14cc06c570 2024-04-25T12:34:13.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.187+0000 7f14cbe00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06c570 0x7f14cc1f0f70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14bc035040 tx=0x7f14bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:13.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.188+0000 7f14c8c00640 1 -- 172.21.15.123:0/1047858021 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc037050 con 0x7f14cc06c570 2024-04-25T12:34:13.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.188+0000 7f14c8c00640 1 -- 172.21.15.123:0/1047858021 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc043070 con 0x7f14cc06c570 2024-04-25T12:34:13.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.188+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14cc1f14b0 con 0x7f14cc06c570 2024-04-25T12:34:13.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.188+0000 7f14c8c00640 1 -- 172.21.15.123:0/1047858021 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc032810 con 0x7f14cc06c570 2024-04-25T12:34:13.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.188+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14cc0cde70 con 0x7f14cc06c570 2024-04-25T12:34:13.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.189+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14cc1f1680 con 0x7f14cc06c570 2024-04-25T12:34:13.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.190+0000 7f14c8c00640 1 -- 172.21.15.123:0/1047858021 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14bc0329b0 con 0x7f14cc06c570 2024-04-25T12:34:13.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.191+0000 7f14c8c00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f14a80744b0 0x7f14a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:13.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.191+0000 7f14c8c00640 1 -- 172.21.15.123:0/1047858021 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f14bc0b5c50 con 0x7f14cc06c570 2024-04-25T12:34:13.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.191+0000 7f14c3e00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f14a80744b0 0x7f14a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:13.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.192+0000 7f14c3e00640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f14a80744b0 0x7f14a80769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14cc1eb040 tx=0x7f14b4007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:13.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.197+0000 7f14c8c00640 1 -- 172.21.15.123:0/1047858021 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14cc1f1680 con 0x7f14cc06c570 2024-04-25T12:34:13.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.422+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f14cc076da0 con 0x7f14cc06c570 2024-04-25T12:34:13.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.934+0000 7f14c8c00640 1 -- 172.21.15.123:0/1047858021 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f14cc145070 con 0x7f14cc06c570 2024-04-25T12:34:13.939 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T12:34:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.942+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f14a80744b0 msgr2=0x7f14a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:13.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.942+0000 7f14d1400640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f14a80744b0 0x7f14a80769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14cc1eb040 tx=0x7f14b4007e80 comp rx=0 tx=0).stop 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.942+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06c570 msgr2=0x7f14cc1f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.942+0000 7f14d1400640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06c570 0x7f14cc1f0f70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14bc035040 tx=0x7f14bc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.943+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 shutdown_connections 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.943+0000 7f14d1400640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f14a80744b0 0x7f14a80769c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.943+0000 7f14d1400640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14cc15ac20 0x7f14cc0715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.943+0000 7f14d1400640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14cc06f700 0x7f14cc15c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.943+0000 7f14d1400640 1 --2- 172.21.15.123:0/1047858021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14cc06c570 0x7f14cc1f0f70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:13.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.943+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 >> 172.21.15.123:0/1047858021 conn(0x7f14cc06e070 msgr2=0x7f14cc075d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.944+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 shutdown_connections 2024-04-25T12:34:13.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:13.944+0000 7f14d1400640 1 -- 172.21.15.123:0/1047858021 wait complete. 2024-04-25T12:34:13.986 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T12:34:13.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T12:34:14.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.234+0000 7ffbbd200640 1 Processor -- start 2024-04-25T12:34:14.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.234+0000 7ffbbd200640 1 -- start start 2024-04-25T12:34:14.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.236+0000 7ffbbd200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffbb815e760 0x7ffbb80c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:14.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.236+0000 7ffbbd200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbb815aff0 0x7ffbb80c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:14.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.236+0000 7ffbbd200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb8072fb0 con 0x7ffbb8074620 2024-04-25T12:34:14.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.236+0000 7ffbbd200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb80c8410 con 0x7ffbb815aff0 2024-04-25T12:34:14.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.236+0000 7ffbbd200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb8073230 con 0x7ffbb815e760 2024-04-25T12:34:14.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbafe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbb815aff0 0x7ffbb80c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:14.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbb6800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ffbb8074620 0x7ffbb80cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55908/0 (socket says 172.21.15.123:55908) 2024-04-25T12:34:14.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbb6800640 1 -- 172.21.15.123:0/592638271 learned_addr learned my addr 172.21.15.123:0/592638271 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:14.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbb5e00640 1 --2- 172.21.15.123:0/592638271 >> v2:172.21.15.153:3300/0 conn(0x7ffbb815e760 0x7ffbb80c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:14.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbafe00640 1 -- 172.21.15.123:0/592638271 >> v2:172.21.15.153:3300/0 conn(0x7ffbb815e760 msgr2=0x7ffbb80c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:14.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbafe00640 1 --2- 172.21.15.123:0/592638271 >> v2:172.21.15.153:3300/0 conn(0x7ffbb815e760 0x7ffbb80c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbafe00640 1 -- 172.21.15.123:0/592638271 >> v1:172.21.15.123:6789/0 conn(0x7ffbb8074620 legacy=0x7ffbb80cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:14.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.237+0000 7ffbafe00640 1 -- 172.21.15.123:0/592638271 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb8073510 con 0x7ffbb815aff0 2024-04-25T12:34:14.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.238+0000 7ffbafe00640 1 --2- 172.21.15.123:0/592638271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815aff0 0x7ffbb80c7ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80cae70 tx=0x7ffba002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bbd786e799cdc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:14.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.238+0000 7ffbb5400640 1 -- 172.21.15.123:0/592638271 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0037070 con 0x7ffbb815aff0 2024-04-25T12:34:14.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbb5400640 1 -- 172.21.15.123:0/592638271 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba0032440 con 0x7ffbb815aff0 2024-04-25T12:34:14.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbb5400640 1 -- 172.21.15.123:0/592638271 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0032810 con 0x7ffbb815aff0 2024-04-25T12:34:14.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbbd200640 1 -- 172.21.15.123:0/592638271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815aff0 msgr2=0x7ffbb80c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:14.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbbd200640 1 --2- 172.21.15.123:0/592638271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815aff0 0x7ffbb80c7ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80cae70 tx=0x7ffba002a250 comp rx=0 tx=0).stop 2024-04-25T12:34:14.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbbd200640 1 -- 172.21.15.123:0/592638271 shutdown_connections 2024-04-25T12:34:14.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbbd200640 1 --2- 172.21.15.123:0/592638271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815aff0 0x7ffbb80c7ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbbd200640 1 --2- 172.21.15.123:0/592638271 >> v2:172.21.15.153:3300/0 conn(0x7ffbb815e760 0x7ffbb80c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.239+0000 7ffbbd200640 1 -- 172.21.15.123:0/592638271 >> 172.21.15.123:0/592638271 conn(0x7ffbb8070fc0 msgr2=0x7ffbb80713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:14.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.240+0000 7ffbbd200640 1 -- 172.21.15.123:0/592638271 shutdown_connections 2024-04-25T12:34:14.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.240+0000 7ffbbd200640 1 -- 172.21.15.123:0/592638271 wait complete. 2024-04-25T12:34:14.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.242+0000 7ffbbd200640 1 Processor -- start 2024-04-25T12:34:14.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.242+0000 7ffbbd200640 1 -- start start 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbbd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb8074620 0x7ffbb81f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbbd200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb815aff0 0x7ffbb8155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbbd200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815e760 0x7ffbb81594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbbd200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb815e240 con 0x7ffbb815e760 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbbd200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb815df60 con 0x7ffbb8074620 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbbd200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb815b400 con 0x7ffbb815aff0 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbb6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815e760 0x7ffbb81594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:14.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbb6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815e760 0x7ffbb81594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41190/0 (socket says 172.21.15.123:41190) 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbb6800640 1 -- 172.21.15.123:0/317847339 learned_addr learned my addr 172.21.15.123:0/317847339 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbb5e00640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb8074620 0x7ffbb81f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.244+0000 7ffbafe00640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb815aff0 0x7ffbb8155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbb6800640 1 -- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb815aff0 msgr2=0x7ffbb8155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbb6800640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb815aff0 0x7ffbb8155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbb6800640 1 -- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb8074620 msgr2=0x7ffbb81f14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbb6800640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb8074620 0x7ffbb81f14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbb6800640 1 -- 172.21.15.123:0/317847339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb8150870 con 0x7ffbb815e760 2024-04-25T12:34:14.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbb6800640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815e760 0x7ffbb81594f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000caa0 tx=0x7ffbb000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbae000640 1 -- 172.21.15.123:0/317847339 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0017070 con 0x7ffbb815e760 2024-04-25T12:34:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbae000640 1 -- 172.21.15.123:0/317847339 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb000aba0 con 0x7ffbb815e760 2024-04-25T12:34:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb815d840 con 0x7ffbb815e760 2024-04-25T12:34:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbae000640 1 -- 172.21.15.123:0/317847339 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0012400 con 0x7ffbb815e760 2024-04-25T12:34:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.245+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb81f44b0 con 0x7ffbb815e760 2024-04-25T12:34:14.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.246+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb8076700 con 0x7ffbb815e760 2024-04-25T12:34:14.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.247+0000 7ffbae000640 1 -- 172.21.15.123:0/317847339 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffbb001a070 con 0x7ffbb815e760 2024-04-25T12:34:14.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.248+0000 7ffbae000640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb940744e0 0x7ffb940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:14.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.248+0000 7ffbae000640 1 -- 172.21.15.123:0/317847339 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffbb0094dd0 con 0x7ffbb815e760 2024-04-25T12:34:14.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.249+0000 7ffbb5e00640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb940744e0 0x7ffb940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:14.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.249+0000 7ffbb5e00640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb940744e0 0x7ffb940769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009aa0 tx=0x7ffba8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:14.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.253+0000 7ffbae000640 1 -- 172.21.15.123:0/317847339 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbb8150870 con 0x7ffbb815e760 2024-04-25T12:34:14.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.447+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ffbb80cb330 con 0x7ffbb815e760 2024-04-25T12:34:14.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.936+0000 7ffbae000640 1 -- 172.21.15.123:0/317847339 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7ffbb8072fb0 con 0x7ffbb815e760 2024-04-25T12:34:14.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.939+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb940744e0 msgr2=0x7ffb940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:14.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.939+0000 7ffbbd200640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb940744e0 0x7ffb940769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009aa0 tx=0x7ffba8009190 comp rx=0 tx=0).stop 2024-04-25T12:34:14.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815e760 msgr2=0x7ffbb81594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:14.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815e760 0x7ffbb81594f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000caa0 tx=0x7ffbb000cf60 comp rx=0 tx=0).stop 2024-04-25T12:34:14.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 shutdown_connections 2024-04-25T12:34:14.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb940744e0 0x7ffb940769f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb815e760 0x7ffbb81594f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb815aff0 0x7ffbb8155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 --2- 172.21.15.123:0/317847339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb8074620 0x7ffbb81f14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:14.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.940+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 >> 172.21.15.123:0/317847339 conn(0x7ffbb8070fc0 msgr2=0x7ffbb80cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:14.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.941+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 shutdown_connections 2024-04-25T12:34:14.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:14.942+0000 7ffbbd200640 1 -- 172.21.15.123:0/317847339 wait complete. 2024-04-25T12:34:14.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T12:34:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.246+0000 7f7c74e00640 1 Processor -- start 2024-04-25T12:34:15.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.246+0000 7f7c74e00640 1 -- start start 2024-04-25T12:34:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.248+0000 7f7c74e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c701565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.248+0000 7f7c74e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c700c81d0 con 0x7f7c700c91c0 2024-04-25T12:34:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.248+0000 7f7c74e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c700c8450 con 0x7f7c700c2720 2024-04-25T12:34:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.248+0000 7f7c74e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7c70159310 con 0x7f7c700c7dc0 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7c700c91c0 0x7f7c700c95d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55922/0 (socket says 172.21.15.123:55922) 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6fe00640 1 -- 172.21.15.123:0/3388957639 learned_addr learned my addr 172.21.15.123:0/3388957639 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6f400640 1 --2- 172.21.15.123:0/3388957639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c701565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6ea00640 1 -- 172.21.15.123:0/3388957639 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352109009 0 0) 0x7f7c68001490 con 0x7f7c700c91c0 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6ea00640 1 -- 172.21.15.123:0/3388957639 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c54003520 con 0x7f7c700c91c0 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6ea00640 1 -- 172.21.15.123:0/3388957639 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650294904 0 0) 0x7f7c700c8450 con 0x7f7c700c2720 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6ea00640 1 -- 172.21.15.123:0/3388957639 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c68001490 con 0x7f7c700c2720 2024-04-25T12:34:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6f400640 1 -- 172.21.15.123:0/3388957639 >> v1:172.21.15.153:6789/0 conn(0x7f7c700c2720 legacy=0x7f7c7006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:15.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.249+0000 7f7c6f400640 1 -- 172.21.15.123:0/3388957639 >> v1:172.21.15.123:6789/0 conn(0x7f7c700c91c0 legacy=0x7f7c700c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:15.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.250+0000 7f7c6f400640 1 -- 172.21.15.123:0/3388957639 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c701595f0 con 0x7f7c700c7dc0 2024-04-25T12:34:15.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.251+0000 7f7c6f400640 1 --2- 172.21.15.123:0/3388957639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c701565f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c600097d0 tx=0x7f7c6002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57fd0077e663ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:15.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.251+0000 7f7c6ea00640 1 -- 172.21.15.123:0/3388957639 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60037070 con 0x7f7c700c7dc0 2024-04-25T12:34:15.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.251+0000 7f7c6ea00640 1 -- 172.21.15.123:0/3388957639 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c60032440 con 0x7f7c700c7dc0 2024-04-25T12:34:15.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.251+0000 7f7c6ea00640 1 -- 172.21.15.123:0/3388957639 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c600327f0 con 0x7f7c700c7dc0 2024-04-25T12:34:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.252+0000 7f7c74e00640 1 -- 172.21.15.123:0/3388957639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 msgr2=0x7f7c701565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.252+0000 7f7c74e00640 1 --2- 172.21.15.123:0/3388957639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c701565f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c600097d0 tx=0x7f7c6002a290 comp rx=0 tx=0).stop 2024-04-25T12:34:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.252+0000 7f7c74e00640 1 -- 172.21.15.123:0/3388957639 shutdown_connections 2024-04-25T12:34:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.252+0000 7f7c74e00640 1 --2- 172.21.15.123:0/3388957639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c701565f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.252+0000 7f7c74e00640 1 -- 172.21.15.123:0/3388957639 >> 172.21.15.123:0/3388957639 conn(0x7f7c70070b70 msgr2=0x7f7c70070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.252+0000 7f7c74e00640 1 -- 172.21.15.123:0/3388957639 shutdown_connections 2024-04-25T12:34:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.252+0000 7f7c74e00640 1 -- 172.21.15.123:0/3388957639 wait complete. 2024-04-25T12:34:15.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.254+0000 7f7c74e00640 1 Processor -- start 2024-04-25T12:34:15.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.255+0000 7f7c74e00640 1 -- start start 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c74e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c700c2720 0x7f7c701f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c74e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c700c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c74e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c700c91c0 0x7f7c7014ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c74e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c700c4cb0 con 0x7f7c700c91c0 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c74e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70159bb0 con 0x7f7c700c2720 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c74e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7c70158fc0 con 0x7f7c700c7dc0 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c6fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c700c91c0 0x7f7c7014ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c6fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c700c91c0 0x7f7c7014ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41192/0 (socket says 172.21.15.123:41192) 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c6fe00640 1 -- 172.21.15.123:0/977315816 learned_addr learned my addr 172.21.15.123:0/977315816 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c6f400640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c700c2720 0x7f7c701f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.256+0000 7f7c67e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c700c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c6fe00640 1 -- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 msgr2=0x7f7c700c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c6fe00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c700c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c6fe00640 1 -- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c700c2720 msgr2=0x7f7c701f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c6fe00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c700c2720 0x7f7c701f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c6fe00640 1 -- 172.21.15.123:0/977315816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c701f4880 con 0x7f7c700c91c0 2024-04-25T12:34:15.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c67e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c700c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:15.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c6fe00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c700c91c0 0x7f7c7014ea70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c68001f20 tx=0x7f7c6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:15.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c67400640 1 -- 172.21.15.123:0/977315816 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68017070 con 0x7f7c700c91c0 2024-04-25T12:34:15.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c67400640 1 -- 172.21.15.123:0/977315816 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c68012400 con 0x7f7c700c91c0 2024-04-25T12:34:15.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c67400640 1 -- 172.21.15.123:0/977315816 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c680127b0 con 0x7f7c700c91c0 2024-04-25T12:34:15.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.257+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c701f37b0 con 0x7f7c700c91c0 2024-04-25T12:34:15.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.258+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c700abc50 con 0x7f7c700c91c0 2024-04-25T12:34:15.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.259+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c700b3110 con 0x7f7c700c91c0 2024-04-25T12:34:15.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.260+0000 7f7c67400640 1 -- 172.21.15.123:0/977315816 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c6801a070 con 0x7f7c700c91c0 2024-04-25T12:34:15.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.261+0000 7f7c67400640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7c4c0744e0 0x7f7c4c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:15.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.261+0000 7f7c67400640 1 -- 172.21.15.123:0/977315816 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f7c680951a0 con 0x7f7c700c91c0 2024-04-25T12:34:15.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.261+0000 7f7c6f400640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7c4c0744e0 0x7f7c4c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:15.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.262+0000 7f7c6f400640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7c4c0744e0 0x7f7c4c0769f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c60034000 tx=0x7f7c6002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:15.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.267+0000 7f7c67400640 1 -- 172.21.15.123:0/977315816 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c700b3110 con 0x7f7c700c91c0 2024-04-25T12:34:15.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:15.455+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7c70076da0 con 0x7f7c700c91c0 2024-04-25T12:34:16.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.962+0000 7f7c67400640 1 -- 172.21.15.123:0/977315816 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f7c700c81d0 con 0x7f7c700c91c0 2024-04-25T12:34:16.963 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data_ec' created 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.966+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7c4c0744e0 msgr2=0x7f7c4c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.966+0000 7f7c74e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7c4c0744e0 0x7f7c4c0769f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c60034000 tx=0x7f7c6002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.966+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c700c91c0 msgr2=0x7f7c7014ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.966+0000 7f7c74e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c700c91c0 0x7f7c7014ea70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c68001f20 tx=0x7f7c6800a430 comp rx=0 tx=0).stop 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 shutdown_connections 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7c4c0744e0 0x7f7c4c0769f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c700c91c0 0x7f7c7014ea70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c700c7dc0 0x7f7c700c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:16.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 --2- 172.21.15.123:0/977315816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c700c2720 0x7f7c701f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:16.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 >> 172.21.15.123:0/977315816 conn(0x7f7c70070b70 msgr2=0x7f7c7006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:16.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 shutdown_connections 2024-04-25T12:34:16.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:16.967+0000 7f7c74e00640 1 -- 172.21.15.123:0/977315816 wait complete. 2024-04-25T12:34:17.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T12:34:17.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.265+0000 7fd37fe00640 1 Processor -- start 2024-04-25T12:34:17.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.265+0000 7fd37fe00640 1 -- start start 2024-04-25T12:34:17.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3800c64f0 0x7fd38015c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:17.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd38006dfe0 0x7fd3801604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:17.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37fe00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3800cb510 con 0x7fd3800cae30 2024-04-25T12:34:17.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd38006d820 con 0x7fd38006dfe0 2024-04-25T12:34:17.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37fe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd38015f9d0 con 0x7fd3800c64f0 2024-04-25T12:34:17.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37f400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd38006dfe0 0x7fd3801604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37e000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd3800cae30 0x7fd3800cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60216/0 (socket says 172.21.15.123:60216) 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37e000640 1 -- 172.21.15.123:0/4243824206 learned_addr learned my addr 172.21.15.123:0/4243824206 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.267+0000 7fd37ea00640 1 --2- 172.21.15.123:0/4243824206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3800c64f0 0x7fd38015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.268+0000 7fd37d600640 1 -- 172.21.15.123:0/4243824206 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 926499401 0 0) 0x7fd3800cb510 con 0x7fd3800cae30 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.268+0000 7fd37d600640 1 -- 172.21.15.123:0/4243824206 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd360003520 con 0x7fd3800cae30 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.268+0000 7fd37ea00640 1 -- 172.21.15.123:0/4243824206 >> v2:172.21.15.123:3300/0 conn(0x7fd38006dfe0 msgr2=0x7fd3801604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.268+0000 7fd37ea00640 1 --2- 172.21.15.123:0/4243824206 >> v2:172.21.15.123:3300/0 conn(0x7fd38006dfe0 0x7fd3801604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.268+0000 7fd37ea00640 1 -- 172.21.15.123:0/4243824206 >> v1:172.21.15.153:6789/0 conn(0x7fd3800cae30 legacy=0x7fd3800cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.268+0000 7fd37ea00640 1 -- 172.21.15.123:0/4243824206 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38015fcb0 con 0x7fd3800c64f0 2024-04-25T12:34:17.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.268+0000 7fd37f400640 1 --2- 172.21.15.123:0/4243824206 >> v2:172.21.15.123:3300/0 conn(0x7fd38006dfe0 0x7fd3801604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:17.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.269+0000 7fd37ea00640 1 --2- 172.21.15.123:0/4243824206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3800c64f0 0x7fd38015c340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097c0 tx=0x7fd37002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf507d7e1da8b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:17.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.269+0000 7fd37d600640 1 -- 172.21.15.123:0/4243824206 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370037070 con 0x7fd3800c64f0 2024-04-25T12:34:17.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.269+0000 7fd37d600640 1 -- 172.21.15.123:0/4243824206 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37002ae20 con 0x7fd3800c64f0 2024-04-25T12:34:17.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.269+0000 7fd37d600640 1 -- 172.21.15.123:0/4243824206 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370032660 con 0x7fd3800c64f0 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 -- 172.21.15.123:0/4243824206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3800c64f0 msgr2=0x7fd38015c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 --2- 172.21.15.123:0/4243824206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3800c64f0 0x7fd38015c340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097c0 tx=0x7fd37002a110 comp rx=0 tx=0).stop 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 -- 172.21.15.123:0/4243824206 shutdown_connections 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 --2- 172.21.15.123:0/4243824206 >> v2:172.21.15.123:3300/0 conn(0x7fd38006dfe0 0x7fd3801604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 --2- 172.21.15.123:0/4243824206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3800c64f0 0x7fd38015c340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 -- 172.21.15.123:0/4243824206 >> 172.21.15.123:0/4243824206 conn(0x7fd380068630 msgr2=0x7fd380068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 -- 172.21.15.123:0/4243824206 shutdown_connections 2024-04-25T12:34:17.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.270+0000 7fd37fe00640 1 -- 172.21.15.123:0/4243824206 wait complete. 2024-04-25T12:34:17.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.272+0000 7fd37fe00640 1 Processor -- start 2024-04-25T12:34:17.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.273+0000 7fd37fe00640 1 -- start start 2024-04-25T12:34:17.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38006dfe0 0x7fd3801f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:17.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3800cae30 0x7fd38015ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:17.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3801555b0 0x7fd38014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:17.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3800cff00 con 0x7fd3800cae30 2024-04-25T12:34:17.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd38006db00 con 0x7fd38006dfe0 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37fe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd3800d0080 con 0x7fd3801555b0 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3800cae30 0x7fd38015ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3800cae30 0x7fd38015ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41210/0 (socket says 172.21.15.123:41210) 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37e000640 1 -- 172.21.15.123:0/530203187 learned_addr learned my addr 172.21.15.123:0/530203187 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37f400640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3801555b0 0x7fd38014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.274+0000 7fd37ea00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38006dfe0 0x7fd3801f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37e000640 1 -- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3801555b0 msgr2=0x7fd38014ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37e000640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3801555b0 0x7fd38014ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37e000640 1 -- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38006dfe0 msgr2=0x7fd3801f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37e000640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38006dfe0 0x7fd3801f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37e000640 1 -- 172.21.15.123:0/530203187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3800c64f0 con 0x7fd3800cae30 2024-04-25T12:34:17.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37e000640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3800cae30 0x7fd38015ea70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd368002230 tx=0x7fd36800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:17.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd376a00640 1 -- 172.21.15.123:0/530203187 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368017070 con 0x7fd3800cae30 2024-04-25T12:34:17.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd376a00640 1 -- 172.21.15.123:0/530203187 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd36800abd0 con 0x7fd3800cae30 2024-04-25T12:34:17.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3800c6ac0 con 0x7fd3800cae30 2024-04-25T12:34:17.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd376a00640 1 -- 172.21.15.123:0/530203187 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368012400 con 0x7fd3800cae30 2024-04-25T12:34:17.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.275+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38014d980 con 0x7fd3800cae30 2024-04-25T12:34:17.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.276+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38015ce40 con 0x7fd3800cae30 2024-04-25T12:34:17.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.278+0000 7fd376a00640 1 -- 172.21.15.123:0/530203187 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd36801a070 con 0x7fd3800cae30 2024-04-25T12:34:17.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.279+0000 7fd376a00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd35c0744b0 0x7fd35c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:17.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.279+0000 7fd376a00640 1 -- 172.21.15.123:0/530203187 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fd368095280 con 0x7fd3800cae30 2024-04-25T12:34:17.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.279+0000 7fd37ea00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd35c0744b0 0x7fd35c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:17.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.280+0000 7fd37ea00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd35c0744b0 0x7fd35c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd370034000 tx=0x7fd37002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:17.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.284+0000 7fd376a00640 1 -- 172.21.15.123:0/530203187 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd38015ce40 con 0x7fd3800cae30 2024-04-25T12:34:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.483+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd380076da0 con 0x7fd3800cae30 2024-04-25T12:34:17.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.973+0000 7fd376a00640 1 -- 172.21.15.123:0/530203187 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7fd36805f000 con 0x7fd3800cae30 2024-04-25T12:34:17.975 INFO:teuthology.orchestra.run.smithi123.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.979+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd35c0744b0 msgr2=0x7fd35c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.979+0000 7fd37fe00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd35c0744b0 0x7fd35c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd370034000 tx=0x7fd37002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.979+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3800cae30 msgr2=0x7fd38015ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.979+0000 7fd37fe00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3800cae30 0x7fd38015ea70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd368002230 tx=0x7fd36800cf70 comp rx=0 tx=0).stop 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.980+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 shutdown_connections 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.980+0000 7fd37fe00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd35c0744b0 0x7fd35c0769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.980+0000 7fd37fe00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3801555b0 0x7fd38014ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.980+0000 7fd37fe00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3800cae30 0x7fd38015ea70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.980+0000 7fd37fe00640 1 --2- 172.21.15.123:0/530203187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd38006dfe0 0x7fd3801f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:17.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.981+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 >> 172.21.15.123:0/530203187 conn(0x7fd380068630 msgr2=0x7fd380154a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:17.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.981+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 shutdown_connections 2024-04-25T12:34:17.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:17.982+0000 7fd37fe00640 1 -- 172.21.15.123:0/530203187 wait complete. 2024-04-25T12:34:18.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T12:34:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.280+0000 7f8563e00640 1 Processor -- start 2024-04-25T12:34:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.281+0000 7f8563e00640 1 -- start start 2024-04-25T12:34:18.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8563e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c154650 0x7f855c15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:18.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8563e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f855c0730e0 0x7f855c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:18.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8563e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f855c15b380 con 0x7f855c0c9f00 2024-04-25T12:34:18.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8563e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f855c074b20 con 0x7f855c0730e0 2024-04-25T12:34:18.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8563e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f855c0c8880 con 0x7f855c154650 2024-04-25T12:34:18.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8561e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f855c0730e0 0x7f855c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:18.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8561400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c154650 0x7f855c15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:18.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8560a00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f855c0c9f00 0x7f855c0c8170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60220/0 (socket says 172.21.15.123:60220) 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.283+0000 7f8560a00640 1 -- 172.21.15.123:0/2081286679 learned_addr learned my addr 172.21.15.123:0/2081286679 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.284+0000 7f855be00640 1 -- 172.21.15.123:0/2081286679 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890993624 0 0) 0x7f854c001110 con 0x7f855c0c9f00 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.284+0000 7f855be00640 1 -- 172.21.15.123:0/2081286679 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8544003520 con 0x7f855c0c9f00 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.284+0000 7f8561400640 1 -- 172.21.15.123:0/2081286679 >> v2:172.21.15.153:3300/0 conn(0x7f855c0730e0 msgr2=0x7f855c0734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.284+0000 7f8561400640 1 --2- 172.21.15.123:0/2081286679 >> v2:172.21.15.153:3300/0 conn(0x7f855c0730e0 0x7f855c0734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.284+0000 7f8561400640 1 -- 172.21.15.123:0/2081286679 >> v1:172.21.15.153:6789/0 conn(0x7f855c0c9f00 legacy=0x7f855c0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.284+0000 7f8561400640 1 -- 172.21.15.123:0/2081286679 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c0ca780 con 0x7f855c154650 2024-04-25T12:34:18.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.284+0000 7f8561e00640 1 --2- 172.21.15.123:0/2081286679 >> v2:172.21.15.153:3300/0 conn(0x7f855c0730e0 0x7f855c0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:18.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.285+0000 7f8561400640 1 --2- 172.21.15.123:0/2081286679 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c154650 0x7f855c15ae40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85500097a0 tx=0x7f855002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7523a8341b62fb6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:18.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.285+0000 7f855be00640 1 -- 172.21.15.123:0/2081286679 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550037070 con 0x7f855c154650 2024-04-25T12:34:18.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.285+0000 7f855be00640 1 -- 172.21.15.123:0/2081286679 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8550032440 con 0x7f855c154650 2024-04-25T12:34:18.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.285+0000 7f855be00640 1 -- 172.21.15.123:0/2081286679 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550032810 con 0x7f855c154650 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 -- 172.21.15.123:0/2081286679 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c154650 msgr2=0x7f855c15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 --2- 172.21.15.123:0/2081286679 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c154650 0x7f855c15ae40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85500097a0 tx=0x7f855002a210 comp rx=0 tx=0).stop 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 -- 172.21.15.123:0/2081286679 shutdown_connections 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 --2- 172.21.15.123:0/2081286679 >> v2:172.21.15.153:3300/0 conn(0x7f855c0730e0 0x7f855c0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 --2- 172.21.15.123:0/2081286679 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c154650 0x7f855c15ae40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 -- 172.21.15.123:0/2081286679 >> 172.21.15.123:0/2081286679 conn(0x7f855c06f440 msgr2=0x7f855c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 -- 172.21.15.123:0/2081286679 shutdown_connections 2024-04-25T12:34:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.286+0000 7f8563e00640 1 -- 172.21.15.123:0/2081286679 wait complete. 2024-04-25T12:34:18.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.289+0000 7f8563e00640 1 Processor -- start 2024-04-25T12:34:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.289+0000 7f8563e00640 1 -- start start 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8563e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c0730e0 0x7f855c148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8563e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855c0c9f00 0x7f855c151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8563e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f855c154650 0x7f855c146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8563e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f855c15dd30 con 0x7f855c154650 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8563e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f855c15da50 con 0x7f855c0c9f00 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8563e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f855c15e010 con 0x7f855c0730e0 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8561400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c0730e0 0x7f855c148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8560a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855c0c9f00 0x7f855c151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8561e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f855c154650 0x7f855c146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8561400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c0730e0 0x7f855c148e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37488/0 (socket says 172.21.15.123:37488) 2024-04-25T12:34:18.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.291+0000 7f8561400640 1 -- 172.21.15.123:0/1945418024 learned_addr learned my addr 172.21.15.123:0/1945418024 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.292+0000 7f8561400640 1 -- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855c0c9f00 msgr2=0x7f855c151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.292+0000 7f8561400640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855c0c9f00 0x7f855c151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.292+0000 7f8561400640 1 -- 172.21.15.123:0/1945418024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f855c154650 msgr2=0x7f855c146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.292+0000 7f8561400640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f855c154650 0x7f855c146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.292+0000 7f8561400640 1 -- 172.21.15.123:0/1945418024 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c0b0e20 con 0x7f855c0730e0 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.292+0000 7f8561400640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c0730e0 0x7f855c148e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85500098c0 tx=0x7f85500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.293+0000 7f8559600640 1 -- 172.21.15.123:0/1945418024 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550041030 con 0x7f855c0730e0 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.293+0000 7f8559600640 1 -- 172.21.15.123:0/1945418024 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855003d420 con 0x7f855c0730e0 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.293+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c0b10e0 con 0x7f855c0730e0 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.293+0000 7f8559600640 1 -- 172.21.15.123:0/1945418024 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550037040 con 0x7f855c0730e0 2024-04-25T12:34:18.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.293+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c15cfa0 con 0x7f855c0730e0 2024-04-25T12:34:18.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.294+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f855c076700 con 0x7f855c0730e0 2024-04-25T12:34:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.296+0000 7f8559600640 1 -- 172.21.15.123:0/1945418024 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8550044070 con 0x7f855c0730e0 2024-04-25T12:34:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.297+0000 7f8559600640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85400744b0 0x7f85400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.297+0000 7f8559600640 1 -- 172.21.15.123:0/1945418024 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f85500b5a40 con 0x7f855c0730e0 2024-04-25T12:34:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.297+0000 7f8560a00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85400744b0 0x7f85400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.298+0000 7f8560a00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85400744b0 0x7f85400769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f855c1f3c80 tx=0x7f854c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:18.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.302+0000 7f8559600640 1 -- 172.21.15.123:0/1945418024 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f855007feb0 con 0x7f855c0730e0 2024-04-25T12:34:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.532+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f855c0ca4c0 con 0x7f855c0730e0 2024-04-25T12:34:18.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.979+0000 7f8559600640 1 -- 172.21.15.123:0/1945418024 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) ==== 137+0+0 (secure 0 0 0) 0x7f855007f7b0 con 0x7f855c0730e0 2024-04-25T12:34:18.981 INFO:teuthology.orchestra.run.smithi123.stderr:added data pool 4 to fsmap 2024-04-25T12:34:18.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85400744b0 msgr2=0x7f85400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:18.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85400744b0 0x7f85400769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f855c1f3c80 tx=0x7f854c009040 comp rx=0 tx=0).stop 2024-04-25T12:34:18.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c0730e0 msgr2=0x7f855c148e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:18.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c0730e0 0x7f855c148e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85500098c0 tx=0x7f85500046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 shutdown_connections 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85400744b0 0x7f85400769c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f855c154650 0x7f855c146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855c0c9f00 0x7f855c151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 --2- 172.21.15.123:0/1945418024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f855c0730e0 0x7f855c148e30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.984+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 >> 172.21.15.123:0/1945418024 conn(0x7f855c06f440 msgr2=0x7f855c0cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.985+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 shutdown_connections 2024-04-25T12:34:18.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:18.985+0000 7f8563e00640 1 -- 172.21.15.123:0/1945418024 wait complete. 2024-04-25T12:34:19.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:34:19.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.274+0000 7f4a9b200640 1 Processor -- start 2024-04-25T12:34:19.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.274+0000 7f4a9b200640 1 -- start start 2024-04-25T12:34:19.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.276+0000 7f4a9b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a9c1567b0 0x7f4a9c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.276+0000 7f4a9b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a9c06ecd0 con 0x7f4a9c15d180 2024-04-25T12:34:19.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.276+0000 7f4a9b200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a9c06efb0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.276+0000 7f4a9b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9c06f290 con 0x7f4a9c1567b0 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a99e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a9c1567b0 0x7f4a9c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a9a800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4a9c15d180 0x7f4a9c0c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55926/0 (socket says 172.21.15.123:55926) 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a9a800640 1 -- 172.21.15.123:0/888684509 learned_addr learned my addr 172.21.15.123:0/888684509 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a98a00640 1 -- 172.21.15.123:0/888684509 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028049765 0 0) 0x7f4a9c06efb0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a98a00640 1 -- 172.21.15.123:0/888684509 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a80003520 con 0x7f4a9c06cab0 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a98a00640 1 -- 172.21.15.123:0/888684509 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827137953 0 0) 0x7f4a9c06ecd0 con 0x7f4a9c15d180 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a99e00640 1 -- 172.21.15.123:0/888684509 >> v1:172.21.15.153:6789/0 conn(0x7f4a9c06cab0 legacy=0x7f4a9c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a99e00640 1 -- 172.21.15.123:0/888684509 >> v1:172.21.15.123:6789/0 conn(0x7f4a9c15d180 legacy=0x7f4a9c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.277+0000 7f4a99e00640 1 -- 172.21.15.123:0/888684509 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c06f570 con 0x7f4a9c1567b0 2024-04-25T12:34:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.279+0000 7f4a99e00640 1 --2- 172.21.15.123:0/888684509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1567b0 0x7f4a9c06c570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0097a0 tx=0x7f4a8c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598cadd25f9e58cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.279+0000 7f4a98a00640 1 -- 172.21.15.123:0/888684509 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c037070 con 0x7f4a9c1567b0 2024-04-25T12:34:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.279+0000 7f4a98a00640 1 -- 172.21.15.123:0/888684509 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c032640 con 0x7f4a9c1567b0 2024-04-25T12:34:19.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.279+0000 7f4a98a00640 1 -- 172.21.15.123:0/888684509 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0329f0 con 0x7f4a9c1567b0 2024-04-25T12:34:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.280+0000 7f4a9b200640 1 -- 172.21.15.123:0/888684509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1567b0 msgr2=0x7f4a9c06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.280+0000 7f4a9b200640 1 --2- 172.21.15.123:0/888684509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1567b0 0x7f4a9c06c570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0097a0 tx=0x7f4a8c02a2c0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.280+0000 7f4a9b200640 1 -- 172.21.15.123:0/888684509 shutdown_connections 2024-04-25T12:34:19.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.280+0000 7f4a9b200640 1 --2- 172.21.15.123:0/888684509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1567b0 0x7f4a9c06c570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.280+0000 7f4a9b200640 1 -- 172.21.15.123:0/888684509 >> 172.21.15.123:0/888684509 conn(0x7f4a9c0cd600 msgr2=0x7f4a9c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.280+0000 7f4a9b200640 1 -- 172.21.15.123:0/888684509 shutdown_connections 2024-04-25T12:34:19.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.280+0000 7f4a9b200640 1 -- 172.21.15.123:0/888684509 wait complete. 2024-04-25T12:34:19.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.282+0000 7f4a9b200640 1 Processor -- start 2024-04-25T12:34:19.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.282+0000 7f4a9b200640 1 -- start start 2024-04-25T12:34:19.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a9b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c06cab0 0x7f4a9c1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a9b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c1567b0 0x7f4a9c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a9b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c15d180 0x7f4a9c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a9b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c156bc0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a9b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c0c9d20 con 0x7f4a9c15d180 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a9b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c156d40 con 0x7f4a9c1567b0 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a99e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c06cab0 0x7f4a9c1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a99e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c06cab0 0x7f4a9c1f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41234/0 (socket says 172.21.15.123:41234) 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a99e00640 1 -- 172.21.15.123:0/995235526 learned_addr learned my addr 172.21.15.123:0/995235526 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a9a800640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c15d180 0x7f4a9c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.284+0000 7f4a99400640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c1567b0 0x7f4a9c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a99e00640 1 -- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c1567b0 msgr2=0x7f4a9c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a99e00640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c1567b0 0x7f4a9c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a99e00640 1 -- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c15d180 msgr2=0x7f4a9c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a99e00640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c15d180 0x7f4a9c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a99e00640 1 -- 172.21.15.123:0/995235526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c15a3e0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a9a800640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c15d180 0x7f4a9c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:19.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a99e00640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c06cab0 0x7f4a9c1f6070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:19.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a92000640 1 -- 172.21.15.123:0/995235526 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c037040 con 0x7f4a9c06cab0 2024-04-25T12:34:19.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a92000640 1 -- 172.21.15.123:0/995235526 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c042070 con 0x7f4a9c06cab0 2024-04-25T12:34:19.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a92000640 1 -- 172.21.15.123:0/995235526 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0309b0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.285+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c1f65b0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.286+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c157ef0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.287+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c15a5b0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.288+0000 7f4a92000640 1 -- 172.21.15.123:0/995235526 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a8c030b50 con 0x7f4a9c06cab0 2024-04-25T12:34:19.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.290+0000 7f4a92000640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a780744e0 0x7f4a780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.290+0000 7f4a92000640 1 -- 172.21.15.123:0/995235526 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4a8c0b5e00 con 0x7f4a9c06cab0 2024-04-25T12:34:19.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.290+0000 7f4a99400640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a780744e0 0x7f4a780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.291+0000 7f4a99400640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a780744e0 0x7f4a780769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c154860 tx=0x7f4a84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:19.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.295+0000 7f4a92000640 1 -- 172.21.15.123:0/995235526 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a8c07fc80 con 0x7f4a9c06cab0 2024-04-25T12:34:19.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.538+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a9c076da0 con 0x7f4a9c06cab0 2024-04-25T12:34:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.539+0000 7f4a92000640 1 -- 172.21.15.123:0/995235526 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+6837 (secure 0 0 0) 0x7f4a8c083f80 con 0x7f4a9c06cab0 2024-04-25T12:34:19.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:19.540 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:18.974727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T12:34:19.540 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 6 2024-04-25T12:34:19.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a780744e0 msgr2=0x7f4a780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a780744e0 0x7f4a780769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c154860 tx=0x7f4a84009040 comp rx=0 tx=0).stop 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c06cab0 msgr2=0x7f4a9c1f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c06cab0 0x7f4a9c1f6070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 shutdown_connections 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a780744e0 0x7f4a780769f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c15d180 0x7f4a9c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c1567b0 0x7f4a9c0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 --2- 172.21.15.123:0/995235526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c06cab0 0x7f4a9c1f6070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 >> 172.21.15.123:0/995235526 conn(0x7f4a9c0cd600 msgr2=0x7f4a9c153bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 shutdown_connections 2024-04-25T12:34:19.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.542+0000 7f4a9b200640 1 -- 172.21.15.123:0/995235526 wait complete. 2024-04-25T12:34:19.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:19.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.832+0000 7f264f400640 1 Processor -- start 2024-04-25T12:34:19.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.832+0000 7f264f400640 1 -- start start 2024-04-25T12:34:19.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264f400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26480c4d50 0x7f2648154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264f400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2648074870 0x7f264815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264f400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26480c8c60 0x7f26480c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264f400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f26480c61f0 con 0x7f26480c8c60 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264f400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f26480c87d0 con 0x7f26480c4d50 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264f400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2648073ce0 con 0x7f2648074870 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264ca00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26480c4d50 0x7f2648154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f263fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2648074870 0x7f264815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f264d400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26480c8c60 0x7f26480c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f263fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2648074870 0x7f264815afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37502/0 (socket says 172.21.15.123:37502) 2024-04-25T12:34:19.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.834+0000 7f263fe00640 1 -- 172.21.15.123:0/823346106 learned_addr learned my addr 172.21.15.123:0/823346106 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:19.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.835+0000 7f264ca00640 1 -- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2648074870 msgr2=0x7f264815afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:19.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.835+0000 7f264ca00640 1 --2- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2648074870 0x7f264815afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.835+0000 7f264ca00640 1 -- 172.21.15.123:0/823346106 >> v2:172.21.15.123:3300/0 conn(0x7f26480c8c60 msgr2=0x7f26480c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:19.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.835+0000 7f264ca00640 1 --2- 172.21.15.123:0/823346106 >> v2:172.21.15.123:3300/0 conn(0x7f26480c8c60 0x7f26480c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.835+0000 7f264ca00640 1 -- 172.21.15.123:0/823346106 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648073fc0 con 0x7f26480c4d50 2024-04-25T12:34:19.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.835+0000 7f263fe00640 1 --2- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2648074870 0x7f264815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:19.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.836+0000 7f264ca00640 1 --2- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c4d50 0x7f2648154890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f26480ce6d0 tx=0x7f263802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149dcc82b5447dab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:19.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.836+0000 7f2647e00640 1 -- 172.21.15.123:0/823346106 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638037070 con 0x7f26480c4d50 2024-04-25T12:34:19.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.836+0000 7f2647e00640 1 -- 172.21.15.123:0/823346106 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638032440 con 0x7f26480c4d50 2024-04-25T12:34:19.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.836+0000 7f2647e00640 1 -- 172.21.15.123:0/823346106 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638032810 con 0x7f26480c4d50 2024-04-25T12:34:19.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 -- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c4d50 msgr2=0x7f2648154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 --2- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c4d50 0x7f2648154890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f26480ce6d0 tx=0x7f263802a1a0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 -- 172.21.15.123:0/823346106 shutdown_connections 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 --2- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2648074870 0x7f264815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 --2- 172.21.15.123:0/823346106 >> v2:172.21.15.123:3300/0 conn(0x7f26480c8c60 0x7f26480c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 --2- 172.21.15.123:0/823346106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c4d50 0x7f2648154890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 -- 172.21.15.123:0/823346106 >> 172.21.15.123:0/823346106 conn(0x7f264806c750 msgr2=0x7f264806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 -- 172.21.15.123:0/823346106 shutdown_connections 2024-04-25T12:34:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.837+0000 7f264f400640 1 -- 172.21.15.123:0/823346106 wait complete. 2024-04-25T12:34:19.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.840+0000 7f264f400640 1 Processor -- start 2024-04-25T12:34:19.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.840+0000 7f264f400640 1 -- start start 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.841+0000 7f264f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648074870 0x7f26481f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.841+0000 7f264f400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26480c4d50 0x7f264814ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.841+0000 7f264f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c8c60 0x7f264814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.841+0000 7f264f400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26480cda90 con 0x7f2648074870 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.841+0000 7f264f400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26480c8460 con 0x7f26480c8c60 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.841+0000 7f264f400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f26480cdc10 con 0x7f26480c4d50 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648074870 0x7f26481f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648074870 0x7f26481f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41252/0 (socket says 172.21.15.123:41252) 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 -- 172.21.15.123:0/313753290 learned_addr learned my addr 172.21.15.123:0/313753290 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264d400640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c8c60 0x7f264814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f263fe00640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26480c4d50 0x7f264814ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 -- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26480c4d50 msgr2=0x7f264814ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26480c4d50 0x7f264814ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 -- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c8c60 msgr2=0x7f264814c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:19.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c8c60 0x7f264814c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:19.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 -- 172.21.15.123:0/313753290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26481533f0 con 0x7f2648074870 2024-04-25T12:34:19.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.842+0000 7f264ca00640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648074870 0x7f26481f37c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2638000c00 tx=0x7f26380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:19.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.843+0000 7f2645600640 1 -- 172.21.15.123:0/313753290 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638041020 con 0x7f2648074870 2024-04-25T12:34:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.843+0000 7f2645600640 1 -- 172.21.15.123:0/313753290 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263803d700 con 0x7f2648074870 2024-04-25T12:34:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.843+0000 7f2645600640 1 -- 172.21.15.123:0/313753290 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638037040 con 0x7f2648074870 2024-04-25T12:34:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.843+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26481f3d00 con 0x7f2648074870 2024-04-25T12:34:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.843+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2648152600 con 0x7f2648074870 2024-04-25T12:34:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.844+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26481535c0 con 0x7f2648074870 2024-04-25T12:34:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.845+0000 7f2645600640 1 -- 172.21.15.123:0/313753290 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2638044020 con 0x7f2648074870 2024-04-25T12:34:19.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.846+0000 7f2645600640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f26240744b0 0x7f26240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:19.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.846+0000 7f2645600640 1 -- 172.21.15.123:0/313753290 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f26380b5c30 con 0x7f2648074870 2024-04-25T12:34:19.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.846+0000 7f263fe00640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f26240744b0 0x7f26240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:19.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.847+0000 7f263fe00640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f26240744b0 0x7f26240769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f264815b5b0 tx=0x7f2630009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:19.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:19.851+0000 7f2645600640 1 -- 172.21.15.123:0/313753290 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f263804b030 con 0x7f2648074870 2024-04-25T12:34:20.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.055+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2648076700 con 0x7f2648074870 2024-04-25T12:34:20.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.057+0000 7f2645600640 1 -- 172.21.15.123:0/313753290 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7f263807f9a0 con 0x7f2648074870 2024-04-25T12:34:20.058 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:20.059 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:20.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.060+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f26240744b0 msgr2=0x7f26240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.060+0000 7f264f400640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f26240744b0 0x7f26240769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f264815b5b0 tx=0x7f2630009240 comp rx=0 tx=0).stop 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.060+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648074870 msgr2=0x7f26481f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.060+0000 7f264f400640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648074870 0x7f26481f37c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2638000c00 tx=0x7f26380046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.060+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 shutdown_connections 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.060+0000 7f264f400640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f26240744b0 0x7f26240769c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.060+0000 7f264f400640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26480c8c60 0x7f264814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.061+0000 7f264f400640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26480c4d50 0x7f264814ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.061+0000 7f264f400640 1 --2- 172.21.15.123:0/313753290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648074870 0x7f26481f37c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.061+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 >> 172.21.15.123:0/313753290 conn(0x7f264806c750 msgr2=0x7f264806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.061+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 shutdown_connections 2024-04-25T12:34:20.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.061+0000 7f264f400640 1 -- 172.21.15.123:0/313753290 wait complete. 2024-04-25T12:34:20.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:20.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.344+0000 7fd2adc00640 1 Processor -- start 2024-04-25T12:34:20.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.345+0000 7fd2adc00640 1 -- start start 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2adc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2a8072d70 0x7fd2a81420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2adc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd2a80c9900 0x7fd2a80c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2adc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a80c9d10 con 0x7fd2a80c94f0 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2adc00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a8159fd0 con 0x7fd2a8072d70 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2adc00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a81593c0 con 0x7fd2a80c9900 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2a7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2a8072d70 0x7fd2a81420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2a7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2a8072d70 0x7fd2a81420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41268/0 (socket says 172.21.15.123:41268) 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2a7e00640 1 -- 172.21.15.123:0/1858714755 learned_addr learned my addr 172.21.15.123:0/1858714755 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.347+0000 7fd2ad200640 1 --2- 172.21.15.123:0/1858714755 >> v2:172.21.15.153:3300/0 conn(0x7fd2a80c9900 0x7fd2a80c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.348+0000 7fd2a7e00640 1 -- 172.21.15.123:0/1858714755 >> v2:172.21.15.153:3300/0 conn(0x7fd2a80c9900 msgr2=0x7fd2a80c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.348+0000 7fd2a7e00640 1 --2- 172.21.15.123:0/1858714755 >> v2:172.21.15.153:3300/0 conn(0x7fd2a80c9900 0x7fd2a80c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.348+0000 7fd2a7e00640 1 -- 172.21.15.123:0/1858714755 >> v1:172.21.15.123:6789/0 conn(0x7fd2a80c94f0 legacy=0x7fd2a80c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.348+0000 7fd2a7e00640 1 -- 172.21.15.123:0/1858714755 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81596a0 con 0x7fd2a8072d70 2024-04-25T12:34:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.348+0000 7fd2a7400640 1 -- 172.21.15.123:0/1858714755 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082755531 0 0) 0x7fd2a80c9d10 con 0x7fd2a80c94f0 2024-04-25T12:34:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.349+0000 7fd2a7e00640 1 --2- 172.21.15.123:0/1858714755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8072d70 0x7fd2a81420e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8070600 tx=0x7fd29802c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179247c6ab03ee5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:20.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.349+0000 7fd2a7400640 1 -- 172.21.15.123:0/1858714755 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298035070 con 0x7fd2a8072d70 2024-04-25T12:34:20.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.349+0000 7fd2a7400640 1 -- 172.21.15.123:0/1858714755 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd298031830 con 0x7fd2a8072d70 2024-04-25T12:34:20.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.349+0000 7fd2a7400640 1 -- 172.21.15.123:0/1858714755 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298031c00 con 0x7fd2a8072d70 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 -- 172.21.15.123:0/1858714755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8072d70 msgr2=0x7fd2a81420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 --2- 172.21.15.123:0/1858714755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8072d70 0x7fd2a81420e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8070600 tx=0x7fd29802c9d0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 -- 172.21.15.123:0/1858714755 shutdown_connections 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 --2- 172.21.15.123:0/1858714755 >> v2:172.21.15.153:3300/0 conn(0x7fd2a80c9900 0x7fd2a80c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 --2- 172.21.15.123:0/1858714755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8072d70 0x7fd2a81420e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 -- 172.21.15.123:0/1858714755 >> 172.21.15.123:0/1858714755 conn(0x7fd2a80cd600 msgr2=0x7fd2a80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 -- 172.21.15.123:0/1858714755 shutdown_connections 2024-04-25T12:34:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.350+0000 7fd2adc00640 1 -- 172.21.15.123:0/1858714755 wait complete. 2024-04-25T12:34:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.352+0000 7fd2adc00640 1 Processor -- start 2024-04-25T12:34:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.352+0000 7fd2adc00640 1 -- start start 2024-04-25T12:34:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2adc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2a8072d70 0x7fd2a80b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2adc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a80c94f0 0x7fd2a815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2adc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2a80c9900 0x7fd2a815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2adc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8142620 con 0x7fd2a80c94f0 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2adc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8159c60 con 0x7fd2a8072d70 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2adc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a81427a0 con 0x7fd2a80c9900 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a80c94f0 0x7fd2a815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a80c94f0 0x7fd2a815c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41276/0 (socket says 172.21.15.123:41276) 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 -- 172.21.15.123:0/2262634480 learned_addr learned my addr 172.21.15.123:0/2262634480 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2a7e00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2a8072d70 0x7fd2a80b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd2ad200640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2a80c9900 0x7fd2a815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 -- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2a80c9900 msgr2=0x7fd2a815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2a80c9900 0x7fd2a815cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 -- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2a8072d70 msgr2=0x7fd2a80b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2a8072d70 0x7fd2a80b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.354+0000 7fd29fe00640 1 -- 172.21.15.123:0/2262634480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a815d210 con 0x7fd2a80c94f0 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.355+0000 7fd29fe00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a80c94f0 0x7fd2a815c620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c440 tx=0x7fd29000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.355+0000 7fd2a4c00640 1 -- 172.21.15.123:0/2262634480 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290017070 con 0x7fd2a80c94f0 2024-04-25T12:34:20.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.355+0000 7fd2a4c00640 1 -- 172.21.15.123:0/2262634480 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd290005480 con 0x7fd2a80c94f0 2024-04-25T12:34:20.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.355+0000 7fd2a4c00640 1 -- 172.21.15.123:0/2262634480 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290003830 con 0x7fd2a80c94f0 2024-04-25T12:34:20.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.355+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a813e6f0 con 0x7fd2a80c94f0 2024-04-25T12:34:20.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.355+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a813e450 con 0x7fd2a80c94f0 2024-04-25T12:34:20.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.356+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a8076da0 con 0x7fd2a80c94f0 2024-04-25T12:34:20.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.357+0000 7fd2a4c00640 1 -- 172.21.15.123:0/2262634480 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd29001a070 con 0x7fd2a80c94f0 2024-04-25T12:34:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.359+0000 7fd2a4c00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2840744e0 0x7fd2840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.359+0000 7fd2a4c00640 1 -- 172.21.15.123:0/2262634480 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd2900947a0 con 0x7fd2a80c94f0 2024-04-25T12:34:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.359+0000 7fd2a7e00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2840744e0 0x7fd2840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.360+0000 7fd2a7e00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2840744e0 0x7fd2840769f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd298037000 tx=0x7fd298032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.364+0000 7fd2a4c00640 1 -- 172.21.15.123:0/2262634480 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd29005e590 con 0x7fd2a80c94f0 2024-04-25T12:34:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.565+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2a806f0f0 con 0x7fd2a80c94f0 2024-04-25T12:34:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.568+0000 7fd2a4c00640 1 -- 172.21.15.123:0/2262634480 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fd290062890 con 0x7fd2a80c94f0 2024-04-25T12:34:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:20.569 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.570+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2840744e0 msgr2=0x7fd2840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2840744e0 0x7fd2840769f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd298037000 tx=0x7fd298032000 comp rx=0 tx=0).stop 2024-04-25T12:34:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a80c94f0 msgr2=0x7fd2a815c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a80c94f0 0x7fd2a815c620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c440 tx=0x7fd29000c900 comp rx=0 tx=0).stop 2024-04-25T12:34:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 shutdown_connections 2024-04-25T12:34:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2840744e0 0x7fd2840769f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2a80c9900 0x7fd2a815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a80c94f0 0x7fd2a815c620 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 --2- 172.21.15.123:0/2262634480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2a8072d70 0x7fd2a80b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 >> 172.21.15.123:0/2262634480 conn(0x7fd2a80cd600 msgr2=0x7fd2a813f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 shutdown_connections 2024-04-25T12:34:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.571+0000 7fd2adc00640 1 -- 172.21.15.123:0/2262634480 wait complete. 2024-04-25T12:34:20.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:34:20.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.894+0000 7fa4fbe00640 1 Processor -- start 2024-04-25T12:34:20.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.895+0000 7fa4fbe00640 1 -- start start 2024-04-25T12:34:20.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4fbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4fc0cd030 0x7fa4fc0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4fbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc1533c0 0x7fa4fc0c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4fbe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4fc0c6030 0x7fa4fc15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4fbe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa4fc0c76c0 con 0x7fa4fc0cd030 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4fbe00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa4fc0c54e0 con 0x7fa4fc0c6030 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4fbe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc159b10 con 0x7fa4fc1533c0 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4faa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4fc0cd030 0x7fa4fc0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4fb400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4fc0c6030 0x7fa4fc15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.897+0000 7fa4f3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc1533c0 0x7fa4fc0c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4f3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc1533c0 0x7fa4fc0c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37524/0 (socket says 172.21.15.123:37524) 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4f3e00640 1 -- 172.21.15.123:0/1947295246 learned_addr learned my addr 172.21.15.123:0/1947295246 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4fb400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4fc0c6030 0x7fa4fc15c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:42206/0 (socket says 172.21.15.123:42206) 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4faa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4fc0cd030 0x7fa4fc0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41284/0 (socket says 172.21.15.123:41284) 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4f3e00640 1 -- 172.21.15.123:0/1947295246 >> v2:172.21.15.153:3300/0 conn(0x7fa4fc0c6030 msgr2=0x7fa4fc15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4f3e00640 1 --2- 172.21.15.123:0/1947295246 >> v2:172.21.15.153:3300/0 conn(0x7fa4fc0c6030 0x7fa4fc15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4f3e00640 1 -- 172.21.15.123:0/1947295246 >> v2:172.21.15.123:3300/0 conn(0x7fa4fc0cd030 msgr2=0x7fa4fc0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4f3e00640 1 --2- 172.21.15.123:0/1947295246 >> v2:172.21.15.123:3300/0 conn(0x7fa4fc0cd030 0x7fa4fc0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4f3e00640 1 -- 172.21.15.123:0/1947295246 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc06c570 con 0x7fa4fc1533c0 2024-04-25T12:34:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4fb400640 1 --2- 172.21.15.123:0/1947295246 >> v2:172.21.15.153:3300/0 conn(0x7fa4fc0c6030 0x7fa4fc15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.898+0000 7fa4faa00640 1 --2- 172.21.15.123:0/1947295246 >> v2:172.21.15.123:3300/0 conn(0x7fa4fc0cd030 0x7fa4fc0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.899+0000 7fa4f3e00640 1 --2- 172.21.15.123:0/1947295246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc1533c0 0x7fa4fc0c5af0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0c5960 tx=0x7fa4e402a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa865657a907d08d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.900+0000 7fa4fa000640 1 -- 172.21.15.123:0/1947295246 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4037070 con 0x7fa4fc1533c0 2024-04-25T12:34:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.900+0000 7fa4fa000640 1 -- 172.21.15.123:0/1947295246 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4032440 con 0x7fa4fc1533c0 2024-04-25T12:34:20.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.900+0000 7fa4fa000640 1 -- 172.21.15.123:0/1947295246 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4032810 con 0x7fa4fc1533c0 2024-04-25T12:34:20.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 -- 172.21.15.123:0/1947295246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc1533c0 msgr2=0x7fa4fc0c5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/1947295246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc1533c0 0x7fa4fc0c5af0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0c5960 tx=0x7fa4e402a1d0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 -- 172.21.15.123:0/1947295246 shutdown_connections 2024-04-25T12:34:20.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/1947295246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc1533c0 0x7fa4fc0c5af0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/1947295246 >> v2:172.21.15.153:3300/0 conn(0x7fa4fc0c6030 0x7fa4fc15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/1947295246 >> v2:172.21.15.123:3300/0 conn(0x7fa4fc0cd030 0x7fa4fc0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 -- 172.21.15.123:0/1947295246 >> 172.21.15.123:0/1947295246 conn(0x7fa4fc0ca780 msgr2=0x7fa4fc0cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 -- 172.21.15.123:0/1947295246 shutdown_connections 2024-04-25T12:34:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.901+0000 7fa4fbe00640 1 -- 172.21.15.123:0/1947295246 wait complete. 2024-04-25T12:34:20.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.903+0000 7fa4fbe00640 1 Processor -- start 2024-04-25T12:34:20.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.903+0000 7fa4fbe00640 1 -- start start 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4fc0c6030 0x7fa4fc1e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4fbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc0cd030 0x7fa4fc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4fbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc1533c0 0x7fa4fc14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4fbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc0c8180 con 0x7fa4fc0c6030 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4fbe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc159fa0 con 0x7fa4fc1533c0 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4fbe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc0cc490 con 0x7fa4fc0cd030 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4fc0c6030 0x7fa4fc1e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4fc0c6030 0x7fa4fc1e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41288/0 (socket says 172.21.15.123:41288) 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4faa00640 1 -- 172.21.15.123:0/383388414 learned_addr learned my addr 172.21.15.123:0/383388414 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4fb400640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc1533c0 0x7fa4fc14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.905+0000 7fa4f3e00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc0cd030 0x7fa4fc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4faa00640 1 -- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc0cd030 msgr2=0x7fa4fc155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4faa00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc0cd030 0x7fa4fc155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4faa00640 1 -- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc1533c0 msgr2=0x7fa4fc14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4faa00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc1533c0 0x7fa4fc14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4faa00640 1 -- 172.21.15.123:0/383388414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc1e5080 con 0x7fa4fc0c6030 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4faa00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4fc0c6030 0x7fa4fc1e4b40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e990 tx=0x7fa4ec00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4f3400640 1 -- 172.21.15.123:0/383388414 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec017070 con 0x7fa4fc0c6030 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4f3400640 1 -- 172.21.15.123:0/383388414 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec00c940 con 0x7fa4fc0c6030 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc1f4ab0 con 0x7fa4fc0c6030 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4f3400640 1 -- 172.21.15.123:0/383388414 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00cd10 con 0x7fa4fc0c6030 2024-04-25T12:34:20.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.906+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc1f44f0 con 0x7fa4fc0c6030 2024-04-25T12:34:20.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.907+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4fc076da0 con 0x7fa4fc0c6030 2024-04-25T12:34:20.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.909+0000 7fa4f3400640 1 -- 172.21.15.123:0/383388414 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa4ec01a070 con 0x7fa4fc0c6030 2024-04-25T12:34:20.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.910+0000 7fa4f3400640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa4d80744e0 0x7fa4d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:20.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.910+0000 7fa4f3400640 1 -- 172.21.15.123:0/383388414 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4ec095350 con 0x7fa4fc0c6030 2024-04-25T12:34:20.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.910+0000 7fa4f3e00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa4d80744e0 0x7fa4d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:20.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.911+0000 7fa4f3e00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa4d80744e0 0x7fa4d80769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4034000 tx=0x7fa4e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:20.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:20.914+0000 7fa4f3400640 1 -- 172.21.15.123:0/383388414 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4ec05f0b0 con 0x7fa4fc0c6030 2024-04-25T12:34:21.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.160+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4fc0cc720 con 0x7fa4fc0c6030 2024-04-25T12:34:21.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.161+0000 7fa4f3400640 1 -- 172.21.15.123:0/383388414 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+6837 (secure 0 0 0) 0x7fa4ec0633b0 con 0x7fa4fc0c6030 2024-04-25T12:34:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:21.162 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:18.974727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T12:34:21.162 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 6 2024-04-25T12:34:21.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa4d80744e0 msgr2=0x7fa4d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa4d80744e0 0x7fa4d80769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4034000 tx=0x7fa4e402e000 comp rx=0 tx=0).stop 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4fc0c6030 msgr2=0x7fa4fc1e4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4fc0c6030 0x7fa4fc1e4b40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e990 tx=0x7fa4ec00ed90 comp rx=0 tx=0).stop 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 shutdown_connections 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa4d80744e0 0x7fa4d80769f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4fc1533c0 0x7fa4fc14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4fc0cd030 0x7fa4fc155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 --2- 172.21.15.123:0/383388414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4fc0c6030 0x7fa4fc1e4b40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 >> 172.21.15.123:0/383388414 conn(0x7fa4fc0ca780 msgr2=0x7fa4fc06d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:21.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 shutdown_connections 2024-04-25T12:34:21.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.164+0000 7fa4fbe00640 1 -- 172.21.15.123:0/383388414 wait complete. 2024-04-25T12:34:21.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.452+0000 7fce47a00640 1 Processor -- start 2024-04-25T12:34:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.453+0000 7fce47a00640 1 -- start start 2024-04-25T12:34:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce47a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fce400c90d0 0x7fce400c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce47a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce4015a730 0x7fce4015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce47a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce400c4810 con 0x7fce400c4240 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce47a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fce4015b080 con 0x7fce400c90d0 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce47a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fce400ce380 con 0x7fce4015a730 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce45a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce4015a730 0x7fce4015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce45000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fce400c90d0 0x7fce400c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce37e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fce400c4240 0x7fce400cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60236/0 (socket says 172.21.15.123:60236) 2024-04-25T12:34:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.455+0000 7fce37e00640 1 -- 172.21.15.123:0/2728762646 learned_addr learned my addr 172.21.15.123:0/2728762646 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.456+0000 7fce3fe00640 1 -- 172.21.15.123:0/2728762646 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1611296443 0 0) 0x7fce400c4810 con 0x7fce400c4240 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.456+0000 7fce3fe00640 1 -- 172.21.15.123:0/2728762646 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce24003520 con 0x7fce400c4240 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.456+0000 7fce45a00640 1 -- 172.21.15.123:0/2728762646 >> v2:172.21.15.153:3300/0 conn(0x7fce400c90d0 msgr2=0x7fce400c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.456+0000 7fce45a00640 1 --2- 172.21.15.123:0/2728762646 >> v2:172.21.15.153:3300/0 conn(0x7fce400c90d0 0x7fce400c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.456+0000 7fce45a00640 1 -- 172.21.15.123:0/2728762646 >> v1:172.21.15.153:6789/0 conn(0x7fce400c4240 legacy=0x7fce400cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.456+0000 7fce45a00640 1 -- 172.21.15.123:0/2728762646 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce400c83f0 con 0x7fce4015a730 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.456+0000 7fce45000640 1 --2- 172.21.15.123:0/2728762646 >> v2:172.21.15.153:3300/0 conn(0x7fce400c90d0 0x7fce400c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.457+0000 7fce45a00640 1 --2- 172.21.15.123:0/2728762646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce4015a730 0x7fce4015ab40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fce38009dc0 tx=0x7fce3802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9703148a3a6a3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.457+0000 7fce3fe00640 1 -- 172.21.15.123:0/2728762646 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38007740 con 0x7fce4015a730 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.457+0000 7fce3fe00640 1 -- 172.21.15.123:0/2728762646 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce380078e0 con 0x7fce4015a730 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.457+0000 7fce3fe00640 1 -- 172.21.15.123:0/2728762646 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38007c90 con 0x7fce4015a730 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 -- 172.21.15.123:0/2728762646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce4015a730 msgr2=0x7fce4015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 --2- 172.21.15.123:0/2728762646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce4015a730 0x7fce4015ab40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fce38009dc0 tx=0x7fce3802a710 comp rx=0 tx=0).stop 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 -- 172.21.15.123:0/2728762646 shutdown_connections 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 --2- 172.21.15.123:0/2728762646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce4015a730 0x7fce4015ab40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 --2- 172.21.15.123:0/2728762646 >> v2:172.21.15.153:3300/0 conn(0x7fce400c90d0 0x7fce400c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 -- 172.21.15.123:0/2728762646 >> 172.21.15.123:0/2728762646 conn(0x7fce4006c750 msgr2=0x7fce4006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 -- 172.21.15.123:0/2728762646 shutdown_connections 2024-04-25T12:34:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.458+0000 7fce47a00640 1 -- 172.21.15.123:0/2728762646 wait complete. 2024-04-25T12:34:21.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.460+0000 7fce47a00640 1 Processor -- start 2024-04-25T12:34:21.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.461+0000 7fce47a00640 1 -- start start 2024-04-25T12:34:21.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce47a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce400c4240 0x7fce4014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:21.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce47a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce400c90d0 0x7fce401fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:21.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce47a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4015a730 0x7fce40154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:21.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce47a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce400c8c90 con 0x7fce400c4240 2024-04-25T12:34:21.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce47a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fce400c89b0 con 0x7fce4015a730 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce47a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fce40073db0 con 0x7fce400c90d0 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce45000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce400c4240 0x7fce4014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce45000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce400c4240 0x7fce4014ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41302/0 (socket says 172.21.15.123:41302) 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce45000640 1 -- 172.21.15.123:0/1698593991 learned_addr learned my addr 172.21.15.123:0/1698593991 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce37e00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce400c90d0 0x7fce401fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.462+0000 7fce45a00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4015a730 0x7fce40154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce45000640 1 -- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce400c90d0 msgr2=0x7fce401fa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce45000640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce400c90d0 0x7fce401fa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce45000640 1 -- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4015a730 msgr2=0x7fce40154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce45000640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4015a730 0x7fce40154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce45000640 1 -- 172.21.15.123:0/1698593991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce401fae00 con 0x7fce400c4240 2024-04-25T12:34:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce45000640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce400c4240 0x7fce4014ce30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e990 tx=0x7fce3000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:21.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce3d600640 1 -- 172.21.15.123:0/1698593991 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30017070 con 0x7fce400c4240 2024-04-25T12:34:21.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce3d600640 1 -- 172.21.15.123:0/1698593991 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3000c9e0 con 0x7fce400c4240 2024-04-25T12:34:21.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce401f7870 con 0x7fce400c4240 2024-04-25T12:34:21.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce3d600640 1 -- 172.21.15.123:0/1698593991 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3000cdb0 con 0x7fce400c4240 2024-04-25T12:34:21.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.463+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce401f6d90 con 0x7fce400c4240 2024-04-25T12:34:21.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.465+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4006d500 con 0x7fce400c4240 2024-04-25T12:34:21.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.466+0000 7fce3d600640 1 -- 172.21.15.123:0/1698593991 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce3001a070 con 0x7fce400c4240 2024-04-25T12:34:21.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.467+0000 7fce3d600640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce1c0744e0 0x7fce1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:21.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.467+0000 7fce3d600640 1 -- 172.21.15.123:0/1698593991 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce30095360 con 0x7fce400c4240 2024-04-25T12:34:21.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.467+0000 7fce37e00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce1c0744e0 0x7fce1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.468+0000 7fce37e00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce1c0744e0 0x7fce1c0769f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce4015e980 tx=0x7fce28009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:21.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.471+0000 7fce3d600640 1 -- 172.21.15.123:0/1698593991 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce3005f150 con 0x7fce400c4240 2024-04-25T12:34:21.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.672+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce40076700 con 0x7fce400c4240 2024-04-25T12:34:21.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.674+0000 7fce3d600640 1 -- 172.21.15.123:0/1698593991 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fce30063450 con 0x7fce400c4240 2024-04-25T12:34:21.675 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:21.675 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce1c0744e0 msgr2=0x7fce1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce1c0744e0 0x7fce1c0769f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce4015e980 tx=0x7fce28009040 comp rx=0 tx=0).stop 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce400c4240 msgr2=0x7fce4014ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce400c4240 0x7fce4014ce30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e990 tx=0x7fce3000ee50 comp rx=0 tx=0).stop 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 shutdown_connections 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce1c0744e0 0x7fce1c0769f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4015a730 0x7fce40154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce400c90d0 0x7fce401fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 --2- 172.21.15.123:0/1698593991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce400c4240 0x7fce4014ce30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:21.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 >> 172.21.15.123:0/1698593991 conn(0x7fce4006c750 msgr2=0x7fce400cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:21.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 shutdown_connections 2024-04-25T12:34:21.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:21.677+0000 7fce47a00640 1 -- 172.21.15.123:0/1698593991 wait complete. 2024-04-25T12:34:22.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T12:34:13.424937+0000', 'modified': '2024-04-25T12:34:18.974727+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6834/1720190095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1720190095}, {'type': 'v1', 'addr': '172.21.15.123:6834', 'nonce': 1720190095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-04-25T12:34:22.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T12:34:22.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T12:34:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.579+0000 7fd6e2000640 1 Processor -- start 2024-04-25T12:34:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.579+0000 7fd6e2000640 1 -- start start 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.581+0000 7fd6e2000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd6dc15ca70 0x7fd6dc15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.581+0000 7fd6e2000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc06ecd0 con 0x7fd6dc1567b0 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.581+0000 7fd6e2000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc06efb0 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6e2000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd6dc06f290 con 0x7fd6dc15ca70 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6e0c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd6dc1567b0 0x7fd6dc06c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55944/0 (socket says 172.21.15.123:55944) 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6e0c00640 1 -- 172.21.15.123:0/1511575142 learned_addr learned my addr 172.21.15.123:0/1511575142 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/1511575142 >> v2:172.21.15.123:3300/0 conn(0x7fd6dc15ca70 0x7fd6dc15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6db400640 1 -- 172.21.15.123:0/1511575142 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055701530 0 0) 0x7fd6dc06ecd0 con 0x7fd6dc1567b0 2024-04-25T12:34:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6db400640 1 -- 172.21.15.123:0/1511575142 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6c4003520 con 0x7fd6dc1567b0 2024-04-25T12:34:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6db400640 1 -- 172.21.15.123:0/1511575142 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880980952 0 0) 0x7fd6dc06efb0 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6db400640 1 -- 172.21.15.123:0/1511575142 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6dc06ecd0 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1511575142 >> v1:172.21.15.153:6789/0 conn(0x7fd6dc0c82b0 legacy=0x7fd6dc0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1511575142 >> v1:172.21.15.123:6789/0 conn(0x7fd6dc1567b0 legacy=0x7fd6dc06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.582+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1511575142 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc06f570 con 0x7fd6dc15ca70 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.584+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/1511575142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc15ca70 0x7fd6dc15ce80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80097a0 tx=0x7fd6c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d272f07faaff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.584+0000 7fd6db400640 1 -- 172.21.15.123:0/1511575142 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8037070 con 0x7fd6dc15ca70 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.584+0000 7fd6db400640 1 -- 172.21.15.123:0/1511575142 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c8032440 con 0x7fd6dc15ca70 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.584+0000 7fd6db400640 1 -- 172.21.15.123:0/1511575142 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80327f0 con 0x7fd6dc15ca70 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.584+0000 7fd6e2000640 1 -- 172.21.15.123:0/1511575142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc15ca70 msgr2=0x7fd6dc15ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.584+0000 7fd6e2000640 1 --2- 172.21.15.123:0/1511575142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc15ca70 0x7fd6dc15ce80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80097a0 tx=0x7fd6c802a230 comp rx=0 tx=0).stop 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.585+0000 7fd6e2000640 1 -- 172.21.15.123:0/1511575142 shutdown_connections 2024-04-25T12:34:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.585+0000 7fd6e2000640 1 --2- 172.21.15.123:0/1511575142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc15ca70 0x7fd6dc15ce80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.585+0000 7fd6e2000640 1 -- 172.21.15.123:0/1511575142 >> 172.21.15.123:0/1511575142 conn(0x7fd6dc0cd600 msgr2=0x7fd6dc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:22.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.585+0000 7fd6e2000640 1 -- 172.21.15.123:0/1511575142 shutdown_connections 2024-04-25T12:34:22.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.585+0000 7fd6e2000640 1 -- 172.21.15.123:0/1511575142 wait complete. 2024-04-25T12:34:22.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.587+0000 7fd6e2000640 1 Processor -- start 2024-04-25T12:34:22.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.587+0000 7fd6e2000640 1 -- start start 2024-04-25T12:34:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc0c82b0 0x7fd6dc0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc1567b0 0x7fd6dc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e2000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6dc15ca70 0x7fd6dc148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e2000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc0c9d20 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e2000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc15d3c0 con 0x7fd6dc1567b0 2024-04-25T12:34:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e2000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc156bc0 con 0x7fd6dc15ca70 2024-04-25T12:34:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc0c82b0 0x7fd6dc0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc0c82b0 0x7fd6dc0b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41330/0 (socket says 172.21.15.123:41330) 2024-04-25T12:34:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 -- 172.21.15.123:0/3756683919 learned_addr learned my addr 172.21.15.123:0/3756683919 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc1567b0 0x7fd6dc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e1600640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6dc15ca70 0x7fd6dc148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 -- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6dc15ca70 msgr2=0x7fd6dc148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6dc15ca70 0x7fd6dc148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 -- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc1567b0 msgr2=0x7fd6dc0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc1567b0 0x7fd6dc0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.589+0000 7fd6e0c00640 1 -- 172.21.15.123:0/3756683919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc0b1030 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.590+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc0c82b0 0x7fd6dc0b0af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c440 tx=0x7fd6d000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.590+0000 7fd6d8c00640 1 -- 172.21.15.123:0/3756683919 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0017070 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.590+0000 7fd6d8c00640 1 -- 172.21.15.123:0/3756683919 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0005d00 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.590+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc1f2c50 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.590+0000 7fd6d8c00640 1 -- 172.21.15.123:0/3756683919 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00072b0 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.590+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc14f0d0 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.592+0000 7fd6d8c00640 1 -- 172.21.15.123:0/3756683919 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6d001a070 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.593+0000 7fd6d8c00640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd6bc0744e0 0x7fd6bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.594+0000 7fd6d8c00640 1 -- 172.21.15.123:0/3756683919 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6d00947d0 con 0x7fd6dc0c82b0 2024-04-25T12:34:22.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.594+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd6bc0744e0 0x7fd6bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.594+0000 7fd6ea5c7740 1 Processor -- start 2024-04-25T12:34:22.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.594+0000 7fd6ea5c7740 1 -- start start 2024-04-25T12:34:22.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.594+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd6bc0744e0 0x7fd6bc0769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8034000 tx=0x7fd6c802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:22.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e3f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e3d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d1e316600 con 0x557d1dfe73b0 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d1e2b3100 con 0x557d1dfef590 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x557d1e3125c0 con 0x557d1e010270 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e3f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e3f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37566/0 (socket says 172.21.15.123:37566) 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 -- 172.21.15.123:0/4256596331 learned_addr learned my addr 172.21.15.123:0/4256596331 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e1600640 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e3d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 -- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 msgr2=0x557d1e3d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e3d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 -- 172.21.15.123:0/4256596331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 msgr2=0x557d1e3d7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3d7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.596+0000 7fd6e0c00640 1 -- 172.21.15.123:0/4256596331 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1e0282d0 con 0x557d1e010270 2024-04-25T12:34:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.597+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e3f3fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd6d005a790 tx=0x7fd6d005e2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bfe7c40b12640b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.597+0000 7fd6cd200640 1 -- 172.21.15.123:0/4256596331 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0067430 con 0x557d1e010270 2024-04-25T12:34:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.597+0000 7fd6cd200640 1 -- 172.21.15.123:0/4256596331 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0066fa0 con 0x557d1e010270 2024-04-25T12:34:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.597+0000 7fd6cd200640 1 -- 172.21.15.123:0/4256596331 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0063ba0 con 0x557d1e010270 2024-04-25T12:34:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 msgr2=0x557d1e3f3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e3f3fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd6d005a790 tx=0x7fd6d005e2e0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/4256596331 shutdown_connections 2024-04-25T12:34:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e3d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/4256596331 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e3f3fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/4256596331 >> 172.21.15.123:0/4256596331 conn(0x557d1e0043c0 msgr2=0x557d1e3b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/4256596331 shutdown_connections 2024-04-25T12:34:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.598+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/4256596331 wait complete. 2024-04-25T12:34:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.599+0000 7fd6ea5c7740 1 -- start start 2024-04-25T12:34:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.599+0000 7fd6ea5c7740 1 Processor -- start 2024-04-25T12:34:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e3f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1e010270 0x557d1e3ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d1e3b1a60 con 0x557d1dfe73b0 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d1e29df70 con 0x557d1e010270 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x557d1e3f7a40 con 0x557d1dfef590 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6ea5c7740 10 client.0 _ll_register_callbacks cb 0x557d1e30c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6e1600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1e010270 0x557d1e3ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e3f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6e1600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1e010270 0x557d1e3ed1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:42254/0 (socket says 172.21.15.123:42254) 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.601+0000 7fd6e1600640 1 -- 172.21.15.123:0/203170692 learned_addr learned my addr 172.21.15.123:0/203170692 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6e1600640 1 -- 172.21.15.123:0/203170692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 msgr2=0x557d1e3f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6e1600640 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e3f6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6e1600640 1 -- 172.21.15.123:0/203170692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 msgr2=0x557d1e3f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6e1600640 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6e1600640 1 -- 172.21.15.123:0/203170692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1e0f26b0 con 0x557d1e010270 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6e1600640 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1e010270 0x557d1e3ed1c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4012690 tx=0x7fd6d4012b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6268d49097151075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6c1600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6c1600640 1 -- 172.21.15.123:0/203170692 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4019020 con 0x557d1e010270 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d0066fa0 con 0x557d1e010270 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6c1600640 1 -- 172.21.15.123:0/203170692 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6d4003970 con 0x557d1e010270 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6c1600640 1 -- 172.21.15.123:0/203170692 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd6d4003b70 con 0x557d1e010270 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6ea5c7740 10 client.4420 fetch_fsmap learned FSMap version 6 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.602+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x557d1e3edd20 con 0x557d1e010270 2024-04-25T12:34:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6c1600640 1 -- 172.21.15.123:0/203170692 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0066fa0 con 0x557d1e010270 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6c1600640 1 -- 172.21.15.123:0/203170692 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 6) ==== 4712+0+0 (secure 0 0 0) 0x7fd6d4026080 con 0x557d1e010270 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6ea5c7740 10 client.4420 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6ea5c7740 10 client.4420 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6ea5c7740 20 client.4420 populate_metadata read hostname 'smithi123' 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1dfcee40 0x557d1e412140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6ea5c7740 4 client.4420 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 --> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d1e324210 con 0x557d1dfcee40 2024-04-25T12:34:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.603+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1dfcee40 0x557d1e412140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.604+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1dfcee40 0x557d1e412140 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d471efbbb92395 server_cookie=1f584838bc7a6d48 in_seq=0 out_seq=0 2024-04-25T12:34:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:22.604+0000 7fd6c1600640 10 client.4420 ms_handle_connect on v2:172.21.15.123:6832/1720190095 2024-04-25T12:34:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.981+0000 7fd6c1600640 1 -- 172.21.15.123:0/203170692 <== mds.0 v2:172.21.15.123:6832/1720190095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d1e324210 con 0x557d1dfcee40 2024-04-25T12:34:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.981+0000 7fd6c1600640 10 client.4420 handle_command_reply: tid=0 2024-04-25T12:34:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.981+0000 7fd6ea5c7740 1 client.4420 shutdown 2024-04-25T12:34:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.981+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1e010270 msgr2=0x557d1e3ed1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.981+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1e010270 0x557d1e3ed1c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4012690 tx=0x7fd6d4012b50 comp rx=0 tx=0).stop 2024-04-25T12:34:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.981+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 shutdown_connections 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1e010270 0x557d1e3ed1c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e3f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1dfcee40 0x557d1e412140 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/203170692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 >> 172.21.15.123:0/203170692 conn(0x557d1e0043c0 msgr2=0x557d1e3dc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6c1600640 0 client.4420 ms_handle_reset on v2:172.21.15.123:6832/1720190095 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 shutdown_connections 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/203170692 wait complete. 2024-04-25T12:34:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:23.982+0000 7fd6ea5c7740 20 client.4420 trim_cache size 0 max 16384 2024-04-25T12:34:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.027+0000 7fd6ea5c7740 1 Processor -- start 2024-04-25T12:34:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.027+0000 7fd6ea5c7740 1 -- start start 2024-04-25T12:34:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.030+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfe73b0 0x557d1e3f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.030+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e4035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.030+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1e010270 0x557d1e403b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.030+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1fe9e0 con 0x557d1e010270 2024-04-25T12:34:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.030+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1ff040 con 0x557d1dfe73b0 2024-04-25T12:34:24.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.030+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x557d1e3ef080 con 0x557d1dfef590 2024-04-25T12:34:24.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1e010270 0x557d1e403b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfe73b0 0x557d1e3f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e4035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfe73b0 0x557d1e3f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34804/0 (socket says 172.21.15.123:34804) 2024-04-25T12:34:24.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1e010270 0x557d1e403b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36664/0 (socket says 172.21.15.123:36664) 2024-04-25T12:34:24.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 -- 172.21.15.123:0/883090913 learned_addr learned my addr 172.21.15.123:0/883090913 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 -- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 msgr2=0x557d1e4035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e4035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 -- 172.21.15.123:0/883090913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1e010270 msgr2=0x557d1e403b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1e010270 0x557d1e403b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e0c00640 1 -- 172.21.15.123:0/883090913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1e3f7840 con 0x557d1dfe73b0 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e4035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.031+0000 7fd6e1600640 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1e010270 0x557d1e403b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.032+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfe73b0 0x557d1e3f6f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6d005ad00 tx=0x7fd6d0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a52b5712ee5eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.032+0000 7fd6c3e00640 1 -- 172.21.15.123:0/883090913 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0066fa0 con 0x557d1dfe73b0 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.032+0000 7fd6c3e00640 1 -- 172.21.15.123:0/883090913 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0067140 con 0x557d1dfe73b0 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.032+0000 7fd6c3e00640 1 -- 172.21.15.123:0/883090913 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0063dc0 con 0x557d1dfe73b0 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfe73b0 msgr2=0x557d1e3f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfe73b0 0x557d1e3f6f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6d005ad00 tx=0x7fd6d0014000 comp rx=0 tx=0).stop 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/883090913 shutdown_connections 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1e010270 0x557d1e403b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1dfef590 0x557d1e4035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/883090913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfe73b0 0x557d1e3f6f60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/883090913 >> 172.21.15.123:0/883090913 conn(0x557d1e0043c0 msgr2=0x557d1e3ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/883090913 shutdown_connections 2024-04-25T12:34:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.033+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/883090913 wait complete. 2024-04-25T12:34:24.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.034+0000 7fd6ea5c7740 1 -- start start 2024-04-25T12:34:24.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.035+0000 7fd6ea5c7740 1 Processor -- start 2024-04-25T12:34:24.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.036+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.036+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e4038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.036+0000 7fd6ea5c7740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e46baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.036+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d1e407920 con 0x557d1dfe73b0 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.036+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d1e407290 con 0x557d1dfef590 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.036+0000 7fd6ea5c7740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x557d1e40e260 con 0x557d1e010270 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.036+0000 7fd6ea5c7740 10 client.0 _ll_register_callbacks cb 0x7fd6d40019b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e4038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6e1600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e46baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e4038a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34812/0 (socket says 172.21.15.123:34812) 2024-04-25T12:34:24.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 -- 172.21.15.123:0/2815864581 learned_addr learned my addr 172.21.15.123:0/2815864581 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:24.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36672/0 (socket says 172.21.15.123:36672) 2024-04-25T12:34:24.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 -- 172.21.15.123:0/2815864581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 msgr2=0x557d1e46baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:24.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e46baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 -- 172.21.15.123:0/2815864581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 msgr2=0x557d1e3f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:24.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.037+0000 7fd6dbe00640 1 -- 172.21.15.123:0/2815864581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1e403de0 con 0x557d1dfef590 2024-04-25T12:34:24.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e4038a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8000c00 tx=0x7fd6c802c070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebfb14813215b47b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:24.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6c1600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-04-25T12:34:24.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6c1600640 1 -- 172.21.15.123:0/2815864581 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8037040 con 0x557d1dfef590 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6c1600640 1 -- 172.21.15.123:0/2815864581 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6c803ed00 con 0x557d1dfef590 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d0067140 con 0x557d1dfef590 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6c1600640 1 -- 172.21.15.123:0/2815864581 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd6c805a050 con 0x557d1dfef590 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6ea5c7740 10 client.4426 fetch_fsmap learned FSMap version 6 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x557d1e3b2fe0 con 0x557d1dfef590 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.038+0000 7fd6c1600640 1 -- 172.21.15.123:0/2815864581 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0067140 con 0x557d1dfef590 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6c1600640 1 -- 172.21.15.123:0/2815864581 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 6) ==== 4712+0+0 (secure 0 0 0) 0x7fd6c805d020 con 0x557d1dfef590 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6ea5c7740 10 client.4426 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6ea5c7740 10 client.4426 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6ea5c7740 20 client.4426 populate_metadata read hostname 'smithi123' 2024-04-25T12:34:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1e3fd930 0x557d1e47e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6ea5c7740 4 client.4426 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 --> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] -- command(tid 0: {"prefix": "status"}) -- 0x557d1e31e080 con 0x557d1e3fd930 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.039+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1e3fd930 0x557d1e47e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.040+0000 7fd6c1600640 10 client.4426 ms_handle_connect on v2:172.21.15.123:6832/1720190095 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.040+0000 7fd6e0c00640 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1e3fd930 0x557d1e47e4d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e71df0c32101c0 server_cookie=f02a5ac618b557b8 in_seq=0 out_seq=0 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.040+0000 7fd6c1600640 1 -- 172.21.15.123:0/2815864581 <== mds.0 v2:172.21.15.123:6832/1720190095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x557d1e31e080 con 0x557d1e3fd930 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.040+0000 7fd6c1600640 10 client.4426 handle_command_reply: tid=0 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.040+0000 7fd6ea5c7740 1 client.4426 shutdown 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 msgr2=0x557d1e4038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e4038a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8000c00 tx=0x7fd6c802c070 comp rx=0 tx=0).stop 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 shutdown_connections 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x557d1e010270 0x557d1e46baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1dfef590 0x557d1e4038a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x557d1e3fd930 0x557d1e47e4d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 --2- 172.21.15.123:0/2815864581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d1dfe73b0 0x557d1e3f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 >> 172.21.15.123:0/2815864581 conn(0x557d1e0043c0 msgr2=0x557d1e3f6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6c1600640 0 client.4426 ms_handle_reset on v2:172.21.15.123:6832/1720190095 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 shutdown_connections 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 1 -- 172.21.15.123:0/2815864581 wait complete. 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.041+0000 7fd6ea5c7740 20 client.4426 trim_cache size 0 max 16384 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:34:24.045 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4270, 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 10.101880928, 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 5, 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 25, 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 16.397363476999999 2024-04-25T12:34:24.046 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:34:24.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd6bc0744e0 msgr2=0x7fd6bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd6bc0744e0 0x7fd6bc0769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8034000 tx=0x7fd6c802e000 comp rx=0 tx=0).stop 2024-04-25T12:34:24.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc0c82b0 msgr2=0x7fd6dc0b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc0c82b0 0x7fd6dc0b0af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c440 tx=0x7fd6d000c900 comp rx=0 tx=0).stop 2024-04-25T12:34:24.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 shutdown_connections 2024-04-25T12:34:24.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd6bc0744e0 0x7fd6bc0769f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6dc15ca70 0x7fd6dc148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc1567b0 0x7fd6dc0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 --2- 172.21.15.123:0/3756683919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc0c82b0 0x7fd6dc0b0af0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 >> 172.21.15.123:0/3756683919 conn(0x7fd6dc0cd600 msgr2=0x7fd6dc073450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:24.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 shutdown_connections 2024-04-25T12:34:24.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.042+0000 7fd6e2000640 1 -- 172.21.15.123:0/3756683919 wait complete. 2024-04-25T12:34:24.073 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T12:34:24.074 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:24.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T12:34:24.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.334+0000 7f3180a00640 1 Processor -- start 2024-04-25T12:34:24.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.335+0000 7f3180a00640 1 -- start start 2024-04-25T12:34:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.336+0000 7f3180a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f317c0c92a0 0x7f317c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.336+0000 7f3180a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.336+0000 7f3180a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c073dc0 con 0x7f317c0c8390 2024-04-25T12:34:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3180a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f317c0ca280 con 0x7f317c0c92a0 2024-04-25T12:34:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3180a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f317c0c5010 con 0x7f317c0c9930 2024-04-25T12:34:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3179e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f317c0c92a0 0x7f317c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f317a800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3179e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f317c0c92a0 0x7f317c156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34814/0 (socket says 172.21.15.123:34814) 2024-04-25T12:34:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f317a800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c0c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:53978/0 (socket says 172.21.15.123:53978) 2024-04-25T12:34:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3179e00640 1 -- 172.21.15.123:0/268222204 learned_addr learned my addr 172.21.15.123:0/268222204 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3179e00640 1 -- 172.21.15.123:0/268222204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 msgr2=0x7f317c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3179e00640 1 --2- 172.21.15.123:0/268222204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3179e00640 1 -- 172.21.15.123:0/268222204 >> v1:172.21.15.123:6789/0 conn(0x7f317c0c8390 legacy=0x7f317c15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.337+0000 7f3179e00640 1 -- 172.21.15.123:0/268222204 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c0c52f0 con 0x7f317c0c92a0 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.338+0000 7f3179e00640 1 --2- 172.21.15.123:0/268222204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 0x7f317c156ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f316c009870 tx=0x7f316c02a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfe17d81daa203b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3178a00640 1 -- 172.21.15.123:0/268222204 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c037070 con 0x7f317c0c92a0 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3178a00640 1 -- 172.21.15.123:0/268222204 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c032640 con 0x7f317c0c92a0 2024-04-25T12:34:24.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3178a00640 1 -- 172.21.15.123:0/268222204 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c032a10 con 0x7f317c0c92a0 2024-04-25T12:34:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3180a00640 1 -- 172.21.15.123:0/268222204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 msgr2=0x7f317c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3180a00640 1 --2- 172.21.15.123:0/268222204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 0x7f317c156ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f316c009870 tx=0x7f316c02a2e0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3180a00640 1 -- 172.21.15.123:0/268222204 shutdown_connections 2024-04-25T12:34:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3180a00640 1 --2- 172.21.15.123:0/268222204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.339+0000 7f3180a00640 1 --2- 172.21.15.123:0/268222204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 0x7f317c156ad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.340+0000 7f3180a00640 1 -- 172.21.15.123:0/268222204 >> 172.21.15.123:0/268222204 conn(0x7f317c06c990 msgr2=0x7f317c06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:24.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.340+0000 7f3180a00640 1 -- 172.21.15.123:0/268222204 shutdown_connections 2024-04-25T12:34:24.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.340+0000 7f3180a00640 1 -- 172.21.15.123:0/268222204 wait complete. 2024-04-25T12:34:24.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.342+0000 7f3180a00640 1 Processor -- start 2024-04-25T12:34:24.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.342+0000 7f3180a00640 1 -- start start 2024-04-25T12:34:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3180a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f317c0c8390 0x7f317c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3180a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 0x7f317c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3180a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c14c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3180a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f317c073b70 con 0x7f317c0c8390 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3180a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f317c0739f0 con 0x7f317c0c92a0 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3180a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f317c15d4a0 con 0x7f317c0c9930 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f317c0c8390 0x7f317c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f317c0c8390 0x7f317c153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36676/0 (socket says 172.21.15.123:36676) 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 -- 172.21.15.123:0/2876876472 learned_addr learned my addr 172.21.15.123:0/2876876472 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f317a800640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c14c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179400640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 0x7f317c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 -- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 msgr2=0x7f317c14c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c14c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 -- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 msgr2=0x7f317c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 0x7f317c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f3179e00640 1 -- 172.21.15.123:0/2876876472 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c1537d0 con 0x7f317c0c8390 2024-04-25T12:34:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.344+0000 7f317a800640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c14c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.345+0000 7f3179e00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f317c0c8390 0x7f317c153290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.345+0000 7f3172000640 1 -- 172.21.15.123:0/2876876472 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c037040 con 0x7f317c0c8390 2024-04-25T12:34:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.345+0000 7f3172000640 1 -- 172.21.15.123:0/2876876472 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c042070 con 0x7f317c0c8390 2024-04-25T12:34:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.345+0000 7f3172000640 1 -- 172.21.15.123:0/2876876472 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0309b0 con 0x7f317c0c8390 2024-04-25T12:34:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.345+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c15e780 con 0x7f317c0c8390 2024-04-25T12:34:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.345+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c151fd0 con 0x7f317c0c8390 2024-04-25T12:34:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.346+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c076700 con 0x7f317c0c8390 2024-04-25T12:34:24.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.347+0000 7f3172000640 1 -- 172.21.15.123:0/2876876472 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f316c030b50 con 0x7f317c0c8390 2024-04-25T12:34:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.348+0000 7f3172000640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31580744b0 0x7f31580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.349+0000 7f3172000640 1 -- 172.21.15.123:0/2876876472 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f316c0b5cc0 con 0x7f317c0c8390 2024-04-25T12:34:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.349+0000 7f3179400640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31580744b0 0x7f31580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:24.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.350+0000 7f3179400640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31580744b0 0x7f31580769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f317c1f3d10 tx=0x7f3164009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:24.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.353+0000 7f3172000640 1 -- 172.21.15.123:0/2876876472 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f316c0bc080 con 0x7f317c0c8390 2024-04-25T12:34:24.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.595+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f317c0c9d40 con 0x7f317c0c8390 2024-04-25T12:34:24.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.596+0000 7f3172000640 1 -- 172.21.15.123:0/2876876472 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7f316c07fb40 con 0x7f317c0c8390 2024-04-25T12:34:24.598 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T12:34:24.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31580744b0 msgr2=0x7f31580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31580744b0 0x7f31580769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f317c1f3d10 tx=0x7f3164009040 comp rx=0 tx=0).stop 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f317c0c8390 msgr2=0x7f317c153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f317c0c8390 0x7f317c153290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 shutdown_connections 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31580744b0 0x7f31580769c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f317c0c9930 0x7f317c14c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f317c0c92a0 0x7f317c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 --2- 172.21.15.123:0/2876876472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f317c0c8390 0x7f317c153290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.598+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 >> 172.21.15.123:0/2876876472 conn(0x7f317c06c990 msgr2=0x7f317c0c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.599+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 shutdown_connections 2024-04-25T12:34:24.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24.599+0000 7f3180a00640 1 -- 172.21.15.123:0/2876876472 wait complete. 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T12:34:24.627 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T12:34:24.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:24.628 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T12:34:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42699sec preferred_lft 42699sec 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-25T12:34:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:34:24.684 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T12:34:24.684 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:34:24.684 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:34:24.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-25T12:34:24.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-25T12:34:24.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-25T12:34:24.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T12:34:24.685 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:34:24.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:34:25.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:34:25.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:25.167 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T12:34:25.236 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-25T12:34:25.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:25.240 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-25T12:34:25.286 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-25T12:34:25.286 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T12:34:25.286 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T12:34:25.286 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T12:34:25.286 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T12:34:25.286 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:34:25.287 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:34:25.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T12:34:25.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T12:34:25.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T12:34:25.287 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:34:25.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:34:26.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:34:26.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:26.134 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T12:34:26.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:26.185 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-25T12:34:26.233 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:34:26.233 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:34:26.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:34:26.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T12:34:26.233 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:34:26.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:34:26.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:34:26.414 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T12:34:26.414 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:34:26.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:34:26.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:34:26.697 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:34:26.698 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:34:26.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-25T12:34:26.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T12:34:26.698 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:34:26.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:34:26.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:34:26.892 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T12:34:26.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T12:34:26.893 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:26.941 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T12:34:26.941 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T12:34:26.941 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:27.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-25T12:34:27.118 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T12:34:27.166 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-25T12:34:27.166 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-25T12:34:27.166 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-25T12:34:27.167 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T12:34:27.168 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:34:27.169 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:34:27.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:27.171 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-25T12:34:27.216 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T12:34:27.216 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T12:34:27.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-25T12:34:27.337 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T12:34:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-25T12:34:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-25T12:34:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-25T12:34:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-25T12:34:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-25T12:34:27.380 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T12:34:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-25T12:34:27.382 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:34:27.382 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:34:27.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:27.383 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-25T12:34:27.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:2024-04-25T12:34:27.521+0000 7fc51c8ce480 -1 init, newargv = 0x5583c43a4000 newargc=15 2024-04-25T12:34:27.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[38459]: starting ceph client 2024-04-25T12:34:27.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[38459]: starting fuse 2024-04-25T12:34:28.549 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-25T12:34:28.549+0000 7f1a9b000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T12:34:28.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-25T12:34:28.663 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T12:34:28.718 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-25T12:34:28.718 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-25T12:34:28.718 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-25T12:34:28.719 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T12:34:28.720 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:34:28.721 INFO:teuthology.orchestra.run.smithi123.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:28.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:28.723 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-25T12:34:28.777 INFO:teuthology.orchestra.run.smithi123.stdout:40 2024-04-25T12:34:28.777 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-25T12:34:28.778 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.admin")) 2024-04-25T12:34:28.779 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-25T12:34:28.949 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.admin.38459.asok 2024-04-25T12:34:28.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:34:28.966 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.38459.asok 2024-04-25T12:34:28.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:28.967 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.38459.asok status 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "admin", 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "38459", 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-25T12:34:29.256 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4441, 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4441 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 938788427 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-04-25T12:34:29.257 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 938788427 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4441 172.21.15.123:0/938788427", 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "172.21.15.123:0/938788427", 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 5, 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 25, 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-25T12:34:29.258 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:34:29.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:29.276 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-25T12:34:29.277 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:29.324 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-25T12:34:29.324 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:29.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:29.324 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:29.403 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T12:34:29.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T12:34:29.538 INFO:teuthology.orchestra.run.smithi123.stdout:# file: . 2024-04-25T12:34:29.538 INFO:teuthology.orchestra.run.smithi123.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T12:34:29.538 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:29.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:34:29.547 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:34:29.547 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:34:29.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:29.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.888+0000 7fcb34400640 1 Processor -- start 2024-04-25T12:34:29.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.888+0000 7fcb34400640 1 -- start start 2024-04-25T12:34:29.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.890+0000 7fcb34400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c154890 0x7fcb2c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:29.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.890+0000 7fcb34400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2c154ca0 con 0x7fcb2c074870 2024-04-25T12:34:29.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.890+0000 7fcb34400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2c073ce0 con 0x7fcb2c0c4d50 2024-04-25T12:34:29.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.890+0000 7fcb34400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2c0c61f0 con 0x7fcb2c154890 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb32400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c154890 0x7fcb2c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb31000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcb2c074870 0x7fcb2c0cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:52728/0 (socket says 172.21.15.123:52728) 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb31000640 1 -- 172.21.15.123:0/1645816682 learned_addr learned my addr 172.21.15.123:0/1645816682 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb31a00640 1 --1- 172.21.15.123:0/1645816682 >> v1:172.21.15.153:6789/0 conn(0x7fcb2c0c4d50 0x7fcb2c0c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:38772/0 (socket says 172.21.15.123:38772) 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb2be00640 1 -- 172.21.15.123:0/1645816682 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356714821 0 0) 0x7fcb2c154ca0 con 0x7fcb2c074870 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb2be00640 1 -- 172.21.15.123:0/1645816682 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb14003520 con 0x7fcb2c074870 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb2be00640 1 -- 172.21.15.123:0/1645816682 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1421889868 0 0) 0x7fcb2c073ce0 con 0x7fcb2c0c4d50 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb32400640 1 -- 172.21.15.123:0/1645816682 >> v1:172.21.15.153:6789/0 conn(0x7fcb2c0c4d50 legacy=0x7fcb2c0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:29.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb32400640 1 -- 172.21.15.123:0/1645816682 >> v1:172.21.15.123:6789/0 conn(0x7fcb2c074870 legacy=0x7fcb2c0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:29.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.891+0000 7fcb32400640 1 -- 172.21.15.123:0/1645816682 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c073fc0 con 0x7fcb2c154890 2024-04-25T12:34:29.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.893+0000 7fcb32400640 1 --2- 172.21.15.123:0/1645816682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c154890 0x7fcb2c15afa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009dc0 tx=0x7fcb2402a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3090bb9e1de23e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:29.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.893+0000 7fcb2be00640 1 -- 172.21.15.123:0/1645816682 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24007740 con 0x7fcb2c154890 2024-04-25T12:34:29.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.893+0000 7fcb2be00640 1 -- 172.21.15.123:0/1645816682 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb240078e0 con 0x7fcb2c154890 2024-04-25T12:34:29.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.893+0000 7fcb2be00640 1 -- 172.21.15.123:0/1645816682 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24007cb0 con 0x7fcb2c154890 2024-04-25T12:34:29.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.894+0000 7fcb34400640 1 -- 172.21.15.123:0/1645816682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c154890 msgr2=0x7fcb2c15afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:29.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.894+0000 7fcb34400640 1 --2- 172.21.15.123:0/1645816682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c154890 0x7fcb2c15afa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009dc0 tx=0x7fcb2402a760 comp rx=0 tx=0).stop 2024-04-25T12:34:29.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.894+0000 7fcb34400640 1 -- 172.21.15.123:0/1645816682 shutdown_connections 2024-04-25T12:34:29.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.894+0000 7fcb34400640 1 --2- 172.21.15.123:0/1645816682 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c154890 0x7fcb2c15afa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:29.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.894+0000 7fcb34400640 1 -- 172.21.15.123:0/1645816682 >> 172.21.15.123:0/1645816682 conn(0x7fcb2c06c750 msgr2=0x7fcb2c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:29.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.894+0000 7fcb34400640 1 -- 172.21.15.123:0/1645816682 shutdown_connections 2024-04-25T12:34:29.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.894+0000 7fcb34400640 1 -- 172.21.15.123:0/1645816682 wait complete. 2024-04-25T12:34:29.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.896+0000 7fcb34400640 1 Processor -- start 2024-04-25T12:34:29.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.896+0000 7fcb34400640 1 -- start start 2024-04-25T12:34:29.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb34400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb2c074870 0x7fcb2c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:29.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb34400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c0c4d50 0x7fcb2c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:29.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb34400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c154890 0x7fcb2c149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:29.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb34400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c0c8460 con 0x7fcb2c154890 2024-04-25T12:34:29.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb34400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c0c87d0 con 0x7fcb2c074870 2024-04-25T12:34:29.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb34400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2c0c9370 con 0x7fcb2c0c4d50 2024-04-25T12:34:29.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb32400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c154890 0x7fcb2c149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb32400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c154890 0x7fcb2c149340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36688/0 (socket says 172.21.15.123:36688) 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb32400640 1 -- 172.21.15.123:0/3791081528 learned_addr learned my addr 172.21.15.123:0/3791081528 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb31000640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c0c4d50 0x7fcb2c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.898+0000 7fcb31a00640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb2c074870 0x7fcb2c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb32400640 1 -- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c0c4d50 msgr2=0x7fcb2c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb32400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c0c4d50 0x7fcb2c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb32400640 1 -- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb2c074870 msgr2=0x7fcb2c159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb32400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb2c074870 0x7fcb2c159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb32400640 1 -- 172.21.15.123:0/3791081528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c149880 con 0x7fcb2c154890 2024-04-25T12:34:29.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb32400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c154890 0x7fcb2c149340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009ee0 tx=0x7fcb24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:29.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb29600640 1 -- 172.21.15.123:0/3791081528 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2403f070 con 0x7fcb2c154890 2024-04-25T12:34:29.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb29600640 1 -- 172.21.15.123:0/3791081528 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24031bc0 con 0x7fcb2c154890 2024-04-25T12:34:29.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c148b80 con 0x7fcb2c154890 2024-04-25T12:34:29.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.899+0000 7fcb29600640 1 -- 172.21.15.123:0/3791081528 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24009450 con 0x7fcb2c154890 2024-04-25T12:34:29.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.900+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c148e20 con 0x7fcb2c154890 2024-04-25T12:34:29.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.901+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c076700 con 0x7fcb2c154890 2024-04-25T12:34:29.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.902+0000 7fcb29600640 1 -- 172.21.15.123:0/3791081528 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb24044460 con 0x7fcb2c154890 2024-04-25T12:34:29.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.903+0000 7fcb29600640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb0c0744e0 0x7fcb0c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:29.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.903+0000 7fcb29600640 1 -- 172.21.15.123:0/3791081528 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcb240b6a90 con 0x7fcb2c154890 2024-04-25T12:34:29.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.903+0000 7fcb31a00640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb0c0744e0 0x7fcb0c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:29.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.904+0000 7fcb31a00640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb0c0744e0 0x7fcb0c0769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb20007bd0 tx=0x7fcb20007b20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:29.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:29.908+0000 7fcb29600640 1 -- 172.21.15.123:0/3791081528 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb24080910 con 0x7fcb2c154890 2024-04-25T12:34:30.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.117+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb2c0b3610 con 0x7fcb2c154890 2024-04-25T12:34:30.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.119+0000 7fcb29600640 1 -- 172.21.15.123:0/3791081528 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fcb24084c10 con 0x7fcb2c154890 2024-04-25T12:34:30.119 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:30.120 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:30.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb0c0744e0 msgr2=0x7fcb0c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb0c0744e0 0x7fcb0c0769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb20007bd0 tx=0x7fcb20007b20 comp rx=0 tx=0).stop 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c154890 msgr2=0x7fcb2c149340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c154890 0x7fcb2c149340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009ee0 tx=0x7fcb24004a40 comp rx=0 tx=0).stop 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 shutdown_connections 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb0c0744e0 0x7fcb0c0769f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb2c154890 0x7fcb2c149340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb2c0c4d50 0x7fcb2c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 --2- 172.21.15.123:0/3791081528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb2c074870 0x7fcb2c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 >> 172.21.15.123:0/3791081528 conn(0x7fcb2c06c750 msgr2=0x7fcb2c14e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 shutdown_connections 2024-04-25T12:34:30.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.122+0000 7fcb34400640 1 -- 172.21.15.123:0/3791081528 wait complete. 2024-04-25T12:34:30.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T12:34:30.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:34:30.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.410+0000 7fc5e9600640 1 Processor -- start 2024-04-25T12:34:30.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.411+0000 7fc5e9600640 1 -- start start 2024-04-25T12:34:30.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e9600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4154720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e9600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc5e40b43b0 0x7fc5e40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e9600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5e40c4d20 con 0x7fc5e415b120 2024-04-25T12:34:30.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e9600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e40c4510 con 0x7fc5e40b43b0 2024-04-25T12:34:30.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e9600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e415ac90 con 0x7fc5e40c5950 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e8c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc5e40b43b0 0x7fc5e40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:54002/0 (socket says 172.21.15.123:54002) 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e8c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc5e40b43b0 0x7fc5e40ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34848/0 (socket says 172.21.15.123:34848) 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5dbe00640 1 -- 172.21.15.123:0/2124826264 learned_addr learned my addr 172.21.15.123:0/2124826264 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.413+0000 7fc5e3e00640 1 --1- 172.21.15.123:0/2124826264 >> v1:172.21.15.153:6789/0 conn(0x7fc5e415b120 0x7fc5e40705e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:38778/0 (socket says 172.21.15.123:38778) 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.414+0000 7fc5dbe00640 1 -- 172.21.15.123:0/2124826264 >> v2:172.21.15.153:3300/0 conn(0x7fc5e40b43b0 msgr2=0x7fc5e40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.414+0000 7fc5dbe00640 1 --2- 172.21.15.123:0/2124826264 >> v2:172.21.15.153:3300/0 conn(0x7fc5e40b43b0 0x7fc5e40ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.414+0000 7fc5dbe00640 1 -- 172.21.15.123:0/2124826264 >> v1:172.21.15.153:6789/0 conn(0x7fc5e415b120 legacy=0x7fc5e40705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.414+0000 7fc5dbe00640 1 -- 172.21.15.123:0/2124826264 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e40c5d60 con 0x7fc5e40c5950 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.414+0000 7fc5e3400640 1 -- 172.21.15.123:0/2124826264 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529936236 0 0) 0x7fc5e40c4d20 con 0x7fc5e415b120 2024-04-25T12:34:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.414+0000 7fc5e8c00640 1 --2- 172.21.15.123:0/2124826264 >> v2:172.21.15.153:3300/0 conn(0x7fc5e40b43b0 0x7fc5e40ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:30.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.415+0000 7fc5dbe00640 1 --2- 172.21.15.123:0/2124826264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4154720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009870 tx=0x7fc5cc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5850610aa8e7995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:30.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.415+0000 7fc5e3400640 1 -- 172.21.15.123:0/2124826264 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc037070 con 0x7fc5e40c5950 2024-04-25T12:34:30.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e3400640 1 -- 172.21.15.123:0/2124826264 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc032440 con 0x7fc5e40c5950 2024-04-25T12:34:30.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e3400640 1 -- 172.21.15.123:0/2124826264 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc032810 con 0x7fc5e40c5950 2024-04-25T12:34:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e9600640 1 -- 172.21.15.123:0/2124826264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 msgr2=0x7fc5e4154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e9600640 1 --2- 172.21.15.123:0/2124826264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4154720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009870 tx=0x7fc5cc02a1c0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e9600640 1 -- 172.21.15.123:0/2124826264 shutdown_connections 2024-04-25T12:34:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e9600640 1 --2- 172.21.15.123:0/2124826264 >> v2:172.21.15.153:3300/0 conn(0x7fc5e40b43b0 0x7fc5e40ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e9600640 1 --2- 172.21.15.123:0/2124826264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4154720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e9600640 1 -- 172.21.15.123:0/2124826264 >> 172.21.15.123:0/2124826264 conn(0x7fc5e4072bc0 msgr2=0x7fc5e4072fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.416+0000 7fc5e9600640 1 -- 172.21.15.123:0/2124826264 shutdown_connections 2024-04-25T12:34:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.417+0000 7fc5e9600640 1 -- 172.21.15.123:0/2124826264 wait complete. 2024-04-25T12:34:30.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.419+0000 7fc5e9600640 1 Processor -- start 2024-04-25T12:34:30.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.419+0000 7fc5e9600640 1 -- start start 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.421+0000 7fc5e9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e40b43b0 0x7fc5e406de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.421+0000 7fc5e9600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.421+0000 7fc5e9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5e415b120 0x7fc5e41510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.421+0000 7fc5e9600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e40c4a40 con 0x7fc5e40b43b0 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.421+0000 7fc5e9600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4154c60 con 0x7fc5e415b120 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.421+0000 7fc5e9600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e415a640 con 0x7fc5e40c5950 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.421+0000 7fc5e3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e40b43b0 0x7fc5e406de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e40b43b0 0x7fc5e406de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36704/0 (socket says 172.21.15.123:36704) 2024-04-25T12:34:30.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e3e00640 1 -- 172.21.15.123:0/611081080 learned_addr learned my addr 172.21.15.123:0/611081080 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:30.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e8c00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5e415b120 0x7fc5e41510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5dbe00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e8c00640 1 -- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 msgr2=0x7fc5e4159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e8c00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e8c00640 1 -- 172.21.15.123:0/611081080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e40b43b0 msgr2=0x7fc5e406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e8c00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e40b43b0 0x7fc5e406de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e8c00640 1 -- 172.21.15.123:0/611081080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e406e370 con 0x7fc5e415b120 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e3e00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e40b43b0 0x7fc5e406de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.422+0000 7fc5e8c00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5e415b120 0x7fc5e41510f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4072740 tx=0x7fc5dc00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.423+0000 7fc5e0c00640 1 -- 172.21.15.123:0/611081080 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc018020 con 0x7fc5e415b120 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.423+0000 7fc5e0c00640 1 -- 172.21.15.123:0/611081080 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5dc014480 con 0x7fc5e415b120 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.423+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e41f5fd0 con 0x7fc5e415b120 2024-04-25T12:34:30.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.423+0000 7fc5e0c00640 1 -- 172.21.15.123:0/611081080 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc014870 con 0x7fc5e415b120 2024-04-25T12:34:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.423+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41f6590 con 0x7fc5e415b120 2024-04-25T12:34:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.424+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e4076da0 con 0x7fc5e415b120 2024-04-25T12:34:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.425+0000 7fc5e0c00640 1 -- 172.21.15.123:0/611081080 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc5dc01b020 con 0x7fc5e415b120 2024-04-25T12:34:30.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.426+0000 7fc5e0c00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc5c00744b0 0x7fc5c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.427+0000 7fc5e0c00640 1 -- 172.21.15.123:0/611081080 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc5dc0961b0 con 0x7fc5e415b120 2024-04-25T12:34:30.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.427+0000 7fc5e3e00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc5c00744b0 0x7fc5c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.427+0000 7fc5e3e00640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc5c00744b0 0x7fc5c00769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4000f60 tx=0x7fc5d4007ea0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:30.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.432+0000 7fc5e0c00640 1 -- 172.21.15.123:0/611081080 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5dc022020 con 0x7fc5e415b120 2024-04-25T12:34:30.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.644+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5e40cc550 con 0x7fc5e415b120 2024-04-25T12:34:30.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.645+0000 7fc5e0c00640 1 -- 172.21.15.123:0/611081080 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fc5dc010040 con 0x7fc5e415b120 2024-04-25T12:34:30.645 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T12:34:30.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc5c00744b0 msgr2=0x7fc5c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc5c00744b0 0x7fc5c00769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4000f60 tx=0x7fc5d4007ea0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5e415b120 msgr2=0x7fc5e41510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5e415b120 0x7fc5e41510f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4072740 tx=0x7fc5dc00ef90 comp rx=0 tx=0).stop 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 shutdown_connections 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc5c00744b0 0x7fc5c00769c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5e415b120 0x7fc5e41510f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5e40c5950 0x7fc5e4159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 --2- 172.21.15.123:0/611081080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e40b43b0 0x7fc5e406de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 >> 172.21.15.123:0/611081080 conn(0x7fc5e4072bc0 msgr2=0x7fc5e40705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 shutdown_connections 2024-04-25T12:34:30.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.648+0000 7fc5e9600640 1 -- 172.21.15.123:0/611081080 wait complete. 2024-04-25T12:34:30.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:34:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.948+0000 7f332be00640 1 Processor -- start 2024-04-25T12:34:30.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.949+0000 7f332be00640 1 -- start start 2024-04-25T12:34:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.950+0000 7f332be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.950+0000 7f332be00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332c152cc0 con 0x7f332c0b34f0 2024-04-25T12:34:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.950+0000 7f332be00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332c1589d0 con 0x7f332c1528b0 2024-04-25T12:34:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.950+0000 7f332be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f332c0c4060 con 0x7f332c0730a0 2024-04-25T12:34:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332aa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f332c0b34f0 0x7f332c0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:52736/0 (socket says 172.21.15.123:52736) 2024-04-25T12:34:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332aa00640 1 -- 172.21.15.123:0/802502201 learned_addr learned my addr 172.21.15.123:0/802502201 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:30.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332b400640 1 --2- 172.21.15.123:0/802502201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332a000640 1 -- 172.21.15.123:0/802502201 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771931565 0 0) 0x7f331c001110 con 0x7f332c0b34f0 2024-04-25T12:34:30.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332a000640 1 -- 172.21.15.123:0/802502201 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3310003440 con 0x7f332c0b34f0 2024-04-25T12:34:30.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332a000640 1 -- 172.21.15.123:0/802502201 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 986483729 0 0) 0x7f332c1589d0 con 0x7f332c1528b0 2024-04-25T12:34:30.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332b400640 1 -- 172.21.15.123:0/802502201 >> v1:172.21.15.153:6789/0 conn(0x7f332c1528b0 legacy=0x7f332c0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332b400640 1 -- 172.21.15.123:0/802502201 >> v1:172.21.15.123:6789/0 conn(0x7f332c0b34f0 legacy=0x7f332c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.951+0000 7f332b400640 1 -- 172.21.15.123:0/802502201 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c158c70 con 0x7f332c0730a0 2024-04-25T12:34:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.952+0000 7f332b400640 1 --2- 172.21.15.123:0/802502201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0734b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f332c158ba0 tx=0x7f332402a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705e5e2cc579fb84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.953+0000 7f332a000640 1 -- 172.21.15.123:0/802502201 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324007740 con 0x7f332c0730a0 2024-04-25T12:34:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.953+0000 7f332a000640 1 -- 172.21.15.123:0/802502201 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33240078e0 con 0x7f332c0730a0 2024-04-25T12:34:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.953+0000 7f332a000640 1 -- 172.21.15.123:0/802502201 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324007c90 con 0x7f332c0730a0 2024-04-25T12:34:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.953+0000 7f332be00640 1 -- 172.21.15.123:0/802502201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 msgr2=0x7f332c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.953+0000 7f332be00640 1 --2- 172.21.15.123:0/802502201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0734b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f332c158ba0 tx=0x7f332402a750 comp rx=0 tx=0).stop 2024-04-25T12:34:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.954+0000 7f332be00640 1 -- 172.21.15.123:0/802502201 shutdown_connections 2024-04-25T12:34:30.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.954+0000 7f332be00640 1 --2- 172.21.15.123:0/802502201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0734b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.954+0000 7f332be00640 1 -- 172.21.15.123:0/802502201 >> 172.21.15.123:0/802502201 conn(0x7f332c070b70 msgr2=0x7f332c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.954+0000 7f332be00640 1 -- 172.21.15.123:0/802502201 shutdown_connections 2024-04-25T12:34:30.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.954+0000 7f332be00640 1 -- 172.21.15.123:0/802502201 wait complete. 2024-04-25T12:34:30.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.956+0000 7f332be00640 1 Processor -- start 2024-04-25T12:34:30.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.956+0000 7f332be00640 1 -- start start 2024-04-25T12:34:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.957+0000 7f332be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.957+0000 7f332be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f332c0b34f0 0x7f332c158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.957+0000 7f332be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f332c1528b0 0x7f332c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.957+0000 7f332be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f332c159230 con 0x7f332c1528b0 2024-04-25T12:34:30.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.957+0000 7f332be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f332c0c88a0 con 0x7f332c0b34f0 2024-04-25T12:34:30.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.957+0000 7f332be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f332c159510 con 0x7f332c0730a0 2024-04-25T12:34:30.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f332c1528b0 0x7f332c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f332c1528b0 0x7f332c156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36712/0 (socket says 172.21.15.123:36712) 2024-04-25T12:34:30.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 -- 172.21.15.123:0/1242149367 learned_addr learned my addr 172.21.15.123:0/1242149367 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:30.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332aa00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f3322000640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f332c0b34f0 0x7f332c158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 -- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 msgr2=0x7f332c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 -- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f332c0b34f0 msgr2=0x7f332c158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f332c0b34f0 0x7f332c158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332b400640 1 -- 172.21.15.123:0/1242149367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c156ea0 con 0x7f332c1528b0 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.958+0000 7f332aa00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.959+0000 7f332b400640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f332c1528b0 0x7f332c156960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3324009eb0 tx=0x7f3324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.959+0000 7f3323e00640 1 -- 172.21.15.123:0/1242149367 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324009450 con 0x7f332c1528b0 2024-04-25T12:34:30.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.959+0000 7f3323e00640 1 -- 172.21.15.123:0/1242149367 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3324043070 con 0x7f332c1528b0 2024-04-25T12:34:30.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.959+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c1f2860 con 0x7f332c1528b0 2024-04-25T12:34:30.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.959+0000 7f3323e00640 1 -- 172.21.15.123:0/1242149367 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33240044a0 con 0x7f332c1528b0 2024-04-25T12:34:30.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.959+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c1f2e20 con 0x7f332c1528b0 2024-04-25T12:34:30.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.960+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332c151260 con 0x7f332c1528b0 2024-04-25T12:34:30.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.961+0000 7f3323e00640 1 -- 172.21.15.123:0/1242149367 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3324004640 con 0x7f332c1528b0 2024-04-25T12:34:30.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.962+0000 7f3323e00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f33080744b0 0x7f33080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:30.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.962+0000 7f3323e00640 1 -- 172.21.15.123:0/1242149367 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f33240b64f0 con 0x7f332c1528b0 2024-04-25T12:34:30.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.963+0000 7f332aa00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f33080744b0 0x7f33080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:30.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.963+0000 7f332aa00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f33080744b0 0x7f33080769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f331c005880 tx=0x7f331c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:30.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:30.967+0000 7f3323e00640 1 -- 172.21.15.123:0/1242149367 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f332c151260 con 0x7f332c1528b0 2024-04-25T12:34:31.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.173+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f332c076da0 con 0x7f332c1528b0 2024-04-25T12:34:31.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.174+0000 7f3323e00640 1 -- 172.21.15.123:0/1242149367 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f33240802f0 con 0x7f332c1528b0 2024-04-25T12:34:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:34:31.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f33080744b0 msgr2=0x7f33080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f33080744b0 0x7f33080769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f331c005880 tx=0x7f331c009040 comp rx=0 tx=0).stop 2024-04-25T12:34:31.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f332c1528b0 msgr2=0x7f332c156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f332c1528b0 0x7f332c156960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3324009eb0 tx=0x7f3324004a40 comp rx=0 tx=0).stop 2024-04-25T12:34:31.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 shutdown_connections 2024-04-25T12:34:31.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f33080744b0 0x7f33080769c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f332c1528b0 0x7f332c156960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f332c0b34f0 0x7f332c158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 --2- 172.21.15.123:0/1242149367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f332c0730a0 0x7f332c0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.177+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 >> 172.21.15.123:0/1242149367 conn(0x7f332c070b70 msgr2=0x7f332c1509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:31.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.178+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 shutdown_connections 2024-04-25T12:34:31.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.178+0000 7f332be00640 1 -- 172.21.15.123:0/1242149367 wait complete. 2024-04-25T12:34:31.201 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:34:31.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.473+0000 7f71a8e00640 1 Processor -- start 2024-04-25T12:34:31.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.473+0000 7f71a8e00640 1 -- start start 2024-04-25T12:34:31.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f71a8e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a4071bd0 0x7f71a4073660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:31.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f71a8e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f71a40c8500 0x7f71a41564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:31.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f71a8e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a40c6ca0 con 0x7f71a40745f0 2024-04-25T12:34:31.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f71a8e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f71a415fcc0 con 0x7f71a40c8500 2024-04-25T12:34:31.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f71a8e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f71a415ffa0 con 0x7f71a4071bd0 2024-04-25T12:34:31.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f71a1e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a4071bd0 0x7f71a4073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:31.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f719be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f71a40745f0 0x7f71a4160430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:52738/0 (socket says 172.21.15.123:52738) 2024-04-25T12:34:31.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f719be00640 1 -- 172.21.15.123:0/1470115853 learned_addr learned my addr 172.21.15.123:0/1470115853 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:31.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.476+0000 7f71a2800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f71a40c8500 0x7f71a41564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:31.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.477+0000 7f71a1e00640 1 -- 172.21.15.123:0/1470115853 >> v2:172.21.15.153:3300/0 conn(0x7f71a40c8500 msgr2=0x7f71a41564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:31.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.477+0000 7f71a1e00640 1 --2- 172.21.15.123:0/1470115853 >> v2:172.21.15.153:3300/0 conn(0x7f71a40c8500 0x7f71a41564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.477+0000 7f71a1e00640 1 -- 172.21.15.123:0/1470115853 >> v1:172.21.15.123:6789/0 conn(0x7f71a40745f0 legacy=0x7f71a4160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.477+0000 7f71a1e00640 1 -- 172.21.15.123:0/1470115853 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a415d590 con 0x7f71a4071bd0 2024-04-25T12:34:31.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.477+0000 7f71a1400640 1 -- 172.21.15.123:0/1470115853 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962716562 0 0) 0x7f71a40c6ca0 con 0x7f71a40745f0 2024-04-25T12:34:31.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.477+0000 7f71a2800640 1 --2- 172.21.15.123:0/1470115853 >> v2:172.21.15.153:3300/0 conn(0x7f71a40c8500 0x7f71a41564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:31.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.478+0000 7f71a1e00640 1 --2- 172.21.15.123:0/1470115853 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a4071bd0 0x7f71a4073660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7194009790 tx=0x7f719402a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fa7acaf1401826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:31.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.478+0000 7f71a1400640 1 -- 172.21.15.123:0/1470115853 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194037070 con 0x7f71a4071bd0 2024-04-25T12:34:31.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.478+0000 7f71a1400640 1 -- 172.21.15.123:0/1470115853 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7194032440 con 0x7f71a4071bd0 2024-04-25T12:34:31.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.478+0000 7f71a1400640 1 -- 172.21.15.123:0/1470115853 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194032810 con 0x7f71a4071bd0 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 -- 172.21.15.123:0/1470115853 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a4071bd0 msgr2=0x7f71a4073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1470115853 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a4071bd0 0x7f71a4073660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7194009790 tx=0x7f719402a200 comp rx=0 tx=0).stop 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 -- 172.21.15.123:0/1470115853 shutdown_connections 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1470115853 >> v2:172.21.15.153:3300/0 conn(0x7f71a40c8500 0x7f71a41564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1470115853 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a4071bd0 0x7f71a4073660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 -- 172.21.15.123:0/1470115853 >> 172.21.15.123:0/1470115853 conn(0x7f71a40ca570 msgr2=0x7f71a40ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 -- 172.21.15.123:0/1470115853 shutdown_connections 2024-04-25T12:34:31.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.479+0000 7f71a8e00640 1 -- 172.21.15.123:0/1470115853 wait complete. 2024-04-25T12:34:31.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.481+0000 7f71a8e00640 1 Processor -- start 2024-04-25T12:34:31.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.482+0000 7f71a8e00640 1 -- start start 2024-04-25T12:34:31.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71a4071bd0 0x7f71a41f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:31.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a8e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a40745f0 0x7f71a415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:31.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71a40c8500 0x7f71a4146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:31.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71a40c5530 con 0x7f71a4071bd0 2024-04-25T12:34:31.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a8e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71a40c5250 con 0x7f71a40c8500 2024-04-25T12:34:31.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a8e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f71a4160b40 con 0x7f71a40745f0 2024-04-25T12:34:31.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f719be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a40745f0 0x7f71a415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:31.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71a4071bd0 0x7f71a41f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:31.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f719be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a40745f0 0x7f71a415ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:54038/0 (socket says 172.21.15.123:54038) 2024-04-25T12:34:31.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f719be00640 1 -- 172.21.15.123:0/1032625793 learned_addr learned my addr 172.21.15.123:0/1032625793 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:31.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.483+0000 7f71a2800640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71a40c8500 0x7f71a4146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:31.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719be00640 1 -- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71a40c8500 msgr2=0x7f71a4146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719be00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71a40c8500 0x7f71a4146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719be00640 1 -- 172.21.15.123:0/1032625793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71a4071bd0 msgr2=0x7f71a41f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719be00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71a4071bd0 0x7f71a41f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719be00640 1 -- 172.21.15.123:0/1032625793 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a41f5b60 con 0x7f71a40745f0 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719be00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a40745f0 0x7f71a415ecd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c440 tx=0x7f718c00c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719a000640 1 -- 172.21.15.123:0/1032625793 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0074e0 con 0x7f71a40745f0 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719a000640 1 -- 172.21.15.123:0/1032625793 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718c0047e0 con 0x7f71a40745f0 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.484+0000 7f719a000640 1 -- 172.21.15.123:0/1032625793 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c004bd0 con 0x7f71a40745f0 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.485+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a414c410 con 0x7f71a40745f0 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.485+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a4159f60 con 0x7f71a40745f0 2024-04-25T12:34:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.486+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a414c170 con 0x7f71a40745f0 2024-04-25T12:34:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.487+0000 7f719a000640 1 -- 172.21.15.123:0/1032625793 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f718c011070 con 0x7f71a40745f0 2024-04-25T12:34:31.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.488+0000 7f719a000640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71800744b0 0x7f71800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:31.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.488+0000 7f719a000640 1 -- 172.21.15.123:0/1032625793 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f718c0950e0 con 0x7f71a40745f0 2024-04-25T12:34:31.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.488+0000 7f71a1e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71800744b0 0x7f71800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:31.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.489+0000 7f71a1e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71800744b0 0x7f71800769c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7194034000 tx=0x7f719402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.494+0000 7f719a000640 1 -- 172.21.15.123:0/1032625793 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71a414c170 con 0x7f71a40745f0 2024-04-25T12:34:31.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.694+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71a4076700 con 0x7f71a40745f0 2024-04-25T12:34:31.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.695+0000 7f719a000640 1 -- 172.21.15.123:0/1032625793 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f718c05f520 con 0x7f71a40745f0 2024-04-25T12:34:31.694 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T12:34:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.697+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71800744b0 msgr2=0x7f71800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.697+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71800744b0 0x7f71800769c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7194034000 tx=0x7f719402e000 comp rx=0 tx=0).stop 2024-04-25T12:34:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.697+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a40745f0 msgr2=0x7f71a415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:31.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.697+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a40745f0 0x7f71a415ecd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c440 tx=0x7f718c00c840 comp rx=0 tx=0).stop 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 shutdown_connections 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71800744b0 0x7f71800769c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71a40c8500 0x7f71a4146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71a40745f0 0x7f71a415ecd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 --2- 172.21.15.123:0/1032625793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71a4071bd0 0x7f71a41f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 >> 172.21.15.123:0/1032625793 conn(0x7f71a40ca570 msgr2=0x7f71a4075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 shutdown_connections 2024-04-25T12:34:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.698+0000 7f71a8e00640 1 -- 172.21.15.123:0/1032625793 wait complete. 2024-04-25T12:34:31.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T12:34:31.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.999+0000 7fcb36600640 1 Processor -- start 2024-04-25T12:34:31.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:31.999+0000 7fcb36600640 1 -- start start 2024-04-25T12:34:32.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb36600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb30154650 0x7fcb3015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb36600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb300c9f00 0x7fcb300c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb36600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb300734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb36600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3015b380 con 0x7fcb300c9f00 2024-04-25T12:34:32.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb36600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcb30074b20 con 0x7fcb30154650 2024-04-25T12:34:32.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb36600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcb300ca780 con 0x7fcb300730e0 2024-04-25T12:34:32.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb2fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb300734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb2f400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb30154650 0x7fcb3015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb2ea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb300c9f00 0x7fcb300c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb2fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb300734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:54048/0 (socket says 172.21.15.123:54048) 2024-04-25T12:34:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.001+0000 7fcb2fe00640 1 -- 172.21.15.123:0/569291000 learned_addr learned my addr 172.21.15.123:0/569291000 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.002+0000 7fcb2f400640 1 -- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 msgr2=0x7fcb300734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.002+0000 7fcb2f400640 1 --2- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb300734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.002+0000 7fcb2f400640 1 -- 172.21.15.123:0/569291000 >> v2:172.21.15.123:3300/0 conn(0x7fcb300c9f00 msgr2=0x7fcb300c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:32.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.002+0000 7fcb2f400640 1 --2- 172.21.15.123:0/569291000 >> v2:172.21.15.123:3300/0 conn(0x7fcb300c9f00 0x7fcb300c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.002+0000 7fcb2f400640 1 -- 172.21.15.123:0/569291000 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb300caa60 con 0x7fcb30154650 2024-04-25T12:34:32.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.002+0000 7fcb2fe00640 1 --2- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb300734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.003+0000 7fcb2f400640 1 --2- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 0x7fcb3015ae40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb30071a20 tx=0x7fcb2002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d60c6b9e3abf5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.003+0000 7fcb2e000640 1 -- 172.21.15.123:0/569291000 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20007740 con 0x7fcb30154650 2024-04-25T12:34:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.003+0000 7fcb2e000640 1 -- 172.21.15.123:0/569291000 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb200078e0 con 0x7fcb30154650 2024-04-25T12:34:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.003+0000 7fcb2e000640 1 -- 172.21.15.123:0/569291000 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20007cb0 con 0x7fcb30154650 2024-04-25T12:34:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 -- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 msgr2=0x7fcb3015ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 --2- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 0x7fcb3015ae40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb30071a20 tx=0x7fcb2002a660 comp rx=0 tx=0).stop 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 -- 172.21.15.123:0/569291000 shutdown_connections 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 --2- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb300734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 --2- 172.21.15.123:0/569291000 >> v2:172.21.15.123:3300/0 conn(0x7fcb300c9f00 0x7fcb300c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 --2- 172.21.15.123:0/569291000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 0x7fcb3015ae40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 -- 172.21.15.123:0/569291000 >> 172.21.15.123:0/569291000 conn(0x7fcb3006f440 msgr2=0x7fcb3006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.004+0000 7fcb36600640 1 -- 172.21.15.123:0/569291000 shutdown_connections 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.005+0000 7fcb36600640 1 -- 172.21.15.123:0/569291000 wait complete. 2024-04-25T12:34:32.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.007+0000 7fcb36600640 1 Processor -- start 2024-04-25T12:34:32.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.008+0000 7fcb36600640 1 -- start start 2024-04-25T12:34:32.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.009+0000 7fcb36600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb30153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.009+0000 7fcb36600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb300c9f00 0x7fcb3014edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.009+0000 7fcb36600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 0x7fcb3014f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.009+0000 7fcb36600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3015e010 con 0x7fcb300c9f00 2024-04-25T12:34:32.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb36600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3015dd30 con 0x7fcb30154650 2024-04-25T12:34:32.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb36600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3015e2f0 con 0x7fcb300730e0 2024-04-25T12:34:32.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb300c9f00 0x7fcb3014edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2f400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb30153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb300c9f00 0x7fcb3014edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36734/0 (socket says 172.21.15.123:36734) 2024-04-25T12:34:32.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 -- 172.21.15.123:0/404790815 learned_addr learned my addr 172.21.15.123:0/404790815 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:32.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2fe00640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 0x7fcb3014f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 -- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 msgr2=0x7fcb30153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb30153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 -- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 msgr2=0x7fcb3014f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 0x7fcb3014f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.010+0000 7fcb2ea00640 1 -- 172.21.15.123:0/404790815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb30153f90 con 0x7fcb300c9f00 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.011+0000 7fcb2ea00640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb300c9f00 0x7fcb3014edd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e870 tx=0x7fcb1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.011+0000 7fcb27400640 1 -- 172.21.15.123:0/404790815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c017070 con 0x7fcb300c9f00 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.011+0000 7fcb27400640 1 -- 172.21.15.123:0/404790815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1c00ee60 con 0x7fcb300c9f00 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.011+0000 7fcb27400640 1 -- 172.21.15.123:0/404790815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00cb60 con 0x7fcb300c9f00 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.011+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb30153cb0 con 0x7fcb300c9f00 2024-04-25T12:34:32.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.011+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb301502b0 con 0x7fcb300c9f00 2024-04-25T12:34:32.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.012+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb30076700 con 0x7fcb300c9f00 2024-04-25T12:34:32.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.014+0000 7fcb27400640 1 -- 172.21.15.123:0/404790815 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb1c01a070 con 0x7fcb300c9f00 2024-04-25T12:34:32.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.015+0000 7fcb27400640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb100744e0 0x7fcb100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.015+0000 7fcb27400640 1 -- 172.21.15.123:0/404790815 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcb1c0956d0 con 0x7fcb300c9f00 2024-04-25T12:34:32.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.015+0000 7fcb2f400640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb100744e0 0x7fcb100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.016+0000 7fcb2f400640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb100744e0 0x7fcb100769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb20034000 tx=0x7fcb2002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:32.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.020+0000 7fcb27400640 1 -- 172.21.15.123:0/404790815 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb1c05f430 con 0x7fcb300c9f00 2024-04-25T12:34:32.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.221+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcb300744e0 con 0x7fcb300c9f00 2024-04-25T12:34:32.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.221+0000 7fcb27400640 1 -- 172.21.15.123:0/404790815 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fcb1c063730 con 0x7fcb300c9f00 2024-04-25T12:34:32.221 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:34:32.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb100744e0 msgr2=0x7fcb100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb100744e0 0x7fcb100769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb20034000 tx=0x7fcb2002f000 comp rx=0 tx=0).stop 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb300c9f00 msgr2=0x7fcb3014edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb300c9f00 0x7fcb3014edd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e870 tx=0x7fcb1c00ed30 comp rx=0 tx=0).stop 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 shutdown_connections 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcb100744e0 0x7fcb100769f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb30154650 0x7fcb3014f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb300c9f00 0x7fcb3014edd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 --2- 172.21.15.123:0/404790815 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb300730e0 0x7fcb30153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 >> 172.21.15.123:0/404790815 conn(0x7fcb3006f440 msgr2=0x7fcb300c5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:32.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 shutdown_connections 2024-04-25T12:34:32.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.224+0000 7fcb36600640 1 -- 172.21.15.123:0/404790815 wait complete. 2024-04-25T12:34:32.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T12:34:32.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.525+0000 7f6c58000640 1 Processor -- start 2024-04-25T12:34:32.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.525+0000 7f6c58000640 1 -- start start 2024-04-25T12:34:32.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.527+0000 7f6c58000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c500c83a0 0x7f6c500ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.527+0000 7f6c58000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5015fbd0 0x7f6c5015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.527+0000 7f6c58000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c500b65a0 con 0x7f6c5006d9d0 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.527+0000 7f6c58000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500c87b0 con 0x7f6c500c83a0 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.527+0000 7f6c58000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6c500b6880 con 0x7f6c5015fbd0 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.527+0000 7f6c55600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c500c83a0 0x7f6c500ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.527+0000 7f6c56000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5015fbd0 0x7f6c5015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.528+0000 7f6c55600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c500c83a0 0x7f6c500ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34908/0 (socket says 172.21.15.123:34908) 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.528+0000 7f6c55600640 1 -- 172.21.15.123:0/1030299017 learned_addr learned my addr 172.21.15.123:0/1030299017 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.528+0000 7f6c54c00640 1 --1- 172.21.15.123:0/1030299017 >> v1:172.21.15.123:6789/0 conn(0x7f6c5006d9d0 0x7f6c5006dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:52746/0 (socket says 172.21.15.123:52746) 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.528+0000 7f6c55600640 1 -- 172.21.15.123:0/1030299017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5015fbd0 msgr2=0x7f6c5015ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.528+0000 7f6c55600640 1 --2- 172.21.15.123:0/1030299017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5015fbd0 0x7f6c5015ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.528+0000 7f6c55600640 1 -- 172.21.15.123:0/1030299017 >> v1:172.21.15.123:6789/0 conn(0x7f6c5006d9d0 legacy=0x7f6c5006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.528+0000 7f6c55600640 1 -- 172.21.15.123:0/1030299017 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500b6b60 con 0x7f6c500c83a0 2024-04-25T12:34:32.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.529+0000 7f6c55600640 1 --2- 172.21.15.123:0/1030299017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500ccbe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009870 tx=0x7f6c4002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6748abe40b31372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:32.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.529+0000 7f6c4fe00640 1 -- 172.21.15.123:0/1030299017 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40037070 con 0x7f6c500c83a0 2024-04-25T12:34:32.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.529+0000 7f6c4fe00640 1 -- 172.21.15.123:0/1030299017 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4002ae50 con 0x7f6c500c83a0 2024-04-25T12:34:32.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.529+0000 7f6c4fe00640 1 -- 172.21.15.123:0/1030299017 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40032860 con 0x7f6c500c83a0 2024-04-25T12:34:32.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 -- 172.21.15.123:0/1030299017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 msgr2=0x7f6c500ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 --2- 172.21.15.123:0/1030299017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500ccbe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009870 tx=0x7f6c4002a300 comp rx=0 tx=0).stop 2024-04-25T12:34:32.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 -- 172.21.15.123:0/1030299017 shutdown_connections 2024-04-25T12:34:32.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 --2- 172.21.15.123:0/1030299017 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5015fbd0 0x7f6c5015ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 --2- 172.21.15.123:0/1030299017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500ccbe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 -- 172.21.15.123:0/1030299017 >> 172.21.15.123:0/1030299017 conn(0x7f6c50068f80 msgr2=0x7f6c500d1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:32.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 -- 172.21.15.123:0/1030299017 shutdown_connections 2024-04-25T12:34:32.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.530+0000 7f6c58000640 1 -- 172.21.15.123:0/1030299017 wait complete. 2024-04-25T12:34:32.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.533+0000 7f6c58000640 1 Processor -- start 2024-04-25T12:34:32.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.533+0000 7f6c58000640 1 -- start start 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.534+0000 7f6c58000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5006d9d0 0x7f6c50151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.534+0000 7f6c58000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.534+0000 7f6c58000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c5015fbd0 0x7f6c501f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.534+0000 7f6c58000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5006d4b0 con 0x7f6c5015fbd0 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.534+0000 7f6c58000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500b7120 con 0x7f6c500c83a0 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.534+0000 7f6c58000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6c500cd120 con 0x7f6c5006d9d0 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c56000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c5015fbd0 0x7f6c501f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c55600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5006d9d0 0x7f6c50151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c55600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5006d9d0 0x7f6c50151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:54070/0 (socket says 172.21.15.123:54070) 2024-04-25T12:34:32.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c55600640 1 -- 172.21.15.123:0/2403112603 learned_addr learned my addr 172.21.15.123:0/2403112603 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c54c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c56000640 1 -- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5006d9d0 msgr2=0x7f6c50151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c56000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5006d9d0 0x7f6c50151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c56000640 1 -- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 msgr2=0x7f6c500c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c56000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c56000640 1 -- 172.21.15.123:0/2403112603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c50153e60 con 0x7f6c5015fbd0 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c55600640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5006d9d0 0x7f6c50151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c54c00640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:32.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.535+0000 7f6c56000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c5015fbd0 0x7f6c501f5d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e7c0 tx=0x7f6c4800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:32.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.536+0000 7f6c4d600640 1 -- 172.21.15.123:0/2403112603 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48017070 con 0x7f6c5015fbd0 2024-04-25T12:34:32.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.536+0000 7f6c4d600640 1 -- 172.21.15.123:0/2403112603 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4800c6d0 con 0x7f6c5015fbd0 2024-04-25T12:34:32.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.536+0000 7f6c4d600640 1 -- 172.21.15.123:0/2403112603 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800cb00 con 0x7f6c5015fbd0 2024-04-25T12:34:32.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.536+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c50154b70 con 0x7f6c5015fbd0 2024-04-25T12:34:32.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.536+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c50154220 con 0x7f6c5015fbd0 2024-04-25T12:34:32.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.537+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c50076700 con 0x7f6c5015fbd0 2024-04-25T12:34:32.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.538+0000 7f6c4d600640 1 -- 172.21.15.123:0/2403112603 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6c4801a070 con 0x7f6c5015fbd0 2024-04-25T12:34:32.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.539+0000 7f6c4d600640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6c340744b0 0x7f6c340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:32.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.540+0000 7f6c4d600640 1 -- 172.21.15.123:0/2403112603 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6c48094f20 con 0x7f6c5015fbd0 2024-04-25T12:34:32.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.540+0000 7f6c55600640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6c340744b0 0x7f6c340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:32.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.541+0000 7f6c55600640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6c340744b0 0x7f6c340769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c400090b0 tx=0x7f6c4002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:32.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.544+0000 7f6c4d600640 1 -- 172.21.15.123:0/2403112603 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c4805ed10 con 0x7f6c5015fbd0 2024-04-25T12:34:32.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.751+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6c50159c40 con 0x7f6c5015fbd0 2024-04-25T12:34:32.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.752+0000 7f6c4d600640 1 -- 172.21.15.123:0/2403112603 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f6c48063010 con 0x7f6c5015fbd0 2024-04-25T12:34:32.751 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T12:34:32.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6c340744b0 msgr2=0x7f6c340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6c340744b0 0x7f6c340769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c400090b0 tx=0x7f6c4002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:32.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c5015fbd0 msgr2=0x7f6c501f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:32.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c5015fbd0 0x7f6c501f5d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e7c0 tx=0x7f6c4800ec80 comp rx=0 tx=0).stop 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 shutdown_connections 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6c340744b0 0x7f6c340769c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c5015fbd0 0x7f6c501f5d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c500c83a0 0x7f6c500c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 --2- 172.21.15.123:0/2403112603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6c5006d9d0 0x7f6c50151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.754+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 >> 172.21.15.123:0/2403112603 conn(0x7f6c50068f80 msgr2=0x7f6c50075f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.755+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 shutdown_connections 2024-04-25T12:34:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:32.755+0000 7f6c58000640 1 -- 172.21.15.123:0/2403112603 wait complete. 2024-04-25T12:34:32.777 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:34:32.777 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:34:32.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:33.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.141+0000 7fa6afe00640 1 Processor -- start 2024-04-25T12:34:33.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.141+0000 7fa6afe00640 1 -- start start 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.143+0000 7fa6afe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6b00c9080 0x7fa6b01546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.143+0000 7fa6afe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6b006c570 0x7fa6b006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.143+0000 7fa6afe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b015a330 con 0x7fa6b00c82b0 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.143+0000 7fa6afe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b0073b10 con 0x7fa6b006c570 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.143+0000 7fa6afe00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b0073df0 con 0x7fa6b00c9080 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.143+0000 7fa6af400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6b006c570 0x7fa6b006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6ae000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa6b00c82b0 0x7fa6b015e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40712/0 (socket says 172.21.15.123:40712) 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6ae000640 1 -- 172.21.15.123:0/1418847694 learned_addr learned my addr 172.21.15.123:0/1418847694 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6aea00640 1 --2- 172.21.15.123:0/1418847694 >> v2:172.21.15.153:3300/0 conn(0x7fa6b00c9080 0x7fa6b01546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6af400640 1 -- 172.21.15.123:0/1418847694 >> v2:172.21.15.153:3300/0 conn(0x7fa6b00c9080 msgr2=0x7fa6b01546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:33.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6af400640 1 --2- 172.21.15.123:0/1418847694 >> v2:172.21.15.153:3300/0 conn(0x7fa6b00c9080 0x7fa6b01546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6af400640 1 -- 172.21.15.123:0/1418847694 >> v1:172.21.15.123:6789/0 conn(0x7fa6b00c82b0 legacy=0x7fa6b015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6af400640 1 -- 172.21.15.123:0/1418847694 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b00740d0 con 0x7fa6b006c570 2024-04-25T12:34:33.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.144+0000 7fa6ad600640 1 -- 172.21.15.123:0/1418847694 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3064761357 0 0) 0x7fa6b015a330 con 0x7fa6b00c82b0 2024-04-25T12:34:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.145+0000 7fa6af400640 1 --2- 172.21.15.123:0/1418847694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b006c980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00ce520 tx=0x7fa6a802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e982807bc77c17db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6ad600640 1 -- 172.21.15.123:0/1418847694 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8007740 con 0x7fa6b006c570 2024-04-25T12:34:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6ad600640 1 -- 172.21.15.123:0/1418847694 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a80078e0 con 0x7fa6b006c570 2024-04-25T12:34:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6ad600640 1 -- 172.21.15.123:0/1418847694 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8007cb0 con 0x7fa6b006c570 2024-04-25T12:34:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6afe00640 1 -- 172.21.15.123:0/1418847694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 msgr2=0x7fa6b006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6afe00640 1 --2- 172.21.15.123:0/1418847694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b006c980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00ce520 tx=0x7fa6a802a5f0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6afe00640 1 -- 172.21.15.123:0/1418847694 shutdown_connections 2024-04-25T12:34:33.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6afe00640 1 --2- 172.21.15.123:0/1418847694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b006c980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6afe00640 1 --2- 172.21.15.123:0/1418847694 >> v2:172.21.15.153:3300/0 conn(0x7fa6b00c9080 0x7fa6b01546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.146+0000 7fa6afe00640 1 -- 172.21.15.123:0/1418847694 >> 172.21.15.123:0/1418847694 conn(0x7fa6b00cd600 msgr2=0x7fa6b00cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:33.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.147+0000 7fa6afe00640 1 -- 172.21.15.123:0/1418847694 shutdown_connections 2024-04-25T12:34:33.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.147+0000 7fa6afe00640 1 -- 172.21.15.123:0/1418847694 wait complete. 2024-04-25T12:34:33.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.149+0000 7fa6afe00640 1 Processor -- start 2024-04-25T12:34:33.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.149+0000 7fa6afe00640 1 -- start start 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.150+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6b00c82b0 0x7fa6b01f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6b00c9080 0x7fa6b00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6afe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b015e760 con 0x7fa6b006c570 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6afe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b015db30 con 0x7fa6b00c9080 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6afe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b0073760 con 0x7fa6b00c82b0 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51316/0 (socket says 172.21.15.123:51316) 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 -- 172.21.15.123:0/3780843579 learned_addr learned my addr 172.21.15.123:0/3780843579 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:33.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6af400640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6b00c9080 0x7fa6b00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6ae000640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6b00c82b0 0x7fa6b01f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 -- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6b00c82b0 msgr2=0x7fa6b01f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6b00c82b0 0x7fa6b01f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 -- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6b00c9080 msgr2=0x7fa6b00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6b00c9080 0x7fa6b00cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.151+0000 7fa6aea00640 1 -- 172.21.15.123:0/3780843579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b01f0590 con 0x7fa6b006c570 2024-04-25T12:34:33.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.152+0000 7fa6aea00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b015d100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000e8a0 tx=0x7fa6a000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:33.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.152+0000 7fa6a6a00640 1 -- 172.21.15.123:0/3780843579 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0017070 con 0x7fa6b006c570 2024-04-25T12:34:33.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.152+0000 7fa6a6a00640 1 -- 172.21.15.123:0/3780843579 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a000c940 con 0x7fa6b006c570 2024-04-25T12:34:33.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.152+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b01f3560 con 0x7fa6b006c570 2024-04-25T12:34:33.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.152+0000 7fa6a6a00640 1 -- 172.21.15.123:0/3780843579 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000cd30 con 0x7fa6b006c570 2024-04-25T12:34:33.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.152+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b01f3820 con 0x7fa6b006c570 2024-04-25T12:34:33.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.153+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b0076da0 con 0x7fa6b006c570 2024-04-25T12:34:33.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.155+0000 7fa6a6a00640 1 -- 172.21.15.123:0/3780843579 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa6a001a070 con 0x7fa6b006c570 2024-04-25T12:34:33.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.156+0000 7fa6a6a00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa6900744b0 0x7fa6900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.156+0000 7fa6a6a00640 1 -- 172.21.15.123:0/3780843579 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa6a0095760 con 0x7fa6b006c570 2024-04-25T12:34:33.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.156+0000 7fa6ae000640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa6900744b0 0x7fa6900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.157+0000 7fa6ae000640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa6900744b0 0x7fa6900769c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00cd510 tx=0x7fa69c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:33.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.160+0000 7fa6a6a00640 1 -- 172.21.15.123:0/3780843579 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6a005f550 con 0x7fa6b006c570 2024-04-25T12:34:33.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.360+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6b00c8e60 con 0x7fa6b006c570 2024-04-25T12:34:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.363+0000 7fa6a6a00640 1 -- 172.21.15.123:0/3780843579 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fa6a0063850 con 0x7fa6b006c570 2024-04-25T12:34:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:33.363 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:33.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa6900744b0 msgr2=0x7fa6900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa6900744b0 0x7fa6900769c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00cd510 tx=0x7fa69c009040 comp rx=0 tx=0).stop 2024-04-25T12:34:33.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 msgr2=0x7fa6b015d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b015d100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000e8a0 tx=0x7fa6a000ed60 comp rx=0 tx=0).stop 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 shutdown_connections 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa6900744b0 0x7fa6900769c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6b00c9080 0x7fa6b00cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6b00c82b0 0x7fa6b01f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 --2- 172.21.15.123:0/3780843579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b006c570 0x7fa6b015d100 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 >> 172.21.15.123:0/3780843579 conn(0x7fa6b00cd600 msgr2=0x7fa6b00c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 shutdown_connections 2024-04-25T12:34:33.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.366+0000 7fa6afe00640 1 -- 172.21.15.123:0/3780843579 wait complete. 2024-04-25T12:34:33.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T12:34:33.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:34:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.664+0000 7fdfb8800640 1 Processor -- start 2024-04-25T12:34:33.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.664+0000 7fdfb8800640 1 -- start start 2024-04-25T12:34:33.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.666+0000 7fdfb8800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb01426a0 0x7fdfb0159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.666+0000 7fdfb8800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb0074400 con 0x7fdfb00c8fd0 2024-04-25T12:34:33.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.666+0000 7fdfb8800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb01595e0 con 0x7fdfb00c8170 2024-04-25T12:34:33.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.666+0000 7fdfb8800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb0074670 con 0x7fdfb01426a0 2024-04-25T12:34:33.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.666+0000 7fdfb5e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdfb00c8fd0 0x7fdfb0153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40722/0 (socket says 172.21.15.123:40722) 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.666+0000 7fdfb5e00640 1 -- 172.21.15.123:0/223504579 learned_addr learned my addr 172.21.15.123:0/223504579 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.666+0000 7fdfb5400640 1 --2- 172.21.15.123:0/223504579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb01426a0 0x7fdfb0159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.667+0000 7fdfb4a00640 1 -- 172.21.15.123:0/223504579 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2217322297 0 0) 0x7fdfb01595e0 con 0x7fdfb00c8170 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.667+0000 7fdfb4a00640 1 -- 172.21.15.123:0/223504579 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf98003520 con 0x7fdfb00c8170 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.667+0000 7fdfb4a00640 1 -- 172.21.15.123:0/223504579 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309603930 0 0) 0x7fdfb0074400 con 0x7fdfb00c8fd0 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.667+0000 7fdfb4a00640 1 -- 172.21.15.123:0/223504579 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfb01595e0 con 0x7fdfb00c8fd0 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.667+0000 7fdfb5400640 1 -- 172.21.15.123:0/223504579 >> v1:172.21.15.153:6789/0 conn(0x7fdfb00c8170 legacy=0x7fdfb00ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.667+0000 7fdfb5400640 1 -- 172.21.15.123:0/223504579 >> v1:172.21.15.123:6789/0 conn(0x7fdfb00c8fd0 legacy=0x7fdfb0153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.667+0000 7fdfb5400640 1 -- 172.21.15.123:0/223504579 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb0074950 con 0x7fdfb01426a0 2024-04-25T12:34:33.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.668+0000 7fdfb5400640 1 --2- 172.21.15.123:0/223504579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb01426a0 0x7fdfb0159dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00097d0 tx=0x7fdfa002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854bd30454e8a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:33.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.668+0000 7fdfb4a00640 1 -- 172.21.15.123:0/223504579 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0037070 con 0x7fdfb01426a0 2024-04-25T12:34:33.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.668+0000 7fdfb4a00640 1 -- 172.21.15.123:0/223504579 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa0032440 con 0x7fdfb01426a0 2024-04-25T12:34:33.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.668+0000 7fdfb4a00640 1 -- 172.21.15.123:0/223504579 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0032810 con 0x7fdfb01426a0 2024-04-25T12:34:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.669+0000 7fdfb8800640 1 -- 172.21.15.123:0/223504579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb01426a0 msgr2=0x7fdfb0159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.669+0000 7fdfb8800640 1 --2- 172.21.15.123:0/223504579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb01426a0 0x7fdfb0159dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00097d0 tx=0x7fdfa002a210 comp rx=0 tx=0).stop 2024-04-25T12:34:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.669+0000 7fdfb8800640 1 -- 172.21.15.123:0/223504579 shutdown_connections 2024-04-25T12:34:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.669+0000 7fdfb8800640 1 --2- 172.21.15.123:0/223504579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb01426a0 0x7fdfb0159dc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.669+0000 7fdfb8800640 1 -- 172.21.15.123:0/223504579 >> 172.21.15.123:0/223504579 conn(0x7fdfb006f440 msgr2=0x7fdfb006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.669+0000 7fdfb8800640 1 -- 172.21.15.123:0/223504579 shutdown_connections 2024-04-25T12:34:33.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.669+0000 7fdfb8800640 1 -- 172.21.15.123:0/223504579 wait complete. 2024-04-25T12:34:33.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.672+0000 7fdfb8800640 1 Processor -- start 2024-04-25T12:34:33.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.672+0000 7fdfb8800640 1 -- start start 2024-04-25T12:34:33.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb8800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb00c8170 0x7fdfb01e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb8800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb00c8fd0 0x7fdfb01e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb8800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb01426a0 0x7fdfb00708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb8800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00c4880 con 0x7fdfb00c8170 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb8800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb015a300 con 0x7fdfb01426a0 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb8800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00ca910 con 0x7fdfb00c8fd0 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb00c8170 0x7fdfb01e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb00c8170 0x7fdfb01e1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51322/0 (socket says 172.21.15.123:51322) 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb5e00640 1 -- 172.21.15.123:0/200318192 learned_addr learned my addr 172.21.15.123:0/200318192 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.673+0000 7fdfb5400640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb00c8fd0 0x7fdfb01e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb6800640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb01426a0 0x7fdfb00708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb5e00640 1 -- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb00c8fd0 msgr2=0x7fdfb01e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb5e00640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb00c8fd0 0x7fdfb01e6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb5e00640 1 -- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb01426a0 msgr2=0x7fdfb00708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb5e00640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb01426a0 0x7fdfb00708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb5e00640 1 -- 172.21.15.123:0/200318192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb01546e0 con 0x7fdfb00c8170 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb5400640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb00c8fd0 0x7fdfb01e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb5e00640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb00c8170 0x7fdfb01e1ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c430 tx=0x7fdfa400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfaa000640 1 -- 172.21.15.123:0/200318192 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4017070 con 0x7fdfb00c8170 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfaa000640 1 -- 172.21.15.123:0/200318192 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa4005ce0 con 0x7fdfb00c8170 2024-04-25T12:34:33.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfaa000640 1 -- 172.21.15.123:0/200318192 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40072f0 con 0x7fdfb00c8170 2024-04-25T12:34:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.674+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb01e6750 con 0x7fdfb00c8170 2024-04-25T12:34:33.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.675+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01e6960 con 0x7fdfb00c8170 2024-04-25T12:34:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.676+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb0076700 con 0x7fdfb00c8170 2024-04-25T12:34:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.677+0000 7fdfaa000640 1 -- 172.21.15.123:0/200318192 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdfa401a070 con 0x7fdfb00c8170 2024-04-25T12:34:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.678+0000 7fdfaa000640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf940744e0 0x7fdf940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.678+0000 7fdfaa000640 1 -- 172.21.15.123:0/200318192 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdfa4094830 con 0x7fdfb00c8170 2024-04-25T12:34:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.679+0000 7fdfb5400640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf940744e0 0x7fdf940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:33.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.679+0000 7fdfb5400640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf940744e0 0x7fdf940769f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0034000 tx=0x7fdfa002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:33.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.683+0000 7fdfaa000640 1 -- 172.21.15.123:0/200318192 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfa405e6b0 con 0x7fdfb00c8170 2024-04-25T12:34:33.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.880+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfb00cb570 con 0x7fdfb00c8170 2024-04-25T12:34:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.881+0000 7fdfaa000640 1 -- 172.21.15.123:0/200318192 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fdfa40629b0 con 0x7fdfb00c8170 2024-04-25T12:34:33.881 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T12:34:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf940744e0 msgr2=0x7fdf940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf940744e0 0x7fdf940769f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0034000 tx=0x7fdfa002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb00c8170 msgr2=0x7fdfb01e1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb00c8170 0x7fdfb01e1ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c430 tx=0x7fdfa400c8f0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 shutdown_connections 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf940744e0 0x7fdf940769f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb01426a0 0x7fdfb00708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdfb00c8fd0 0x7fdfb01e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 --2- 172.21.15.123:0/200318192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb00c8170 0x7fdfb01e1ea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 >> 172.21.15.123:0/200318192 conn(0x7fdfb006f440 msgr2=0x7fdfb00ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 shutdown_connections 2024-04-25T12:34:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:33.884+0000 7fdfb8800640 1 -- 172.21.15.123:0/200318192 wait complete. 2024-04-25T12:34:33.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:34:34.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.166+0000 7fea60c00640 1 Processor -- start 2024-04-25T12:34:34.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.166+0000 7fea60c00640 1 -- start start 2024-04-25T12:34:34.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.168+0000 7fea60c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.168+0000 7fea60c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c074290 con 0x7fea5c0730d0 2024-04-25T12:34:34.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.168+0000 7fea60c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c074570 con 0x7fea5c072cc0 2024-04-25T12:34:34.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.168+0000 7fea60c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c074850 con 0x7fea5c0c7e30 2024-04-25T12:34:34.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fea5c0730d0 0x7fea5c15dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40726/0 (socket says 172.21.15.123:40726) 2024-04-25T12:34:34.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5be00640 1 -- 172.21.15.123:0/1214226537 learned_addr learned my addr 172.21.15.123:0/1214226537 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:34.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5b400640 1 --2- 172.21.15.123:0/1214226537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5a000640 1 -- 172.21.15.123:0/1214226537 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 135170178 0 0) 0x7fea5c074290 con 0x7fea5c0730d0 2024-04-25T12:34:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5a000640 1 -- 172.21.15.123:0/1214226537 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea40003520 con 0x7fea5c0730d0 2024-04-25T12:34:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5a000640 1 -- 172.21.15.123:0/1214226537 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1758538201 0 0) 0x7fea5c074570 con 0x7fea5c072cc0 2024-04-25T12:34:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5a000640 1 -- 172.21.15.123:0/1214226537 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea5c074290 con 0x7fea5c072cc0 2024-04-25T12:34:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5b400640 1 -- 172.21.15.123:0/1214226537 >> v1:172.21.15.153:6789/0 conn(0x7fea5c072cc0 legacy=0x7fea5c06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5b400640 1 -- 172.21.15.123:0/1214226537 >> v1:172.21.15.123:6789/0 conn(0x7fea5c0730d0 legacy=0x7fea5c15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.169+0000 7fea5b400640 1 -- 172.21.15.123:0/1214226537 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c074b30 con 0x7fea5c0c7e30 2024-04-25T12:34:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.170+0000 7fea5b400640 1 --2- 172.21.15.123:0/1214226537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c154280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0097d0 tx=0x7fea4c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64a5fff40a24688 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea5a000640 1 -- 172.21.15.123:0/1214226537 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c037070 con 0x7fea5c0c7e30 2024-04-25T12:34:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea5a000640 1 -- 172.21.15.123:0/1214226537 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c032440 con 0x7fea5c0c7e30 2024-04-25T12:34:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea5a000640 1 -- 172.21.15.123:0/1214226537 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c032810 con 0x7fea5c0c7e30 2024-04-25T12:34:34.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea60c00640 1 -- 172.21.15.123:0/1214226537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 msgr2=0x7fea5c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea60c00640 1 --2- 172.21.15.123:0/1214226537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c154280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0097d0 tx=0x7fea4c02a210 comp rx=0 tx=0).stop 2024-04-25T12:34:34.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea60c00640 1 -- 172.21.15.123:0/1214226537 shutdown_connections 2024-04-25T12:34:34.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea60c00640 1 --2- 172.21.15.123:0/1214226537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c154280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.171+0000 7fea60c00640 1 -- 172.21.15.123:0/1214226537 >> 172.21.15.123:0/1214226537 conn(0x7fea5c070b70 msgr2=0x7fea5c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:34.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.172+0000 7fea60c00640 1 -- 172.21.15.123:0/1214226537 shutdown_connections 2024-04-25T12:34:34.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.172+0000 7fea60c00640 1 -- 172.21.15.123:0/1214226537 wait complete. 2024-04-25T12:34:34.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.174+0000 7fea60c00640 1 Processor -- start 2024-04-25T12:34:34.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.174+0000 7fea60c00640 1 -- start start 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.175+0000 7fea60c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea5c072cc0 0x7fea5c159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.175+0000 7fea60c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5c0730d0 0x7fea5c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.175+0000 7fea60c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.175+0000 7fea60c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0c8240 con 0x7fea5c0730d0 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.175+0000 7fea60c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1547c0 con 0x7fea5c072cc0 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.175+0000 7fea60c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c0c83c0 con 0x7fea5c0c7e30 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5c0730d0 0x7fea5c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea5c072cc0 0x7fea5c159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:50708/0 (socket says 172.21.15.123:50708) 2024-04-25T12:34:34.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5be00640 1 -- 172.21.15.123:0/2569322004 learned_addr learned my addr 172.21.15.123:0/2569322004 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5aa00640 1 -- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 msgr2=0x7fea5c158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5aa00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5aa00640 1 -- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea5c072cc0 msgr2=0x7fea5c159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5aa00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea5c072cc0 0x7fea5c159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5aa00640 1 -- 172.21.15.123:0/2569322004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c1551a0 con 0x7fea5c0730d0 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.176+0000 7fea5aa00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5c0730d0 0x7fea5c154c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea4800c3c0 tx=0x7fea4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.177+0000 7fea53400640 1 -- 172.21.15.123:0/2569322004 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48017070 con 0x7fea5c0730d0 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.177+0000 7fea53400640 1 -- 172.21.15.123:0/2569322004 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea480052f0 con 0x7fea5c0730d0 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.177+0000 7fea53400640 1 -- 172.21.15.123:0/2569322004 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48003830 con 0x7fea5c0730d0 2024-04-25T12:34:34.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.177+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c1f04c0 con 0x7fea5c0730d0 2024-04-25T12:34:34.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.177+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1f0a80 con 0x7fea5c0730d0 2024-04-25T12:34:34.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.178+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c076da0 con 0x7fea5c0730d0 2024-04-25T12:34:34.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.179+0000 7fea53400640 1 -- 172.21.15.123:0/2569322004 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea4801a070 con 0x7fea5c0730d0 2024-04-25T12:34:34.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.180+0000 7fea53400640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fea3c0744b0 0x7fea3c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.180+0000 7fea53400640 1 -- 172.21.15.123:0/2569322004 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fea48094790 con 0x7fea5c0730d0 2024-04-25T12:34:34.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.180+0000 7fea5b400640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fea3c0744b0 0x7fea3c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.181+0000 7fea5b400640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fea3c0744b0 0x7fea3c0769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea4c034000 tx=0x7fea4c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:34.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.185+0000 7fea53400640 1 -- 172.21.15.123:0/2569322004 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea4805e620 con 0x7fea5c0730d0 2024-04-25T12:34:34.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.385+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fea5c14e090 con 0x7fea5c0730d0 2024-04-25T12:34:34.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.386+0000 7fea53400640 1 -- 172.21.15.123:0/2569322004 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fea48062920 con 0x7fea5c0730d0 2024-04-25T12:34:34.386 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:34:34.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fea3c0744b0 msgr2=0x7fea3c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fea3c0744b0 0x7fea3c0769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea4c034000 tx=0x7fea4c02e000 comp rx=0 tx=0).stop 2024-04-25T12:34:34.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5c0730d0 msgr2=0x7fea5c154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5c0730d0 0x7fea5c154c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea4800c3c0 tx=0x7fea4800c880 comp rx=0 tx=0).stop 2024-04-25T12:34:34.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 shutdown_connections 2024-04-25T12:34:34.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fea3c0744b0 0x7fea3c0769c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea5c0c7e30 0x7fea5c158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5c0730d0 0x7fea5c154c60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 --2- 172.21.15.123:0/2569322004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea5c072cc0 0x7fea5c159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.389+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 >> 172.21.15.123:0/2569322004 conn(0x7fea5c070b70 msgr2=0x7fea5c15b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:34.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.390+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 shutdown_connections 2024-04-25T12:34:34.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.390+0000 7fea60c00640 1 -- 172.21.15.123:0/2569322004 wait complete. 2024-04-25T12:34:34.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:34:34.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.676+0000 7fde48a00640 1 Processor -- start 2024-04-25T12:34:34.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.676+0000 7fde48a00640 1 -- start start 2024-04-25T12:34:34.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde48a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c2790 0x7fde440c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde48a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde440c8170 0x7fde4415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde48a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fde4406ef20 0x7fde44156360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde48a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fde440c7490 con 0x7fde440c8170 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde48a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fde440c4290 con 0x7fde4406ef20 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde48a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fde44074580 con 0x7fde440c2790 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde41400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde440c8170 0x7fde4415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde41400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde440c8170 0x7fde4415cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51334/0 (socket says 172.21.15.123:51334) 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde41400640 1 -- 172.21.15.123:0/2893116748 learned_addr learned my addr 172.21.15.123:0/2893116748 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde42800640 1 --2- 172.21.15.123:0/2893116748 >> v2:172.21.15.153:3300/0 conn(0x7fde4406ef20 0x7fde44156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.679+0000 7fde41e00640 1 --2- 172.21.15.123:0/2893116748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c2790 0x7fde440c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.680+0000 7fde41400640 1 -- 172.21.15.123:0/2893116748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c2790 msgr2=0x7fde440c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:34.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.680+0000 7fde41400640 1 --2- 172.21.15.123:0/2893116748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c2790 0x7fde440c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.680+0000 7fde41400640 1 -- 172.21.15.123:0/2893116748 >> v2:172.21.15.153:3300/0 conn(0x7fde4406ef20 msgr2=0x7fde44156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:34.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.680+0000 7fde41400640 1 --2- 172.21.15.123:0/2893116748 >> v2:172.21.15.153:3300/0 conn(0x7fde4406ef20 0x7fde44156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.680+0000 7fde41400640 1 -- 172.21.15.123:0/2893116748 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4406f9e0 con 0x7fde440c8170 2024-04-25T12:34:34.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.681+0000 7fde41400640 1 --2- 172.21.15.123:0/2893116748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde440c8170 0x7fde4415cff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde4406fe90 tx=0x7fde3402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f04b51620079ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:34.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.681+0000 7fde40a00640 1 -- 172.21.15.123:0/2893116748 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34037070 con 0x7fde440c8170 2024-04-25T12:34:34.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.681+0000 7fde40a00640 1 -- 172.21.15.123:0/2893116748 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde34032440 con 0x7fde440c8170 2024-04-25T12:34:34.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.681+0000 7fde40a00640 1 -- 172.21.15.123:0/2893116748 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34032810 con 0x7fde440c8170 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 -- 172.21.15.123:0/2893116748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde440c8170 msgr2=0x7fde4415cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 --2- 172.21.15.123:0/2893116748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde440c8170 0x7fde4415cff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde4406fe90 tx=0x7fde3402a1e0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 -- 172.21.15.123:0/2893116748 shutdown_connections 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 --2- 172.21.15.123:0/2893116748 >> v2:172.21.15.153:3300/0 conn(0x7fde4406ef20 0x7fde44156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 --2- 172.21.15.123:0/2893116748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde440c8170 0x7fde4415cff0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 --2- 172.21.15.123:0/2893116748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c2790 0x7fde440c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 -- 172.21.15.123:0/2893116748 >> 172.21.15.123:0/2893116748 conn(0x7fde440ca270 msgr2=0x7fde440ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 -- 172.21.15.123:0/2893116748 shutdown_connections 2024-04-25T12:34:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.682+0000 7fde48a00640 1 -- 172.21.15.123:0/2893116748 wait complete. 2024-04-25T12:34:34.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.685+0000 7fde48a00640 1 Processor -- start 2024-04-25T12:34:34.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.685+0000 7fde48a00640 1 -- start start 2024-04-25T12:34:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.686+0000 7fde48a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4406ef20 0x7fde440c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.686+0000 7fde48a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde440c2790 0x7fde44154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.686+0000 7fde48a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c8170 0x7fde44159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.686+0000 7fde48a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde440c4810 con 0x7fde4406ef20 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.686+0000 7fde48a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde440c4530 con 0x7fde440c2790 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.686+0000 7fde48a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fde440c8580 con 0x7fde440c8170 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4406ef20 0x7fde440c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde42800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c8170 0x7fde44159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4406ef20 0x7fde440c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51344/0 (socket says 172.21.15.123:51344) 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde440c2790 0x7fde44154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 -- 172.21.15.123:0/2547574719 learned_addr learned my addr 172.21.15.123:0/2547574719 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 -- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c8170 msgr2=0x7fde44159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c8170 0x7fde44159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 -- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde440c2790 msgr2=0x7fde44154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde440c2790 0x7fde44154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde41e00640 1 -- 172.21.15.123:0/2547574719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde44154980 con 0x7fde4406ef20 2024-04-25T12:34:34.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.687+0000 7fde42800640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c8170 0x7fde44159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:34.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.688+0000 7fde41e00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4406ef20 0x7fde440c6a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00e8a0 tx=0x7fde2c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:34.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.688+0000 7fde3a000640 1 -- 172.21.15.123:0/2547574719 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c017070 con 0x7fde4406ef20 2024-04-25T12:34:34.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.688+0000 7fde3a000640 1 -- 172.21.15.123:0/2547574719 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2c00c940 con 0x7fde4406ef20 2024-04-25T12:34:34.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.688+0000 7fde3a000640 1 -- 172.21.15.123:0/2547574719 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00cd30 con 0x7fde4406ef20 2024-04-25T12:34:34.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.688+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde44157840 con 0x7fde4406ef20 2024-04-25T12:34:34.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.688+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde44157e00 con 0x7fde4406ef20 2024-04-25T12:34:34.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.689+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde44076700 con 0x7fde4406ef20 2024-04-25T12:34:34.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.690+0000 7fde3a000640 1 -- 172.21.15.123:0/2547574719 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde2c01a070 con 0x7fde4406ef20 2024-04-25T12:34:34.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.691+0000 7fde3a000640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744e0 0x7fde200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:34.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.691+0000 7fde3a000640 1 -- 172.21.15.123:0/2547574719 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fde2c095260 con 0x7fde4406ef20 2024-04-25T12:34:34.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.692+0000 7fde41400640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744e0 0x7fde200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:34.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.693+0000 7fde41400640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744e0 0x7fde200769f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde34034000 tx=0x7fde3402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:34.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.696+0000 7fde3a000640 1 -- 172.21.15.123:0/2547574719 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde2c05f180 con 0x7fde4406ef20 2024-04-25T12:34:34.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.899+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde44069920 con 0x7fde4406ef20 2024-04-25T12:34:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.900+0000 7fde3a000640 1 -- 172.21.15.123:0/2547574719 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fde2c063480 con 0x7fde4406ef20 2024-04-25T12:34:34.899 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T12:34:34.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.902+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744e0 msgr2=0x7fde200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.902+0000 7fde48a00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744e0 0x7fde200769f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde34034000 tx=0x7fde3402e000 comp rx=0 tx=0).stop 2024-04-25T12:34:34.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.902+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4406ef20 msgr2=0x7fde440c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:34.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.902+0000 7fde48a00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4406ef20 0x7fde440c6a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00e8a0 tx=0x7fde2c00ed60 comp rx=0 tx=0).stop 2024-04-25T12:34:34.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 shutdown_connections 2024-04-25T12:34:34.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744e0 0x7fde200769f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde440c8170 0x7fde44159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde440c2790 0x7fde44154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 --2- 172.21.15.123:0/2547574719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4406ef20 0x7fde440c6a70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:34.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 >> 172.21.15.123:0/2547574719 conn(0x7fde440ca270 msgr2=0x7fde4415aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:34.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 shutdown_connections 2024-04-25T12:34:34.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:34.903+0000 7fde48a00640 1 -- 172.21.15.123:0/2547574719 wait complete. 2024-04-25T12:34:34.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T12:34:35.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.198+0000 7ff440e00640 1 Processor -- start 2024-04-25T12:34:35.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.199+0000 7ff440e00640 1 -- start start 2024-04-25T12:34:35.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.201+0000 7ff440e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff43c0caa70 0x7ff43c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.201+0000 7ff440e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43c0c43d0 con 0x7ff43c074840 2024-04-25T12:34:35.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.201+0000 7ff440e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43c06ca00 con 0x7ff43c15c420 2024-04-25T12:34:35.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.201+0000 7ff440e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff43c06c7b0 con 0x7ff43c0caa70 2024-04-25T12:34:35.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.201+0000 7ff43aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff43c0caa70 0x7ff43c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.201+0000 7ff43be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff43c074840 0x7ff43c166cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40728/0 (socket says 172.21.15.123:40728) 2024-04-25T12:34:35.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.201+0000 7ff43be00640 1 -- 172.21.15.123:0/500175542 learned_addr learned my addr 172.21.15.123:0/500175542 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:35.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.202+0000 7ff43a000640 1 -- 172.21.15.123:0/500175542 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270032011 0 0) 0x7ff43c06ca00 con 0x7ff43c15c420 2024-04-25T12:34:35.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.202+0000 7ff43a000640 1 -- 172.21.15.123:0/500175542 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff420003520 con 0x7ff43c15c420 2024-04-25T12:34:35.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.202+0000 7ff43aa00640 1 -- 172.21.15.123:0/500175542 >> v1:172.21.15.153:6789/0 conn(0x7ff43c15c420 legacy=0x7ff43c0c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.202+0000 7ff43aa00640 1 -- 172.21.15.123:0/500175542 >> v1:172.21.15.123:6789/0 conn(0x7ff43c074840 legacy=0x7ff43c166cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.202+0000 7ff43a000640 1 -- 172.21.15.123:0/500175542 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559506459 0 0) 0x7ff43c0c43d0 con 0x7ff43c074840 2024-04-25T12:34:35.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.202+0000 7ff43aa00640 1 -- 172.21.15.123:0/500175542 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c06cce0 con 0x7ff43c0caa70 2024-04-25T12:34:35.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.203+0000 7ff43aa00640 1 --2- 172.21.15.123:0/500175542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0c6fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff43c06c930 tx=0x7ff42802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c77eb0d77f43608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:35.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.203+0000 7ff43a000640 1 -- 172.21.15.123:0/500175542 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428037070 con 0x7ff43c0caa70 2024-04-25T12:34:35.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.203+0000 7ff43a000640 1 -- 172.21.15.123:0/500175542 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428032440 con 0x7ff43c0caa70 2024-04-25T12:34:35.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.203+0000 7ff43a000640 1 -- 172.21.15.123:0/500175542 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280327f0 con 0x7ff43c0caa70 2024-04-25T12:34:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.204+0000 7ff440e00640 1 -- 172.21.15.123:0/500175542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 msgr2=0x7ff43c0c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.204+0000 7ff440e00640 1 --2- 172.21.15.123:0/500175542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0c6fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff43c06c930 tx=0x7ff42802a210 comp rx=0 tx=0).stop 2024-04-25T12:34:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.204+0000 7ff440e00640 1 -- 172.21.15.123:0/500175542 shutdown_connections 2024-04-25T12:34:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.204+0000 7ff440e00640 1 --2- 172.21.15.123:0/500175542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0c6fb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.204+0000 7ff440e00640 1 -- 172.21.15.123:0/500175542 >> 172.21.15.123:0/500175542 conn(0x7ff43c070b70 msgr2=0x7ff43c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.204+0000 7ff440e00640 1 -- 172.21.15.123:0/500175542 shutdown_connections 2024-04-25T12:34:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.204+0000 7ff440e00640 1 -- 172.21.15.123:0/500175542 wait complete. 2024-04-25T12:34:35.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.206+0000 7ff440e00640 1 Processor -- start 2024-04-25T12:34:35.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.207+0000 7ff440e00640 1 -- start start 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff440e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff43c074840 0x7ff43c160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff440e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff440e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c15c420 0x7ff43c15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff440e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c0c3f70 con 0x7ff43c0caa70 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff440e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c0c7dc0 con 0x7ff43c15c420 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff440e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff43c0c40f0 con 0x7ff43c074840 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff43aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff43b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff43c074840 0x7ff43c160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff43aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51356/0 (socket says 172.21.15.123:51356) 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff43aa00640 1 -- 172.21.15.123:0/2170960268 learned_addr learned my addr 172.21.15.123:0/2170960268 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.208+0000 7ff43be00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c15c420 0x7ff43c15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff43b400640 1 -- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c15c420 msgr2=0x7ff43c15b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff43b400640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c15c420 0x7ff43c15b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff43b400640 1 -- 172.21.15.123:0/2170960268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 msgr2=0x7ff43c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff43b400640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff43b400640 1 -- 172.21.15.123:0/2170960268 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c0cbd50 con 0x7ff43c074840 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff43aa00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff43b400640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff43c074840 0x7ff43c160470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c710 tx=0x7ff42c00cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff433400640 1 -- 172.21.15.123:0/2170960268 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c003b70 con 0x7ff43c074840 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff433400640 1 -- 172.21.15.123:0/2170960268 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42c003d10 con 0x7ff43c074840 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff433400640 1 -- 172.21.15.123:0/2170960268 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00aa90 con 0x7ff43c074840 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.209+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c1f2f10 con 0x7ff43c074840 2024-04-25T12:34:35.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.210+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c15e200 con 0x7ff43c074840 2024-04-25T12:34:35.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.211+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c076da0 con 0x7ff43c074840 2024-04-25T12:34:35.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.212+0000 7ff433400640 1 -- 172.21.15.123:0/2170960268 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff42c00ac30 con 0x7ff43c074840 2024-04-25T12:34:35.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.213+0000 7ff433400640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff41c0744e0 0x7ff41c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.213+0000 7ff433400640 1 -- 172.21.15.123:0/2170960268 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff42c095330 con 0x7ff43c074840 2024-04-25T12:34:35.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.213+0000 7ff43aa00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff41c0744e0 0x7ff41c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.214+0000 7ff43aa00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff41c0744e0 0x7ff41c0769f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff43c156c20 tx=0x7ff42802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:35.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.218+0000 7ff433400640 1 -- 172.21.15.123:0/2170960268 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff42c00ce20 con 0x7ff43c074840 2024-04-25T12:34:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.417+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff43c0ccf10 con 0x7ff43c074840 2024-04-25T12:34:35.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.418+0000 7ff433400640 1 -- 172.21.15.123:0/2170960268 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7ff42c0950f0 con 0x7ff43c074840 2024-04-25T12:34:35.418 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:34:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff41c0744e0 msgr2=0x7ff41c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff41c0744e0 0x7ff41c0769f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff43c156c20 tx=0x7ff42802e000 comp rx=0 tx=0).stop 2024-04-25T12:34:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff43c074840 msgr2=0x7ff43c160470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff43c074840 0x7ff43c160470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c710 tx=0x7ff42c00cbd0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 shutdown_connections 2024-04-25T12:34:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff41c0744e0 0x7ff41c0769f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c15c420 0x7ff43c15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff43c0caa70 0x7ff43c0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 --2- 172.21.15.123:0/2170960268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff43c074840 0x7ff43c160470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 >> 172.21.15.123:0/2170960268 conn(0x7ff43c070b70 msgr2=0x7ff43c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 shutdown_connections 2024-04-25T12:34:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.422+0000 7ff440e00640 1 -- 172.21.15.123:0/2170960268 wait complete. 2024-04-25T12:34:35.449 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T12:34:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.721+0000 7fcc66a00640 1 Processor -- start 2024-04-25T12:34:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.722+0000 7fcc66a00640 1 -- start start 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.723+0000 7fcc66a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc60072d70 0x7fcc601420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.723+0000 7fcc66a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc600c94f0 0x7fcc600c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.723+0000 7fcc66a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc601593c0 con 0x7fcc600c2720 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.723+0000 7fcc66a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fcc60159fd0 con 0x7fcc600c94f0 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.723+0000 7fcc66a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcc601596a0 con 0x7fcc60072d70 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc64c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc600c94f0 0x7fcc600c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc64c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc600c94f0 0x7fcc600c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51360/0 (socket says 172.21.15.123:51360) 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc64c00640 1 -- 172.21.15.123:0/520847456 learned_addr learned my addr 172.21.15.123:0/520847456 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc65600640 1 --2- 172.21.15.123:0/520847456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc60072d70 0x7fcc601420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc64c00640 1 -- 172.21.15.123:0/520847456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc60072d70 msgr2=0x7fcc601420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc64c00640 1 --2- 172.21.15.123:0/520847456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc60072d70 0x7fcc601420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc64c00640 1 -- 172.21.15.123:0/520847456 >> v1:172.21.15.153:6789/0 conn(0x7fcc600c2720 legacy=0x7fcc600c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.724+0000 7fcc64c00640 1 -- 172.21.15.123:0/520847456 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc60159980 con 0x7fcc600c94f0 2024-04-25T12:34:35.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.725+0000 7fcc64c00640 1 --2- 172.21.15.123:0/520847456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc600c94f0 0x7fcc600c9900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009870 tx=0x7fcc4c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f00ecd94f7a219a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:35.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.726+0000 7fcc5fe00640 1 -- 172.21.15.123:0/520847456 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c037070 con 0x7fcc600c94f0 2024-04-25T12:34:35.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.726+0000 7fcc5fe00640 1 -- 172.21.15.123:0/520847456 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c02ae50 con 0x7fcc600c94f0 2024-04-25T12:34:35.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.726+0000 7fcc5fe00640 1 -- 172.21.15.123:0/520847456 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c032860 con 0x7fcc600c94f0 2024-04-25T12:34:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.726+0000 7fcc66a00640 1 -- 172.21.15.123:0/520847456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc600c94f0 msgr2=0x7fcc600c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.726+0000 7fcc66a00640 1 --2- 172.21.15.123:0/520847456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc600c94f0 0x7fcc600c9900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009870 tx=0x7fcc4c02a300 comp rx=0 tx=0).stop 2024-04-25T12:34:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.727+0000 7fcc66a00640 1 -- 172.21.15.123:0/520847456 shutdown_connections 2024-04-25T12:34:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.727+0000 7fcc66a00640 1 --2- 172.21.15.123:0/520847456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc600c94f0 0x7fcc600c9900 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.727+0000 7fcc66a00640 1 --2- 172.21.15.123:0/520847456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc60072d70 0x7fcc601420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.727+0000 7fcc66a00640 1 -- 172.21.15.123:0/520847456 >> 172.21.15.123:0/520847456 conn(0x7fcc600cd600 msgr2=0x7fcc600cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:35.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.727+0000 7fcc66a00640 1 -- 172.21.15.123:0/520847456 shutdown_connections 2024-04-25T12:34:35.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.727+0000 7fcc66a00640 1 -- 172.21.15.123:0/520847456 wait complete. 2024-04-25T12:34:35.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.729+0000 7fcc66a00640 1 Processor -- start 2024-04-25T12:34:35.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.729+0000 7fcc66a00640 1 -- start start 2024-04-25T12:34:35.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.730+0000 7fcc66a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc60072d70 0x7fcc60154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.730+0000 7fcc66a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc600c2720 0x7fcc600ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.730+0000 7fcc66a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc600c94f0 0x7fcc601f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.730+0000 7fcc66a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc601427a0 con 0x7fcc60072d70 2024-04-25T12:34:35.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.730+0000 7fcc66a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60142620 con 0x7fcc600c94f0 2024-04-25T12:34:35.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.730+0000 7fcc66a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcc600c8810 con 0x7fcc600c2720 2024-04-25T12:34:35.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc65600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc60072d70 0x7fcc60154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc64c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc600c2720 0x7fcc600ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc65600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc60072d70 0x7fcc60154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51362/0 (socket says 172.21.15.123:51362) 2024-04-25T12:34:35.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc65600640 1 -- 172.21.15.123:0/2779128915 learned_addr learned my addr 172.21.15.123:0/2779128915 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:35.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc66000640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc600c94f0 0x7fcc601f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc64c00640 1 -- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc600c94f0 msgr2=0x7fcc601f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc64c00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc600c94f0 0x7fcc601f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc64c00640 1 -- 172.21.15.123:0/2779128915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc60072d70 msgr2=0x7fcc60154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc64c00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc60072d70 0x7fcc60154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc64c00640 1 -- 172.21.15.123:0/2779128915 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6013e3a0 con 0x7fcc600c2720 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc66000640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc600c94f0 0x7fcc601f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.731+0000 7fcc65600640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc60072d70 0x7fcc60154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.732+0000 7fcc64c00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc600c2720 0x7fcc600ccd20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc6006feb0 tx=0x7fcc4c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.732+0000 7fcc5d600640 1 -- 172.21.15.123:0/2779128915 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c041020 con 0x7fcc600c2720 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.732+0000 7fcc5d600640 1 -- 172.21.15.123:0/2779128915 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c03c890 con 0x7fcc600c2720 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.732+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6013e660 con 0x7fcc600c2720 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.732+0000 7fcc5d600640 1 -- 172.21.15.123:0/2779128915 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c037040 con 0x7fcc600c2720 2024-04-25T12:34:35.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.732+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6013cb80 con 0x7fcc600c2720 2024-04-25T12:34:35.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.733+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc60076da0 con 0x7fcc600c2720 2024-04-25T12:34:35.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.735+0000 7fcc5d600640 1 -- 172.21.15.123:0/2779128915 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcc4c044020 con 0x7fcc600c2720 2024-04-25T12:34:35.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.735+0000 7fcc5d600640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcc400744e0 0x7fcc400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:35.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.736+0000 7fcc5d600640 1 -- 172.21.15.123:0/2779128915 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcc4c0b5720 con 0x7fcc600c2720 2024-04-25T12:34:35.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.736+0000 7fcc65600640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcc400744e0 0x7fcc400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:35.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.736+0000 7fcc65600640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcc400744e0 0x7fcc400769f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009f50 tx=0x7fcc54009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.741+0000 7fcc5d600640 1 -- 172.21.15.123:0/2779128915 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc4c04b020 con 0x7fcc600c2720 2024-04-25T12:34:35.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.952+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcc60070380 con 0x7fcc600c2720 2024-04-25T12:34:35.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.953+0000 7fcc5d600640 1 -- 172.21.15.123:0/2779128915 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fcc4c07f5b0 con 0x7fcc600c2720 2024-04-25T12:34:35.952 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T12:34:35.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcc400744e0 msgr2=0x7fcc400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcc400744e0 0x7fcc400769f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009f50 tx=0x7fcc54009190 comp rx=0 tx=0).stop 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc600c2720 msgr2=0x7fcc600ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc600c2720 0x7fcc600ccd20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc6006feb0 tx=0x7fcc4c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 shutdown_connections 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcc400744e0 0x7fcc400769f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc600c94f0 0x7fcc601f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc600c2720 0x7fcc600ccd20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 --2- 172.21.15.123:0/2779128915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc60072d70 0x7fcc60154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 >> 172.21.15.123:0/2779128915 conn(0x7fcc600cd600 msgr2=0x7fcc60074350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 shutdown_connections 2024-04-25T12:34:35.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:35.956+0000 7fcc66a00640 1 -- 172.21.15.123:0/2779128915 wait complete. 2024-04-25T12:34:35.984 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-04-25T12:34:35.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:35.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T12:34:36.063 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[38459]: fuse finished with error 0 and tester_r 0 2024-04-25T12:34:36.069 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:34:37.103 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:34:37.111 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:34:42.171 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:34:42.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:34:42.171 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T12:34:42.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:42.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.515+0000 7fcafb200640 1 Processor -- start 2024-04-25T12:34:42.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.516+0000 7fcafb200640 1 -- start start 2024-04-25T12:34:42.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.518+0000 7fcafb200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcafc0c9900 0x7fcafc0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:42.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.518+0000 7fcafb200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcafc0c9d10 con 0x7fcafc072d70 2024-04-25T12:34:42.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.518+0000 7fcafb200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcafc1593c0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.518+0000 7fcafb200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fcafc159fd0 con 0x7fcafc0c9900 2024-04-25T12:34:42.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.518+0000 7fcaf3e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcafc0c94f0 0x7fcafc0c2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:45944/0 (socket says 172.21.15.123:45944) 2024-04-25T12:34:42.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.518+0000 7fcafa800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcafc0c9900 0x7fcafc0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:42.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.518+0000 7fcaf3e00640 1 -- 172.21.15.123:0/3460115 learned_addr learned my addr 172.21.15.123:0/3460115 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:42.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.519+0000 7fcaf9400640 1 -- 172.21.15.123:0/3460115 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752450463 0 0) 0x7fcafc1593c0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.519+0000 7fcaf9400640 1 -- 172.21.15.123:0/3460115 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcae0003520 con 0x7fcafc0c94f0 2024-04-25T12:34:42.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.519+0000 7fcafa800640 1 -- 172.21.15.123:0/3460115 >> v1:172.21.15.153:6789/0 conn(0x7fcafc0c94f0 legacy=0x7fcafc0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:42.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.519+0000 7fcafa800640 1 -- 172.21.15.123:0/3460115 >> v1:172.21.15.123:6789/0 conn(0x7fcafc072d70 legacy=0x7fcafc1420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:42.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.519+0000 7fcaf9400640 1 -- 172.21.15.123:0/3460115 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142262264 0 0) 0x7fcafc0c9d10 con 0x7fcafc072d70 2024-04-25T12:34:42.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.519+0000 7fcafa800640 1 -- 172.21.15.123:0/3460115 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc1596a0 con 0x7fcafc0c9900 2024-04-25T12:34:42.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.520+0000 7fcafa800640 1 --2- 172.21.15.123:0/3460115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c9900 0x7fcafc0c8100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009d90 tx=0x7fcaf402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9106d0c868d3610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:42.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.520+0000 7fcaf9400640 1 -- 172.21.15.123:0/3460115 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4037070 con 0x7fcafc0c9900 2024-04-25T12:34:42.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.520+0000 7fcaf9400640 1 -- 172.21.15.123:0/3460115 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf4007740 con 0x7fcafc0c9900 2024-04-25T12:34:42.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.520+0000 7fcaf9400640 1 -- 172.21.15.123:0/3460115 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4007b10 con 0x7fcafc0c9900 2024-04-25T12:34:42.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.521+0000 7fcafb200640 1 -- 172.21.15.123:0/3460115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c9900 msgr2=0x7fcafc0c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:42.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.521+0000 7fcafb200640 1 --2- 172.21.15.123:0/3460115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c9900 0x7fcafc0c8100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009d90 tx=0x7fcaf402a780 comp rx=0 tx=0).stop 2024-04-25T12:34:42.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.521+0000 7fcafb200640 1 -- 172.21.15.123:0/3460115 shutdown_connections 2024-04-25T12:34:42.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.521+0000 7fcafb200640 1 --2- 172.21.15.123:0/3460115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c9900 0x7fcafc0c8100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:42.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.521+0000 7fcafb200640 1 -- 172.21.15.123:0/3460115 >> 172.21.15.123:0/3460115 conn(0x7fcafc0cd600 msgr2=0x7fcafc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:42.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.521+0000 7fcafb200640 1 -- 172.21.15.123:0/3460115 shutdown_connections 2024-04-25T12:34:42.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.521+0000 7fcafb200640 1 -- 172.21.15.123:0/3460115 wait complete. 2024-04-25T12:34:42.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.524+0000 7fcafb200640 1 Processor -- start 2024-04-25T12:34:42.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.524+0000 7fcafb200640 1 -- start start 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.525+0000 7fcafb200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcafc072d70 0x7fcafc15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.525+0000 7fcafb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c94f0 0x7fcafc0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.525+0000 7fcafb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcafc0c9900 0x7fcafc137be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.525+0000 7fcafb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc0c2e30 con 0x7fcafc0c94f0 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.525+0000 7fcafb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc159c60 con 0x7fcafc0c9900 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.525+0000 7fcafb200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcafc0c2fb0 con 0x7fcafc072d70 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c94f0 0x7fcafc0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:42.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcafa800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcafc0c9900 0x7fcafc137be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:42.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf9e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcafc072d70 0x7fcafc15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:42.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcafa800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcafc0c9900 0x7fcafc137be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:52360/0 (socket says 172.21.15.123:52360) 2024-04-25T12:34:42.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcafa800640 1 -- 172.21.15.123:0/3813447644 learned_addr learned my addr 172.21.15.123:0/3813447644 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:42.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c94f0 0x7fcafc0c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51378/0 (socket says 172.21.15.123:51378) 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 -- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcafc072d70 msgr2=0x7fcafc15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcafc072d70 0x7fcafc15ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 -- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcafc0c9900 msgr2=0x7fcafc137be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcafc0c9900 0x7fcafc137be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 -- 172.21.15.123:0/3813447644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc15d1c0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.526+0000 7fcaf3e00640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c94f0 0x7fcafc0c75a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcae400c3c0 tx=0x7fcae4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.527+0000 7fcaf2000640 1 -- 172.21.15.123:0/3813447644 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4017070 con 0x7fcafc0c94f0 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.527+0000 7fcaf2000640 1 -- 172.21.15.123:0/3813447644 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae4005ce0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.527+0000 7fcaf2000640 1 -- 172.21.15.123:0/3813447644 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4007310 con 0x7fcafc0c94f0 2024-04-25T12:34:42.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.527+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc15b0b0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.527+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc0b1ed0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.528+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc076da0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.529+0000 7fcaf2000640 1 -- 172.21.15.123:0/3813447644 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcae401a070 con 0x7fcafc0c94f0 2024-04-25T12:34:42.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.530+0000 7fcaf2000640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcad80744b0 0x7fcad80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:42.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.530+0000 7fcaf2000640 1 -- 172.21.15.123:0/3813447644 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcae40947a0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.531+0000 7fcaf9e00640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcad80744b0 0x7fcad80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:42.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.531+0000 7fcaf9e00640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcad80744b0 0x7fcad80769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000f60 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:42.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.535+0000 7fcaf2000640 1 -- 172.21.15.123:0/3813447644 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcafc15d1c0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.739+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcafc156a90 con 0x7fcafc0c94f0 2024-04-25T12:34:42.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.742+0000 7fcaf2000640 1 -- 172.21.15.123:0/3813447644 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fcae405e5a0 con 0x7fcafc0c94f0 2024-04-25T12:34:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:42.742 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcad80744b0 msgr2=0x7fcad80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcad80744b0 0x7fcad80769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000f60 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c94f0 msgr2=0x7fcafc0c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c94f0 0x7fcafc0c75a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcae400c3c0 tx=0x7fcae4001e70 comp rx=0 tx=0).stop 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 shutdown_connections 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcad80744b0 0x7fcad80769c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcafc0c9900 0x7fcafc137be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcafc0c94f0 0x7fcafc0c75a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 --2- 172.21.15.123:0/3813447644 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcafc072d70 0x7fcafc15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 >> 172.21.15.123:0/3813447644 conn(0x7fcafc0cd600 msgr2=0x7fcafc06cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 shutdown_connections 2024-04-25T12:34:42.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:42.745+0000 7fcafb200640 1 -- 172.21.15.123:0/3813447644 wait complete. 2024-04-25T12:34:42.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.044+0000 7fa7bb200640 1 Processor -- start 2024-04-25T12:34:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.045+0000 7fa7bb200640 1 -- start start 2024-04-25T12:34:43.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7bb200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b40c9080 0x7fa7b41546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7bb200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa7b40c82b0 0x7fa7b415e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7bb200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b415a330 con 0x7fa7b415e590 2024-04-25T12:34:43.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7bb200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b4073b10 con 0x7fa7b40c82b0 2024-04-25T12:34:43.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7bb200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b4073df0 con 0x7fa7b40c9080 2024-04-25T12:34:43.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7b9400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa7b40c82b0 0x7fa7b415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7b9e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b40c9080 0x7fa7b41546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7ba800640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa7b415e590 0x7fa7b406c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:41438/0 (socket says 172.21.15.123:41438) 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.047+0000 7fa7ba800640 1 -- 172.21.15.123:0/2390392799 learned_addr learned my addr 172.21.15.123:0/2390392799 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.048+0000 7fa7b8a00640 1 -- 172.21.15.123:0/2390392799 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911282006 0 0) 0x7fa7b415a330 con 0x7fa7b415e590 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.048+0000 7fa7b9400640 1 -- 172.21.15.123:0/2390392799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b40c9080 msgr2=0x7fa7b41546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.048+0000 7fa7b9400640 1 --2- 172.21.15.123:0/2390392799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b40c9080 0x7fa7b41546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.048+0000 7fa7b9400640 1 -- 172.21.15.123:0/2390392799 >> v1:172.21.15.153:6789/0 conn(0x7fa7b415e590 legacy=0x7fa7b406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.048+0000 7fa7b9400640 1 -- 172.21.15.123:0/2390392799 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40740d0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.048+0000 7fa7b9e00640 1 --2- 172.21.15.123:0/2390392799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b40c9080 0x7fa7b41546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.049+0000 7fa7b9400640 1 --2- 172.21.15.123:0/2390392799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b415e050 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7b415a500 tx=0x7fa7a002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5959c6b4c73578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7b8a00640 1 -- 172.21.15.123:0/2390392799 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0037070 con 0x7fa7b40c82b0 2024-04-25T12:34:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7b8a00640 1 -- 172.21.15.123:0/2390392799 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a0032440 con 0x7fa7b40c82b0 2024-04-25T12:34:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7b8a00640 1 -- 172.21.15.123:0/2390392799 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0032810 con 0x7fa7b40c82b0 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7bb200640 1 -- 172.21.15.123:0/2390392799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 msgr2=0x7fa7b415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7bb200640 1 --2- 172.21.15.123:0/2390392799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b415e050 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7b415a500 tx=0x7fa7a002a180 comp rx=0 tx=0).stop 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7bb200640 1 -- 172.21.15.123:0/2390392799 shutdown_connections 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7bb200640 1 --2- 172.21.15.123:0/2390392799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b415e050 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7bb200640 1 --2- 172.21.15.123:0/2390392799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b40c9080 0x7fa7b41546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.050+0000 7fa7bb200640 1 -- 172.21.15.123:0/2390392799 >> 172.21.15.123:0/2390392799 conn(0x7fa7b40cd600 msgr2=0x7fa7b40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.051+0000 7fa7bb200640 1 -- 172.21.15.123:0/2390392799 shutdown_connections 2024-04-25T12:34:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.051+0000 7fa7bb200640 1 -- 172.21.15.123:0/2390392799 wait complete. 2024-04-25T12:34:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.053+0000 7fa7bb200640 1 Processor -- start 2024-04-25T12:34:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.053+0000 7fa7bb200640 1 -- start start 2024-04-25T12:34:43.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.054+0000 7fa7bb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b4151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.054+0000 7fa7bb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40c9080 0x7fa7b41f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7bb200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e590 0x7fa7b415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7bb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4073760 con 0x7fa7b40c82b0 2024-04-25T12:34:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7bb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b415db30 con 0x7fa7b40c9080 2024-04-25T12:34:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7bb200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b41f3430 con 0x7fa7b415e590 2024-04-25T12:34:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b4151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b4151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35192/0 (socket says 172.21.15.123:35192) 2024-04-25T12:34:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 -- 172.21.15.123:0/1705673976 learned_addr learned my addr 172.21.15.123:0/1705673976 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7ba800640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e590 0x7fa7b415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9400640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40c9080 0x7fa7b41f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 -- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e590 msgr2=0x7fa7b415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e590 0x7fa7b415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 -- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40c9080 msgr2=0x7fa7b41f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40c9080 0x7fa7b41f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.055+0000 7fa7b9e00640 1 -- 172.21.15.123:0/1705673976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40b2080 con 0x7fa7b40c82b0 2024-04-25T12:34:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.056+0000 7fa7b9e00640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b4151da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800e990 tx=0x7fa7a800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.056+0000 7fa7ae000640 1 -- 172.21.15.123:0/1705673976 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8017070 con 0x7fa7b40c82b0 2024-04-25T12:34:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.056+0000 7fa7ae000640 1 -- 172.21.15.123:0/1705673976 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a800c9e0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.056+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b41f2f60 con 0x7fa7b40c82b0 2024-04-25T12:34:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.056+0000 7fa7ae000640 1 -- 172.21.15.123:0/1705673976 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800cdb0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.056+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b40b17e0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.058+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b40cafe0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.059+0000 7fa7ae000640 1 -- 172.21.15.123:0/1705673976 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa7a801a070 con 0x7fa7b40c82b0 2024-04-25T12:34:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.060+0000 7fa7ae000640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7940744b0 0x7fa7940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.060+0000 7fa7ae000640 1 -- 172.21.15.123:0/1705673976 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa7a80953f0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.060+0000 7fa7b9400640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7940744b0 0x7fa7940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.061+0000 7fa7b9400640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7940744b0 0x7fa7940769c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0034000 tx=0x7fa7a002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:43.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.065+0000 7fa7ae000640 1 -- 172.21.15.123:0/1705673976 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7b40cafe0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.266+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7b41f2660 con 0x7fa7b40c82b0 2024-04-25T12:34:43.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.270+0000 7fa7ae000640 1 -- 172.21.15.123:0/1705673976 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fa7a805f1e0 con 0x7fa7b40c82b0 2024-04-25T12:34:43.270 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:43.270 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.272+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7940744b0 msgr2=0x7fa7940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.272+0000 7fa7bb200640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7940744b0 0x7fa7940769c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0034000 tx=0x7fa7a002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 msgr2=0x7fa7b4151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b4151da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800e990 tx=0x7fa7a800ee50 comp rx=0 tx=0).stop 2024-04-25T12:34:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 shutdown_connections 2024-04-25T12:34:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7940744b0 0x7fa7940769c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e590 0x7fa7b415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40c9080 0x7fa7b41f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 --2- 172.21.15.123:0/1705673976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b40c82b0 0x7fa7b4151da0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 >> 172.21.15.123:0/1705673976 conn(0x7fa7b40cd600 msgr2=0x7fa7b40c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:43.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 shutdown_connections 2024-04-25T12:34:43.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.273+0000 7fa7bb200640 1 -- 172.21.15.123:0/1705673976 wait complete. 2024-04-25T12:34:43.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T12:34:43.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.568+0000 7f3c23800640 1 Processor -- start 2024-04-25T12:34:43.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.568+0000 7f3c23800640 1 -- start start 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.570+0000 7f3c23800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3c1c0730e0 0x7f3c1c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.570+0000 7f3c23800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c074b20 con 0x7f3c1c154650 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.570+0000 7f3c23800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c0c8880 con 0x7f3c1c0c9f00 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.570+0000 7f3c23800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1c0ca780 con 0x7f3c1c0730e0 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c20e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3c1c154650 0x7f3c1c15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55946/0 (socket says 172.21.15.123:55946) 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c20e00640 1 -- 172.21.15.123:0/3269846817 learned_addr learned my addr 172.21.15.123:0/3269846817 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c21800640 1 --2- 172.21.15.123:0/3269846817 >> v2:172.21.15.153:3300/0 conn(0x7f3c1c0730e0 0x7f3c1c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c1b400640 1 -- 172.21.15.123:0/3269846817 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289931016 0 0) 0x7f3c1c074b20 con 0x7f3c1c154650 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c1b400640 1 -- 172.21.15.123:0/3269846817 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c04003520 con 0x7f3c1c154650 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c1b400640 1 -- 172.21.15.123:0/3269846817 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 446043226 0 0) 0x7f3c1c0c8880 con 0x7f3c1c0c9f00 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c1b400640 1 -- 172.21.15.123:0/3269846817 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c1c074b20 con 0x7f3c1c0c9f00 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c21800640 1 -- 172.21.15.123:0/3269846817 >> v1:172.21.15.153:6789/0 conn(0x7f3c1c0c9f00 legacy=0x7f3c1c0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c21800640 1 -- 172.21.15.123:0/3269846817 >> v1:172.21.15.123:6789/0 conn(0x7f3c1c154650 legacy=0x7f3c1c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.571+0000 7f3c21800640 1 -- 172.21.15.123:0/3269846817 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c0caa60 con 0x7f3c1c0730e0 2024-04-25T12:34:43.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.572+0000 7f3c21800640 1 --2- 172.21.15.123:0/3269846817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c0730e0 0x7f3c1c0734f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009d90 tx=0x7f3c1402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8694717bdcfd55c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:43.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.573+0000 7f3c1b400640 1 -- 172.21.15.123:0/3269846817 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14007740 con 0x7f3c1c0730e0 2024-04-25T12:34:43.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.573+0000 7f3c1b400640 1 -- 172.21.15.123:0/3269846817 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c140078e0 con 0x7f3c1c0730e0 2024-04-25T12:34:43.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.573+0000 7f3c1b400640 1 -- 172.21.15.123:0/3269846817 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14007c90 con 0x7f3c1c0730e0 2024-04-25T12:34:43.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.573+0000 7f3c23800640 1 -- 172.21.15.123:0/3269846817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c0730e0 msgr2=0x7f3c1c0734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.573+0000 7f3c23800640 1 --2- 172.21.15.123:0/3269846817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c0730e0 0x7f3c1c0734f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009d90 tx=0x7f3c1402a6f0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.574+0000 7f3c23800640 1 -- 172.21.15.123:0/3269846817 shutdown_connections 2024-04-25T12:34:43.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.574+0000 7f3c23800640 1 --2- 172.21.15.123:0/3269846817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c0730e0 0x7f3c1c0734f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.574+0000 7f3c23800640 1 -- 172.21.15.123:0/3269846817 >> 172.21.15.123:0/3269846817 conn(0x7f3c1c06f440 msgr2=0x7f3c1c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:43.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.574+0000 7f3c23800640 1 -- 172.21.15.123:0/3269846817 shutdown_connections 2024-04-25T12:34:43.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.574+0000 7f3c23800640 1 -- 172.21.15.123:0/3269846817 wait complete. 2024-04-25T12:34:43.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.576+0000 7f3c23800640 1 Processor -- start 2024-04-25T12:34:43.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.576+0000 7f3c23800640 1 -- start start 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c23800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c1c0730e0 0x7f3c1c0c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c23800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c1c0c9f00 0x7f3c1c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c23800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c154650 0x7f3c1c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c23800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c15e010 con 0x7f3c1c0c9f00 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c23800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c15dd30 con 0x7f3c1c154650 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c23800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c15e2f0 con 0x7f3c1c0730e0 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c1c0c9f00 0x7f3c1c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c20e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c1c0730e0 0x7f3c1c0c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c21800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c154650 0x7f3c1c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c21800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c154650 0x7f3c1c0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55168/0 (socket says 172.21.15.123:55168) 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c1c0c9f00 0x7f3c1c0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35194/0 (socket says 172.21.15.123:35194) 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.578+0000 7f3c21800640 1 -- 172.21.15.123:0/759369590 learned_addr learned my addr 172.21.15.123:0/759369590 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c21800640 1 -- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c1c0730e0 msgr2=0x7f3c1c0c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c21800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c1c0730e0 0x7f3c1c0c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c21800640 1 -- 172.21.15.123:0/759369590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c1c0c9f00 msgr2=0x7f3c1c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c21800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c1c0c9f00 0x7f3c1c0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c21800640 1 -- 172.21.15.123:0/759369590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c1f3300 con 0x7f3c1c154650 2024-04-25T12:34:43.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c21800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c154650 0x7f3c1c0b28c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009e80 tx=0x7f3c14004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:43.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c18c00640 1 -- 172.21.15.123:0/759369590 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1403f070 con 0x7f3c1c154650 2024-04-25T12:34:43.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c18c00640 1 -- 172.21.15.123:0/759369590 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c14031bf0 con 0x7f3c1c154650 2024-04-25T12:34:43.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c15b580 con 0x7f3c1c154650 2024-04-25T12:34:43.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c18c00640 1 -- 172.21.15.123:0/759369590 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14009450 con 0x7f3c1c154650 2024-04-25T12:34:43.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.579+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1f2820 con 0x7f3c1c154650 2024-04-25T12:34:43.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.580+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c14eab0 con 0x7f3c1c154650 2024-04-25T12:34:43.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.582+0000 7f3c18c00640 1 -- 172.21.15.123:0/759369590 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c14043410 con 0x7f3c1c154650 2024-04-25T12:34:43.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.583+0000 7f3c18c00640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3bfc0744b0 0x7f3bfc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:43.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.583+0000 7f3c18c00640 1 -- 172.21.15.123:0/759369590 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3c140b6830 con 0x7f3c1c154650 2024-04-25T12:34:43.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.583+0000 7f3c20e00640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3bfc0744b0 0x7f3bfc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:43.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.584+0000 7f3c20e00640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3bfc0744b0 0x7f3bfc0769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000f60 tx=0x7f3c10007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:43.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.589+0000 7f3c18c00640 1 -- 172.21.15.123:0/759369590 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c14080590 con 0x7f3c1c154650 2024-04-25T12:34:43.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:43.830+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3c1c076700 con 0x7f3c1c154650 2024-04-25T12:34:44.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.086+0000 7f3c18c00640 1 -- 172.21.15.123:0/759369590 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f3c14084890 con 0x7f3c1c154650 2024-04-25T12:34:44.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.090+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3bfc0744b0 msgr2=0x7f3bfc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:44.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.090+0000 7f3c23800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3bfc0744b0 0x7f3bfc0769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000f60 tx=0x7f3c10007d50 comp rx=0 tx=0).stop 2024-04-25T12:34:44.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.090+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c154650 msgr2=0x7f3c1c0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:44.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.090+0000 7f3c23800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c154650 0x7f3c1c0b28c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009e80 tx=0x7f3c14004a40 comp rx=0 tx=0).stop 2024-04-25T12:34:44.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 shutdown_connections 2024-04-25T12:34:44.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3bfc0744b0 0x7f3bfc0769c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c154650 0x7f3c1c0b28c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c1c0c9f00 0x7f3c1c0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 --2- 172.21.15.123:0/759369590 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3c1c0730e0 0x7f3c1c0c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 >> 172.21.15.123:0/759369590 conn(0x7f3c1c06f440 msgr2=0x7f3c1c15c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 shutdown_connections 2024-04-25T12:34:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.091+0000 7f3c23800640 1 -- 172.21.15.123:0/759369590 wait complete. 2024-04-25T12:34:44.124 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300} 2024-04-25T12:34:44.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T12:34:44.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.403+0000 7f393fe00640 1 Processor -- start 2024-04-25T12:34:44.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.403+0000 7f393fe00640 1 -- start start 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f39400b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3940156800 0x7f39400c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393fe00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f394015c760 con 0x7f39400cdd60 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f394015ca40 con 0x7f3940156800 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393fe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f394015cd20 con 0x7f39400c8c10 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393ea00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f39400cdd60 0x7f394006d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:41450/0 (socket says 172.21.15.123:41450) 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393ea00640 1 -- 172.21.15.123:0/3729823849 learned_addr learned my addr 172.21.15.123:0/3729823849 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.405+0000 7f393e000640 1 --2- 172.21.15.123:0/3729823849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f39400b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:44.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.406+0000 7f393f400640 1 --2- 172.21.15.123:0/3729823849 >> v2:172.21.15.123:3300/0 conn(0x7f3940156800 0x7f39400c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:44.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.406+0000 7f393e000640 1 -- 172.21.15.123:0/3729823849 >> v2:172.21.15.123:3300/0 conn(0x7f3940156800 msgr2=0x7f39400c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:44.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.406+0000 7f393e000640 1 --2- 172.21.15.123:0/3729823849 >> v2:172.21.15.123:3300/0 conn(0x7f3940156800 0x7f39400c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.406+0000 7f393e000640 1 -- 172.21.15.123:0/3729823849 >> v1:172.21.15.153:6789/0 conn(0x7f39400cdd60 legacy=0x7f394006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:44.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.406+0000 7f393e000640 1 -- 172.21.15.123:0/3729823849 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394015d000 con 0x7f39400c8c10 2024-04-25T12:34:44.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.406+0000 7f393d600640 1 -- 172.21.15.123:0/3729823849 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2135905920 0 0) 0x7f394015c760 con 0x7f39400cdd60 2024-04-25T12:34:44.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.407+0000 7f393e000640 1 --2- 172.21.15.123:0/3729823849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f39400b7300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f392c0097a0 tx=0x7f392c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a773d1a4abca65f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.407+0000 7f393d600640 1 -- 172.21.15.123:0/3729823849 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c037070 con 0x7f39400c8c10 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.407+0000 7f393d600640 1 -- 172.21.15.123:0/3729823849 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392c032440 con 0x7f39400c8c10 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.407+0000 7f393d600640 1 -- 172.21.15.123:0/3729823849 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c032810 con 0x7f39400c8c10 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 -- 172.21.15.123:0/3729823849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 msgr2=0x7f39400b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 --2- 172.21.15.123:0/3729823849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f39400b7300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f392c0097a0 tx=0x7f392c02a180 comp rx=0 tx=0).stop 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 -- 172.21.15.123:0/3729823849 shutdown_connections 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 --2- 172.21.15.123:0/3729823849 >> v2:172.21.15.123:3300/0 conn(0x7f3940156800 0x7f39400c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 --2- 172.21.15.123:0/3729823849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f39400b7300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 -- 172.21.15.123:0/3729823849 >> 172.21.15.123:0/3729823849 conn(0x7f39400d1810 msgr2=0x7f39400d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 -- 172.21.15.123:0/3729823849 shutdown_connections 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.408+0000 7f393fe00640 1 -- 172.21.15.123:0/3729823849 wait complete. 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.411+0000 7f393fe00640 1 Processor -- start 2024-04-25T12:34:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.411+0000 7f393fe00640 1 -- start start 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.412+0000 7f393fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f394015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.412+0000 7f393fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39400cdd60 0x7f39401e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.412+0000 7f393fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3940156800 0x7f39401e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.412+0000 7f393fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f394006e0c0 con 0x7f39400cdd60 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.412+0000 7f393fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3940156c10 con 0x7f3940156800 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.412+0000 7f393fe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f394006e240 con 0x7f39400c8c10 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f394015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3940156800 0x7f39401e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f394015a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35802/0 (socket says 172.21.15.123:35802) 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 -- 172.21.15.123:0/1905994813 learned_addr learned my addr 172.21.15.123:0/1905994813 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:44.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393e000640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39400cdd60 0x7f39401e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 -- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3940156800 msgr2=0x7f39401e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3940156800 0x7f39401e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 -- 172.21.15.123:0/1905994813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39400cdd60 msgr2=0x7f39401e5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39400cdd60 0x7f39401e5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393ea00640 1 -- 172.21.15.123:0/1905994813 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394015ab20 con 0x7f39400c8c10 2024-04-25T12:34:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.413+0000 7f393e000640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39400cdd60 0x7f39401e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.414+0000 7f393ea00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f394015a5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f393000c410 tx=0x7f393000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.414+0000 7f3936a00640 1 -- 172.21.15.123:0/1905994813 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300074e0 con 0x7f39400c8c10 2024-04-25T12:34:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.414+0000 7f3936a00640 1 -- 172.21.15.123:0/1905994813 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3930007680 con 0x7f39400c8c10 2024-04-25T12:34:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.414+0000 7f3936a00640 1 -- 172.21.15.123:0/1905994813 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930004a70 con 0x7f39400c8c10 2024-04-25T12:34:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.414+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3940153e20 con 0x7f39400c8c10 2024-04-25T12:34:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.414+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39401543c0 con 0x7f39400c8c10 2024-04-25T12:34:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.415+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3940076da0 con 0x7f39400c8c10 2024-04-25T12:34:44.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.417+0000 7f3936a00640 1 -- 172.21.15.123:0/1905994813 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3930011070 con 0x7f39400c8c10 2024-04-25T12:34:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.418+0000 7f3936a00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39200744b0 0x7f39200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.418+0000 7f3936a00640 1 -- 172.21.15.123:0/1905994813 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f39300950f0 con 0x7f39400c8c10 2024-04-25T12:34:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.418+0000 7f393e000640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39200744b0 0x7f39200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:44.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.419+0000 7f393e000640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39200744b0 0x7f39200769c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f392c034000 tx=0x7f392c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:44.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.422+0000 7f3936a00640 1 -- 172.21.15.123:0/1905994813 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f393005eed0 con 0x7f39400c8c10 2024-04-25T12:34:44.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:44.660+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f3940154a10 con 0x7f39400c8c10 2024-04-25T12:34:45.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.105+0000 7f3936a00640 1 -- 172.21.15.123:0/1905994813 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v8) ==== 105+0+0 (secure 0 0 0) 0x7f39300631d0 con 0x7f39400c8c10 2024-04-25T12:34:45.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39200744b0 msgr2=0x7f39200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39200744b0 0x7f39200769c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f392c034000 tx=0x7f392c02e000 comp rx=0 tx=0).stop 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 msgr2=0x7f394015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f394015a5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f393000c410 tx=0x7f393000c8d0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 shutdown_connections 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39200744b0 0x7f39200769c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3940156800 0x7f39401e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39400cdd60 0x7f39401e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 --2- 172.21.15.123:0/1905994813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39400c8c10 0x7f394015a5e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 >> 172.21.15.123:0/1905994813 conn(0x7f39400d1810 msgr2=0x7f39400c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 shutdown_connections 2024-04-25T12:34:45.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.110+0000 7f393fe00640 1 -- 172.21.15.123:0/1905994813 wait complete. 2024-04-25T12:34:45.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T12:34:45.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.415+0000 7fd24aa00640 1 Processor -- start 2024-04-25T12:34:45.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.416+0000 7fd24aa00640 1 -- start start 2024-04-25T12:34:45.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd24aa00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd2440ca230 0x7fd2440c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:45.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd24aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd24415da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:45.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd24aa00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2440ccd10 con 0x7fd2440c5a60 2024-04-25T12:34:45.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd24aa00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440ca990 con 0x7fd2440ca230 2024-04-25T12:34:45.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd24aa00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd2440cbc70 con 0x7fd2440c4bd0 2024-04-25T12:34:45.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd243e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd2440c5a60 0x7fd244071a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55960/0 (socket says 172.21.15.123:55960) 2024-04-25T12:34:45.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd243e00640 1 -- 172.21.15.123:0/3253849031 learned_addr learned my addr 172.21.15.123:0/3253849031 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd248a00640 1 --2- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd24415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.418+0000 7fd243400640 1 --2- 172.21.15.123:0/3253849031 >> v2:172.21.15.153:3300/0 conn(0x7fd2440ca230 0x7fd2440c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.419+0000 7fd242a00640 1 -- 172.21.15.123:0/3253849031 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740297185 0 0) 0x7fd2440ccd10 con 0x7fd2440c5a60 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.419+0000 7fd242a00640 1 -- 172.21.15.123:0/3253849031 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd224003520 con 0x7fd2440c5a60 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.419+0000 7fd243400640 1 -- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 msgr2=0x7fd24415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.419+0000 7fd243400640 1 --2- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd24415da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.419+0000 7fd243400640 1 -- 172.21.15.123:0/3253849031 >> v1:172.21.15.123:6789/0 conn(0x7fd2440c5a60 legacy=0x7fd244071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.419+0000 7fd243400640 1 -- 172.21.15.123:0/3253849031 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2440cbf50 con 0x7fd2440ca230 2024-04-25T12:34:45.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.419+0000 7fd248a00640 1 --2- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd24415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:45.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.420+0000 7fd243400640 1 --2- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440ca230 0x7fd2440c4690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0097a0 tx=0x7fd22c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b590bee18d87591d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:45.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.420+0000 7fd242a00640 1 -- 172.21.15.123:0/3253849031 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c037070 con 0x7fd2440ca230 2024-04-25T12:34:45.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.420+0000 7fd242a00640 1 -- 172.21.15.123:0/3253849031 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22c032440 con 0x7fd2440ca230 2024-04-25T12:34:45.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.420+0000 7fd242a00640 1 -- 172.21.15.123:0/3253849031 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c032810 con 0x7fd2440ca230 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 -- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440ca230 msgr2=0x7fd2440c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 --2- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440ca230 0x7fd2440c4690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0097a0 tx=0x7fd22c02a180 comp rx=0 tx=0).stop 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 -- 172.21.15.123:0/3253849031 shutdown_connections 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 --2- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd24415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 --2- 172.21.15.123:0/3253849031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440ca230 0x7fd2440c4690 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 -- 172.21.15.123:0/3253849031 >> 172.21.15.123:0/3253849031 conn(0x7fd244072d70 msgr2=0x7fd244073180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 -- 172.21.15.123:0/3253849031 shutdown_connections 2024-04-25T12:34:45.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.421+0000 7fd24aa00640 1 -- 172.21.15.123:0/3253849031 wait complete. 2024-04-25T12:34:45.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.424+0000 7fd24aa00640 1 Processor -- start 2024-04-25T12:34:45.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.424+0000 7fd24aa00640 1 -- start start 2024-04-25T12:34:45.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.425+0000 7fd24aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd244158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:45.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.425+0000 7fd24aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440c5a60 0x7fd244158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.425+0000 7fd24aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2440ca230 0x7fd244157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.425+0000 7fd24aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2440cc7f0 con 0x7fd2440ca230 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.425+0000 7fd24aa00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2440cc510 con 0x7fd2440c5a60 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.425+0000 7fd24aa00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd24415e780 con 0x7fd2440c4bd0 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.425+0000 7fd248a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2440ca230 0x7fd244157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2440ca230 0x7fd244157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35208/0 (socket says 172.21.15.123:35208) 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 -- 172.21.15.123:0/2905688207 learned_addr learned my addr 172.21.15.123:0/2905688207 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd243400640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440c5a60 0x7fd244158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd243e00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd244158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 -- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 msgr2=0x7fd244158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd244158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 -- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440c5a60 msgr2=0x7fd244158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:45.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440c5a60 0x7fd244158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:45.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 -- 172.21.15.123:0/2905688207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd244158880 con 0x7fd2440ca230 2024-04-25T12:34:45.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.426+0000 7fd248a00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2440ca230 0x7fd244157270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009e00 tx=0x7fd23c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.427+0000 7fd23be00640 1 -- 172.21.15.123:0/2905688207 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c018030 con 0x7fd2440ca230 2024-04-25T12:34:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.427+0000 7fd23be00640 1 -- 172.21.15.123:0/2905688207 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c00cdf0 con 0x7fd2440ca230 2024-04-25T12:34:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.427+0000 7fd23be00640 1 -- 172.21.15.123:0/2905688207 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003bb0 con 0x7fd2440ca230 2024-04-25T12:34:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.427+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd244159360 con 0x7fd2440ca230 2024-04-25T12:34:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.427+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441f29e0 con 0x7fd2440ca230 2024-04-25T12:34:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.428+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd244076700 con 0x7fd2440ca230 2024-04-25T12:34:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.429+0000 7fd23be00640 1 -- 172.21.15.123:0/2905688207 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd23c005d10 con 0x7fd2440ca230 2024-04-25T12:34:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.430+0000 7fd23be00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2200744b0 0x7fd2200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:45.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.430+0000 7fd23be00640 1 -- 172.21.15.123:0/2905688207 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd23c095e10 con 0x7fd2440ca230 2024-04-25T12:34:45.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.430+0000 7fd243e00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2200744b0 0x7fd2200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:45.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.431+0000 7fd243e00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2200744b0 0x7fd2200769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd234005880 tx=0x7fd234009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:45.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.436+0000 7fd23be00640 1 -- 172.21.15.123:0/2905688207 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd23c099030 con 0x7fd2440ca230 2024-04-25T12:34:45.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:45.679+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd2440cb5e0 con 0x7fd2440ca230 2024-04-25T12:34:46.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.111+0000 7fd23be00640 1 -- 172.21.15.123:0/2905688207 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fd23c022070 con 0x7fd2440ca230 2024-04-25T12:34:46.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.114+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2200744b0 msgr2=0x7fd2200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:46.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.114+0000 7fd24aa00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2200744b0 0x7fd2200769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd234005880 tx=0x7fd234009040 comp rx=0 tx=0).stop 2024-04-25T12:34:46.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.114+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2440ca230 msgr2=0x7fd244157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:46.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.114+0000 7fd24aa00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2440ca230 0x7fd244157270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009e00 tx=0x7fd23c00c490 comp rx=0 tx=0).stop 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 shutdown_connections 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd2200744b0 0x7fd2200769c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2440ca230 0x7fd244157270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2440c5a60 0x7fd244158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 --2- 172.21.15.123:0/2905688207 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2440c4bd0 0x7fd244158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 >> 172.21.15.123:0/2905688207 conn(0x7fd244072d70 msgr2=0x7fd24406ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 shutdown_connections 2024-04-25T12:34:46.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.115+0000 7fd24aa00640 1 -- 172.21.15.123:0/2905688207 wait complete. 2024-04-25T12:34:46.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T12:34:46.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.413+0000 7f1995800640 1 Processor -- start 2024-04-25T12:34:46.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.413+0000 7f1995800640 1 -- start start 2024-04-25T12:34:46.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1995800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19900b34f0 0x7f19901528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1995800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f199006c7b0 0x7f19900c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1995800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f19900c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1995800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f19900c4060 con 0x7f199006c7b0 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1995800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f19901589d0 con 0x7f19900b34f0 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1995800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1990158c70 con 0x7f199006cbc0 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19900b34f0 0x7f19901528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1987400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f199006c7b0 0x7f19900c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19900b34f0 0x7f19901528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55190/0 (socket says 172.21.15.123:55190) 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 -- 172.21.15.123:0/2848555400 learned_addr learned my addr 172.21.15.123:0/2848555400 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f1994e00640 1 --2- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f19900c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 -- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 msgr2=0x7f19900c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 --2- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f19900c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 -- 172.21.15.123:0/2848555400 >> v2:172.21.15.123:3300/0 conn(0x7f199006c7b0 msgr2=0x7f19900c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 --2- 172.21.15.123:0/2848555400 >> v2:172.21.15.123:3300/0 conn(0x7f199006c7b0 0x7f19900c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.416+0000 7f198fe00640 1 -- 172.21.15.123:0/2848555400 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990158f50 con 0x7f19900b34f0 2024-04-25T12:34:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.417+0000 7f198fe00640 1 --2- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19900b34f0 0x7f19901528b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f199006d630 tx=0x7f198002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6bf413151bfdf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.418+0000 7f198f400640 1 -- 172.21.15.123:0/2848555400 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980037070 con 0x7f19900b34f0 2024-04-25T12:34:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.418+0000 7f198f400640 1 -- 172.21.15.123:0/2848555400 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1980032440 con 0x7f19900b34f0 2024-04-25T12:34:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.418+0000 7f198f400640 1 -- 172.21.15.123:0/2848555400 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980032810 con 0x7f19900b34f0 2024-04-25T12:34:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 -- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19900b34f0 msgr2=0x7f19901528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 --2- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19900b34f0 0x7f19901528b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f199006d630 tx=0x7f198002a270 comp rx=0 tx=0).stop 2024-04-25T12:34:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 -- 172.21.15.123:0/2848555400 shutdown_connections 2024-04-25T12:34:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 --2- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f19900c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 --2- 172.21.15.123:0/2848555400 >> v2:172.21.15.123:3300/0 conn(0x7f199006c7b0 0x7f19900c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 --2- 172.21.15.123:0/2848555400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19900b34f0 0x7f19901528b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 -- 172.21.15.123:0/2848555400 >> 172.21.15.123:0/2848555400 conn(0x7f1990070b70 msgr2=0x7f1990070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 -- 172.21.15.123:0/2848555400 shutdown_connections 2024-04-25T12:34:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.419+0000 7f1995800640 1 -- 172.21.15.123:0/2848555400 wait complete. 2024-04-25T12:34:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.422+0000 7f1995800640 1 Processor -- start 2024-04-25T12:34:46.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.422+0000 7f1995800640 1 -- start start 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.423+0000 7f1995800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f199006c7b0 0x7f19900ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1995800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f1990147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1995800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19900b34f0 0x7f1990157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1995800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19900c86d0 con 0x7f19900b34f0 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1995800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1990159510 con 0x7f199006c7b0 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1995800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f19900c8850 con 0x7f199006cbc0 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19900b34f0 0x7f1990157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f198fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f199006c7b0 0x7f19900ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19900b34f0 0x7f1990157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35234/0 (socket says 172.21.15.123:35234) 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 -- 172.21.15.123:0/491617509 learned_addr learned my addr 172.21.15.123:0/491617509 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1987400640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f1990147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 -- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 msgr2=0x7f1990147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f1990147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 -- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f199006c7b0 msgr2=0x7f19900ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f199006c7b0 0x7f19900ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f1994e00640 1 -- 172.21.15.123:0/491617509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19901f59f0 con 0x7f19900b34f0 2024-04-25T12:34:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.424+0000 7f198fe00640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f199006c7b0 0x7f19900ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.425+0000 7f1994e00640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19900b34f0 0x7f1990157d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f198800ef90 tx=0x7f198800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.425+0000 7f198cc00640 1 -- 172.21.15.123:0/491617509 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988018020 con 0x7f19900b34f0 2024-04-25T12:34:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.425+0000 7f198cc00640 1 -- 172.21.15.123:0/491617509 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1988014480 con 0x7f19900b34f0 2024-04-25T12:34:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.425+0000 7f198cc00640 1 -- 172.21.15.123:0/491617509 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988009e60 con 0x7f19900b34f0 2024-04-25T12:34:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.425+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19901f5c90 con 0x7f19900b34f0 2024-04-25T12:34:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.425+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901e2810 con 0x7f19900b34f0 2024-04-25T12:34:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.426+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19901e16a0 con 0x7f19900b34f0 2024-04-25T12:34:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.427+0000 7f198cc00640 1 -- 172.21.15.123:0/491617509 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1988005d10 con 0x7f19900b34f0 2024-04-25T12:34:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.429+0000 7f198cc00640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f196c0744b0 0x7f196c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.429+0000 7f198cc00640 1 -- 172.21.15.123:0/491617509 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1988096230 con 0x7f19900b34f0 2024-04-25T12:34:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.429+0000 7f198fe00640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f196c0744b0 0x7f196c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:46.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.430+0000 7f198fe00640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f196c0744b0 0x7f196c0769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1980034000 tx=0x7f198002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:46.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.434+0000 7f198cc00640 1 -- 172.21.15.123:0/491617509 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19901e16a0 con 0x7f19900b34f0 2024-04-25T12:34:46.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:46.659+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f19901e1950 con 0x7f19900b34f0 2024-04-25T12:34:47.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.121+0000 7f198cc00640 1 -- 172.21.15.123:0/491617509 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v11) ==== 115+0+0 (secure 0 0 0) 0x7f1988022030 con 0x7f19900b34f0 2024-04-25T12:34:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.124+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f196c0744b0 msgr2=0x7f196c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.124+0000 7f1995800640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f196c0744b0 0x7f196c0769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1980034000 tx=0x7f198002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.124+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19900b34f0 msgr2=0x7f1990157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.124+0000 7f1995800640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19900b34f0 0x7f1990157d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f198800ef90 tx=0x7f198800c490 comp rx=0 tx=0).stop 2024-04-25T12:34:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 shutdown_connections 2024-04-25T12:34:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f196c0744b0 0x7f196c0769c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19900b34f0 0x7f1990157d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f199006cbc0 0x7f1990147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 --2- 172.21.15.123:0/491617509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f199006c7b0 0x7f19900ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 >> 172.21.15.123:0/491617509 conn(0x7f1990070b70 msgr2=0x7f19900c2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 shutdown_connections 2024-04-25T12:34:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.125+0000 7f1995800640 1 -- 172.21.15.123:0/491617509 wait complete. 2024-04-25T12:34:47.160 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T12:34:47.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:34:47.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.430+0000 7fc4d9600640 1 Processor -- start 2024-04-25T12:34:47.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.430+0000 7fc4d9600640 1 -- start start 2024-04-25T12:34:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d9600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc4d4154890 0x7fc4d415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d9600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d4154ca0 con 0x7fc4d40c4d50 2024-04-25T12:34:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d9600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d4073ce0 con 0x7fc4d4074870 2024-04-25T12:34:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d9600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d40c61f0 con 0x7fc4d4154890 2024-04-25T12:34:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d2a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc4d40c4d50 0x7fc4d40c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55970/0 (socket says 172.21.15.123:55970) 2024-04-25T12:34:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d2a00640 1 -- 172.21.15.123:0/761940865 learned_addr learned my addr 172.21.15.123:0/761940865 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d3400640 1 --2- 172.21.15.123:0/761940865 >> v2:172.21.15.153:3300/0 conn(0x7fc4d4154890 0x7fc4d415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d2000640 1 -- 172.21.15.123:0/761940865 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936871307 0 0) 0x7fc4d4154ca0 con 0x7fc4d40c4d50 2024-04-25T12:34:47.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d2000640 1 -- 172.21.15.123:0/761940865 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4b8003520 con 0x7fc4d40c4d50 2024-04-25T12:34:47.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d2000640 1 -- 172.21.15.123:0/761940865 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3135800793 0 0) 0x7fc4d4073ce0 con 0x7fc4d4074870 2024-04-25T12:34:47.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.433+0000 7fc4d2000640 1 -- 172.21.15.123:0/761940865 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4d4154ca0 con 0x7fc4d4074870 2024-04-25T12:34:47.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.434+0000 7fc4d3400640 1 -- 172.21.15.123:0/761940865 >> v1:172.21.15.153:6789/0 conn(0x7fc4d4074870 legacy=0x7fc4d40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.434+0000 7fc4d3400640 1 -- 172.21.15.123:0/761940865 >> v1:172.21.15.123:6789/0 conn(0x7fc4d40c4d50 legacy=0x7fc4d40c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.434+0000 7fc4d3400640 1 -- 172.21.15.123:0/761940865 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4073fc0 con 0x7fc4d4154890 2024-04-25T12:34:47.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.435+0000 7fc4d3400640 1 --2- 172.21.15.123:0/761940865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 0x7fc4d415afa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009d90 tx=0x7fc4cc02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703e63325ecd3f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:47.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.435+0000 7fc4d2000640 1 -- 172.21.15.123:0/761940865 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc037070 con 0x7fc4d4154890 2024-04-25T12:34:47.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.435+0000 7fc4d2000640 1 -- 172.21.15.123:0/761940865 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc007740 con 0x7fc4d4154890 2024-04-25T12:34:47.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.435+0000 7fc4d2000640 1 -- 172.21.15.123:0/761940865 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc007af0 con 0x7fc4d4154890 2024-04-25T12:34:47.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.436+0000 7fc4d9600640 1 -- 172.21.15.123:0/761940865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 msgr2=0x7fc4d415afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.436+0000 7fc4d9600640 1 --2- 172.21.15.123:0/761940865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 0x7fc4d415afa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009d90 tx=0x7fc4cc02a780 comp rx=0 tx=0).stop 2024-04-25T12:34:47.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.436+0000 7fc4d9600640 1 -- 172.21.15.123:0/761940865 shutdown_connections 2024-04-25T12:34:47.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.436+0000 7fc4d9600640 1 --2- 172.21.15.123:0/761940865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 0x7fc4d415afa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.436+0000 7fc4d9600640 1 -- 172.21.15.123:0/761940865 >> 172.21.15.123:0/761940865 conn(0x7fc4d406c750 msgr2=0x7fc4d406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:47.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.436+0000 7fc4d9600640 1 -- 172.21.15.123:0/761940865 shutdown_connections 2024-04-25T12:34:47.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.436+0000 7fc4d9600640 1 -- 172.21.15.123:0/761940865 wait complete. 2024-04-25T12:34:47.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.438+0000 7fc4d9600640 1 Processor -- start 2024-04-25T12:34:47.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.438+0000 7fc4d9600640 1 -- start start 2024-04-25T12:34:47.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.439+0000 7fc4d9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4074870 0x7fc4d41f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d9600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c4d50 0x7fc4d41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 0x7fc4d414a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d9600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40c8460 con 0x7fc4d4074870 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d9600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40c87d0 con 0x7fc4d4154890 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d9600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40c9370 con 0x7fc4d40c4d50 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4074870 0x7fc4d41f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4074870 0x7fc4d41f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35246/0 (socket says 172.21.15.123:35246) 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 -- 172.21.15.123:0/2605784166 learned_addr learned my addr 172.21.15.123:0/2605784166 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d3400640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 0x7fc4d414a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4cbe00640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c4d50 0x7fc4d41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 -- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c4d50 msgr2=0x7fc4d41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c4d50 0x7fc4d41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 -- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 msgr2=0x7fc4d414a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 0x7fc4d414a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.440+0000 7fc4d2a00640 1 -- 172.21.15.123:0/2605784166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d41f13f0 con 0x7fc4d4074870 2024-04-25T12:34:47.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.441+0000 7fc4d2a00640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4074870 0x7fc4d41f0eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c410 tx=0x7fc4c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:47.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.441+0000 7fc4caa00640 1 -- 172.21.15.123:0/2605784166 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4017070 con 0x7fc4d4074870 2024-04-25T12:34:47.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.441+0000 7fc4caa00640 1 -- 172.21.15.123:0/2605784166 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c4005ce0 con 0x7fc4d4074870 2024-04-25T12:34:47.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.441+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d4154ec0 con 0x7fc4d4074870 2024-04-25T12:34:47.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.441+0000 7fc4caa00640 1 -- 172.21.15.123:0/2605784166 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40072d0 con 0x7fc4d4074870 2024-04-25T12:34:47.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.441+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d4155770 con 0x7fc4d4074870 2024-04-25T12:34:47.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.442+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d41f15c0 con 0x7fc4d4074870 2024-04-25T12:34:47.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.443+0000 7fc4caa00640 1 -- 172.21.15.123:0/2605784166 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc4c401a070 con 0x7fc4d4074870 2024-04-25T12:34:47.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.445+0000 7fc4caa00640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b00744e0 0x7fc4b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.445+0000 7fc4caa00640 1 -- 172.21.15.123:0/2605784166 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc4c4094880 con 0x7fc4d4074870 2024-04-25T12:34:47.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.445+0000 7fc4cbe00640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b00744e0 0x7fc4b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.446+0000 7fc4cbe00640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b00744e0 0x7fc4b00769f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40c78f0 tx=0x7fc4bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:47.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.450+0000 7fc4caa00640 1 -- 172.21.15.123:0/2605784166 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4c405e670 con 0x7fc4d4074870 2024-04-25T12:34:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.691+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4d40cb180 con 0x7fc4d4074870 2024-04-25T12:34:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.693+0000 7fc4caa00640 1 -- 172.21.15.123:0/2605784166 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6856 (secure 0 0 0) 0x7fc4d40cb180 con 0x7fc4d4074870 2024-04-25T12:34:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:47.693 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:46.661728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4270,"mds_1":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"f","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T12:34:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 11 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b00744e0 msgr2=0x7fc4b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b00744e0 0x7fc4b00769f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40c78f0 tx=0x7fc4bc009040 comp rx=0 tx=0).stop 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4074870 msgr2=0x7fc4d41f0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4074870 0x7fc4d41f0eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c410 tx=0x7fc4c400c8d0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 shutdown_connections 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b00744e0 0x7fc4b00769f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d4154890 0x7fc4d414a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c4d50 0x7fc4d41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 --2- 172.21.15.123:0/2605784166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4074870 0x7fc4d41f0eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 >> 172.21.15.123:0/2605784166 conn(0x7fc4d406c750 msgr2=0x7fc4d414c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 shutdown_connections 2024-04-25T12:34:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.695+0000 7fc4d9600640 1 -- 172.21.15.123:0/2605784166 wait complete. 2024-04-25T12:34:47.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:47.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.987+0000 7fd3cac00640 1 Processor -- start 2024-04-25T12:34:47.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.988+0000 7fd3cac00640 1 -- start start 2024-04-25T12:34:47.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3cac00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd3c40c4d50 0x7fd3c4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3cac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd3c4074870 0x7fd3c415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3cac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3cac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c40c61f0 con 0x7fd3c4074870 2024-04-25T12:34:47.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3cac00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c40c87d0 con 0x7fd3c40c4d50 2024-04-25T12:34:47.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3cac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c4073ce0 con 0x7fd3c40c8c60 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3c3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd3c4074870 0x7fd3c415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3c3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd3c4074870 0x7fd3c415afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35250/0 (socket says 172.21.15.123:35250) 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3c3400640 1 -- 172.21.15.123:0/1178223561 learned_addr learned my addr 172.21.15.123:0/1178223561 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/1178223561 >> v2:172.21.15.153:3300/0 conn(0x7fd3c40c4d50 0x7fd3c4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.990+0000 7fd3c8c00640 1 --2- 172.21.15.123:0/1178223561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.991+0000 7fd3c3400640 1 -- 172.21.15.123:0/1178223561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 msgr2=0x7fd3c40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.991+0000 7fd3c3400640 1 --2- 172.21.15.123:0/1178223561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.991+0000 7fd3c3400640 1 -- 172.21.15.123:0/1178223561 >> v2:172.21.15.153:3300/0 conn(0x7fd3c40c4d50 msgr2=0x7fd3c4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.991+0000 7fd3c3400640 1 --2- 172.21.15.123:0/1178223561 >> v2:172.21.15.153:3300/0 conn(0x7fd3c40c4d50 0x7fd3c4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.991+0000 7fd3c3400640 1 -- 172.21.15.123:0/1178223561 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c4073fc0 con 0x7fd3c4074870 2024-04-25T12:34:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.991+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/1178223561 >> v2:172.21.15.153:3300/0 conn(0x7fd3c40c4d50 0x7fd3c4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.992+0000 7fd3c3400640 1 --2- 172.21.15.123:0/1178223561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c415afa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40ce6d0 tx=0x7fd3b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83a3e440eb69c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.992+0000 7fd3c2a00640 1 -- 172.21.15.123:0/1178223561 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4037070 con 0x7fd3c4074870 2024-04-25T12:34:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.992+0000 7fd3c2a00640 1 -- 172.21.15.123:0/1178223561 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b4032440 con 0x7fd3c4074870 2024-04-25T12:34:47.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.992+0000 7fd3c2a00640 1 -- 172.21.15.123:0/1178223561 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4032810 con 0x7fd3c4074870 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 -- 172.21.15.123:0/1178223561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 msgr2=0x7fd3c415afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 --2- 172.21.15.123:0/1178223561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c415afa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40ce6d0 tx=0x7fd3b402a270 comp rx=0 tx=0).stop 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 -- 172.21.15.123:0/1178223561 shutdown_connections 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 --2- 172.21.15.123:0/1178223561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 --2- 172.21.15.123:0/1178223561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c415afa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 --2- 172.21.15.123:0/1178223561 >> v2:172.21.15.153:3300/0 conn(0x7fd3c40c4d50 0x7fd3c4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 -- 172.21.15.123:0/1178223561 >> 172.21.15.123:0/1178223561 conn(0x7fd3c406c750 msgr2=0x7fd3c406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 -- 172.21.15.123:0/1178223561 shutdown_connections 2024-04-25T12:34:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.993+0000 7fd3cac00640 1 -- 172.21.15.123:0/1178223561 wait complete. 2024-04-25T12:34:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.995+0000 7fd3cac00640 1 Processor -- start 2024-04-25T12:34:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.996+0000 7fd3cac00640 1 -- start start 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3cac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c41e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3cac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40c4d50 0x7fd3c40c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3cac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3cac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c40cda90 con 0x7fd3c4074870 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3cac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c40c8460 con 0x7fd3c40c4d50 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3cac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c40cdc10 con 0x7fd3c40c8c60 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c41e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c41e54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35252/0 (socket says 172.21.15.123:35252) 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3c3e00640 1 -- 172.21.15.123:0/199940518 learned_addr learned my addr 172.21.15.123:0/199940518 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3c3400640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40c4d50 0x7fd3c40c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.997+0000 7fd3c8c00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:47.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3c3e00640 1 -- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 msgr2=0x7fd3c40c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3c3e00640 1 -- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40c4d50 msgr2=0x7fd3c40c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:47.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40c4d50 0x7fd3c40c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:47.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3c3e00640 1 -- 172.21.15.123:0/199940518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c41e5a10 con 0x7fd3c4074870 2024-04-25T12:34:48.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c41e54d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00e960 tx=0x7fd3ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:48.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3bbe00640 1 -- 172.21.15.123:0/199940518 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac017070 con 0x7fd3c4074870 2024-04-25T12:34:48.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3bbe00640 1 -- 172.21.15.123:0/199940518 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac00ca00 con 0x7fd3c4074870 2024-04-25T12:34:48.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c414c7b0 con 0x7fd3c4074870 2024-04-25T12:34:48.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.998+0000 7fd3bbe00640 1 -- 172.21.15.123:0/199940518 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00cdd0 con 0x7fd3c4074870 2024-04-25T12:34:48.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:47.999+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c414b7a0 con 0x7fd3c4074870 2024-04-25T12:34:48.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.000+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c4076700 con 0x7fd3c4074870 2024-04-25T12:34:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.001+0000 7fd3bbe00640 1 -- 172.21.15.123:0/199940518 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3ac01a070 con 0x7fd3c4074870 2024-04-25T12:34:48.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.002+0000 7fd3bbe00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a00744e0 0x7fd3a00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.002+0000 7fd3bbe00640 1 -- 172.21.15.123:0/199940518 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3ac095480 con 0x7fd3c4074870 2024-04-25T12:34:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.002+0000 7fd3c3400640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a00744e0 0x7fd3a00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.003+0000 7fd3c3400640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a00744e0 0x7fd3a00769f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4034000 tx=0x7fd3b402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:48.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.006+0000 7fd3bbe00640 1 -- 172.21.15.123:0/199940518 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3ac05f1e0 con 0x7fd3c4074870 2024-04-25T12:34:48.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.210+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3c40c7510 con 0x7fd3c4074870 2024-04-25T12:34:48.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.213+0000 7fd3bbe00640 1 -- 172.21.15.123:0/199940518 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fd3ac0634e0 con 0x7fd3c4074870 2024-04-25T12:34:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:48.214 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.215+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a00744e0 msgr2=0x7fd3a00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.215+0000 7fd3cac00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a00744e0 0x7fd3a00769f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4034000 tx=0x7fd3b402e000 comp rx=0 tx=0).stop 2024-04-25T12:34:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.215+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 msgr2=0x7fd3c41e54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.215+0000 7fd3cac00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c41e54d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00e960 tx=0x7fd3ac00ee20 comp rx=0 tx=0).stop 2024-04-25T12:34:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 shutdown_connections 2024-04-25T12:34:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a00744e0 0x7fd3a00769f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40c8c60 0x7fd3c40c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40c4d50 0x7fd3c40c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 --2- 172.21.15.123:0/199940518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c4074870 0x7fd3c41e54d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 >> 172.21.15.123:0/199940518 conn(0x7fd3c406c750 msgr2=0x7fd3c4150490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 shutdown_connections 2024-04-25T12:34:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.216+0000 7fd3cac00640 1 -- 172.21.15.123:0/199940518 wait complete. 2024-04-25T12:34:48.241 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T12:34:48.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:34:48.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.499+0000 7f18ea800640 1 Processor -- start 2024-04-25T12:34:48.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.499+0000 7f18ea800640 1 -- start start 2024-04-25T12:34:48.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.501+0000 7f18ea800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18e406c570 0x7f18e406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:48.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.501+0000 7f18ea800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e4164390 con 0x7f18e415ac10 2024-04-25T12:34:48.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.501+0000 7f18ea800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e4164630 con 0x7f18e4163ec0 2024-04-25T12:34:48.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.501+0000 7f18ea800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f18e4164e70 con 0x7f18e406c570 2024-04-25T12:34:48.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.501+0000 7f18e9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18e406c570 0x7f18e406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:48.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.501+0000 7f18e8a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f18e415ac10 0x7f18e4161000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55972/0 (socket says 172.21.15.123:55972) 2024-04-25T12:34:48.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.501+0000 7f18e8a00640 1 -- 172.21.15.123:0/3981077677 learned_addr learned my addr 172.21.15.123:0/3981077677 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:48.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.502+0000 7f18e9e00640 1 -- 172.21.15.123:0/3981077677 >> v1:172.21.15.153:6789/0 conn(0x7f18e4163ec0 legacy=0x7f18e40c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.502+0000 7f18e9e00640 1 -- 172.21.15.123:0/3981077677 >> v1:172.21.15.123:6789/0 conn(0x7f18e415ac10 legacy=0x7f18e4161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.502+0000 7f18e3e00640 1 -- 172.21.15.123:0/3981077677 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247301926 0 0) 0x7f18e4164630 con 0x7f18e4163ec0 2024-04-25T12:34:48.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.502+0000 7f18e9e00640 1 -- 172.21.15.123:0/3981077677 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e40c8c30 con 0x7f18e406c570 2024-04-25T12:34:48.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.503+0000 7f18e9e00640 1 --2- 172.21.15.123:0/3981077677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e406c570 0x7f18e406c980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009d90 tx=0x7f18dc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeec3ca0137c45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:48.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18e3e00640 1 -- 172.21.15.123:0/3981077677 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007740 con 0x7f18e406c570 2024-04-25T12:34:48.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18e3e00640 1 -- 172.21.15.123:0/3981077677 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18dc0078e0 con 0x7f18e406c570 2024-04-25T12:34:48.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18e3e00640 1 -- 172.21.15.123:0/3981077677 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007cb0 con 0x7f18e406c570 2024-04-25T12:34:48.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18ea800640 1 -- 172.21.15.123:0/3981077677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e406c570 msgr2=0x7f18e406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18ea800640 1 --2- 172.21.15.123:0/3981077677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e406c570 0x7f18e406c980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009d90 tx=0x7f18dc02a5f0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18ea800640 1 -- 172.21.15.123:0/3981077677 shutdown_connections 2024-04-25T12:34:48.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18ea800640 1 --2- 172.21.15.123:0/3981077677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e406c570 0x7f18e406c980 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.504+0000 7f18ea800640 1 -- 172.21.15.123:0/3981077677 >> 172.21.15.123:0/3981077677 conn(0x7f18e40cd600 msgr2=0x7f18e40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:48.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.505+0000 7f18ea800640 1 -- 172.21.15.123:0/3981077677 shutdown_connections 2024-04-25T12:34:48.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.505+0000 7f18ea800640 1 -- 172.21.15.123:0/3981077677 wait complete. 2024-04-25T12:34:48.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.507+0000 7f18ea800640 1 Processor -- start 2024-04-25T12:34:48.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.507+0000 7f18ea800640 1 -- start start 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.508+0000 7f18ea800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e406c570 0x7f18e4158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18ea800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e415ac10 0x7f18e4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18ea800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18e4163ec0 0x7f18e41ec070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18ea800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4160b70 con 0x7f18e415ac10 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18ea800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18e415b020 con 0x7f18e406c570 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18ea800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f18e4164b90 con 0x7f18e4163ec0 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e406c570 0x7f18e4158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e415ac10 0x7f18e4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e415ac10 0x7f18e4159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35260/0 (socket says 172.21.15.123:35260) 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e406c570 0x7f18e4158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55242/0 (socket says 172.21.15.123:55242) 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18e4163ec0 0x7f18e41ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 -- 172.21.15.123:0/3224190892 learned_addr learned my addr 172.21.15.123:0/3224190892 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:48.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 -- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18e4163ec0 msgr2=0x7f18e41ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18e4163ec0 0x7f18e41ec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 -- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e406c570 msgr2=0x7f18e4158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e406c570 0x7f18e4158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.509+0000 7f18e8a00640 1 -- 172.21.15.123:0/3224190892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e41f7f50 con 0x7f18e415ac10 2024-04-25T12:34:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.510+0000 7f18e8a00640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e415ac10 0x7f18e4159d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c4a0 tx=0x7f18d400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.510+0000 7f18e1600640 1 -- 172.21.15.123:0/3224190892 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4017070 con 0x7f18e415ac10 2024-04-25T12:34:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.510+0000 7f18e1600640 1 -- 172.21.15.123:0/3224190892 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d4005d00 con 0x7f18e415ac10 2024-04-25T12:34:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.510+0000 7f18e1600640 1 -- 172.21.15.123:0/3224190892 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d40072b0 con 0x7f18e415ac10 2024-04-25T12:34:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.510+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e4157f60 con 0x7f18e415ac10 2024-04-25T12:34:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.510+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e4158520 con 0x7f18e415ac10 2024-04-25T12:34:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.511+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e40747e0 con 0x7f18e415ac10 2024-04-25T12:34:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.512+0000 7f18e1600640 1 -- 172.21.15.123:0/3224190892 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f18d401a070 con 0x7f18e415ac10 2024-04-25T12:34:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.513+0000 7f18e1600640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f18c80744e0 0x7f18c80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.514+0000 7f18e1600640 1 -- 172.21.15.123:0/3224190892 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f18d4094840 con 0x7f18e415ac10 2024-04-25T12:34:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.514+0000 7f18e9400640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f18c80744e0 0x7f18c80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.514+0000 7f18e9400640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f18c80744e0 0x7f18c80769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f18d8000f60 tx=0x7f18d8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.517+0000 7f18e1600640 1 -- 172.21.15.123:0/3224190892 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18e40747e0 con 0x7f18e415ac10 2024-04-25T12:34:48.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.701+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18e4076da0 con 0x7f18c80744e0 2024-04-25T12:34:48.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.712+0000 7f18e1600640 1 -- 172.21.15.123:0/3224190892 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332599 (secure 0 0 0) 0x7f18e4076da0 con 0x7f18c80744e0 2024-04-25T12:34:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:48.719 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:34:48.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f18c80744e0 msgr2=0x7f18c80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f18c80744e0 0x7f18c80769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f18d8000f60 tx=0x7f18d8009040 comp rx=0 tx=0).stop 2024-04-25T12:34:48.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e415ac10 msgr2=0x7f18e4159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:48.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e415ac10 0x7f18e4159d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c4a0 tx=0x7f18d400c960 comp rx=0 tx=0).stop 2024-04-25T12:34:48.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 shutdown_connections 2024-04-25T12:34:48.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f18c80744e0 0x7f18c80769f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18e4163ec0 0x7f18e41ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e415ac10 0x7f18e4159d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 --2- 172.21.15.123:0/3224190892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e406c570 0x7f18e4158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:48.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 >> 172.21.15.123:0/3224190892 conn(0x7f18e40cd600 msgr2=0x7f18e4160f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:48.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 shutdown_connections 2024-04-25T12:34:48.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:48.719+0000 7f18ea800640 1 -- 172.21.15.123:0/3224190892 wait complete. 2024-04-25T12:34:48.746 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T12:34:46.799677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218376,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756344,"statfs":{"total":773094113280,"available":772870496256,"internally_reserved":0,"allocated":2260992,"data_stored":2013480,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003881"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783414+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:34:19.783414+0000","last_peered":"2024-04-25T12:34:19.783414+0000","last_clean":"2024-04-25T12:34:19.783414+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:34:19.783414+0000","last_undegraded":"2024-04-25T12:34:19.783414+0000","last_fullsized":"2024-04-25T12:34:19.783414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782890+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:34:19.782890+0000","last_peered":"2024-04-25T12:34:19.782890+0000","last_clean":"2024-04-25T12:34:19.782890+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:34:19.782890+0000","last_undegraded":"2024-04-25T12:34:19.782890+0000","last_fullsized":"2024-04-25T12:34:19.782890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780793+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:34:19.780793+0000","last_peered":"2024-04-25T12:34:19.780793+0000","last_clean":"2024-04-25T12:34:19.780793+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:34:19.780793+0000","last_undegraded":"2024-04-25T12:34:19.780793+0000","last_fullsized":"2024-04-25T12:34:19.780793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782997+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:34:19.782997+0000","last_peered":"2024-04-25T12:34:19.782997+0000","last_clean":"2024-04-25T12:34:19.782997+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:34:19.782997+0000","last_undegraded":"2024-04-25T12:34:19.782997+0000","last_fullsized":"2024-04-25T12:34:19.782997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782606+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:34:19.782606+0000","last_peered":"2024-04-25T12:34:19.782606+0000","last_clean":"2024-04-25T12:34:19.782606+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:34:19.782606+0000","last_undegraded":"2024-04-25T12:34:19.782606+0000","last_fullsized":"2024-04-25T12:34:19.782606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784500+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:34:19.784500+0000","last_peered":"2024-04-25T12:34:19.784500+0000","last_clean":"2024-04-25T12:34:19.784500+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:34:19.784500+0000","last_undegraded":"2024-04-25T12:34:19.784500+0000","last_fullsized":"2024-04-25T12:34:19.784500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785563+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:34:19.785563+0000","last_peered":"2024-04-25T12:34:19.785563+0000","last_clean":"2024-04-25T12:34:19.785563+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:34:19.785563+0000","last_undegraded":"2024-04-25T12:34:19.785563+0000","last_fullsized":"2024-04-25T12:34:19.785563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.790406+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:34:19.790406+0000","last_peered":"2024-04-25T12:34:19.790406+0000","last_clean":"2024-04-25T12:34:19.790406+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:34:19.790406+0000","last_undegraded":"2024-04-25T12:34:19.790406+0000","last_fullsized":"2024-04-25T12:34:19.790406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788036+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:34:19.788036+0000","last_peered":"2024-04-25T12:34:19.788036+0000","last_clean":"2024-04-25T12:34:19.788036+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:34:19.788036+0000","last_undegraded":"2024-04-25T12:34:19.788036+0000","last_fullsized":"2024-04-25T12:34:19.788036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784075+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:34:19.784075+0000","last_peered":"2024-04-25T12:34:19.784075+0000","last_clean":"2024-04-25T12:34:19.784075+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:34:19.784075+0000","last_undegraded":"2024-04-25T12:34:19.784075+0000","last_fullsized":"2024-04-25T12:34:19.784075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982844+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:34:18.982844+0000","last_peered":"2024-04-25T12:34:18.982844+0000","last_clean":"2024-04-25T12:34:18.982844+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:34:18.982844+0000","last_undegraded":"2024-04-25T12:34:18.982844+0000","last_fullsized":"2024-04-25T12:34:18.982844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786691+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:34:19.786691+0000","last_peered":"2024-04-25T12:34:19.786691+0000","last_clean":"2024-04-25T12:34:19.786691+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:34:19.786691+0000","last_undegraded":"2024-04-25T12:34:19.786691+0000","last_fullsized":"2024-04-25T12:34:19.786691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982590+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:34:18.982590+0000","last_peered":"2024-04-25T12:34:18.982590+0000","last_clean":"2024-04-25T12:34:18.982590+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:34:18.982590+0000","last_undegraded":"2024-04-25T12:34:18.982590+0000","last_fullsized":"2024-04-25T12:34:18.982590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983302+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:34:18.983302+0000","last_peered":"2024-04-25T12:34:18.983302+0000","last_clean":"2024-04-25T12:34:18.983302+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:34:18.983302+0000","last_undegraded":"2024-04-25T12:34:18.983302+0000","last_fullsized":"2024-04-25T12:34:18.983302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982528+0000","last_change":"2024-04-25T12:34:18.007956+0000","last_active":"2024-04-25T12:34:18.982528+0000","last_peered":"2024-04-25T12:34:18.982528+0000","last_clean":"2024-04-25T12:34:18.982528+0000","last_became_active":"2024-04-25T12:34:18.007500+0000","last_became_peered":"2024-04-25T12:34:18.007500+0000","last_unstale":"2024-04-25T12:34:18.982528+0000","last_undegraded":"2024-04-25T12:34:18.982528+0000","last_fullsized":"2024-04-25T12:34:18.982528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:56.400953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782265+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:34:19.782265+0000","last_peered":"2024-04-25T12:34:19.782265+0000","last_clean":"2024-04-25T12:34:19.782265+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:34:19.782265+0000","last_undegraded":"2024-04-25T12:34:19.782265+0000","last_fullsized":"2024-04-25T12:34:19.782265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980152+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:34:18.980152+0000","last_peered":"2024-04-25T12:34:18.980152+0000","last_clean":"2024-04-25T12:34:18.980152+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:34:18.980152+0000","last_undegraded":"2024-04-25T12:34:18.980152+0000","last_fullsized":"2024-04-25T12:34:18.980152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783204+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:34:19.783204+0000","last_peered":"2024-04-25T12:34:19.783204+0000","last_clean":"2024-04-25T12:34:19.783204+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:34:19.783204+0000","last_undegraded":"2024-04-25T12:34:19.783204+0000","last_fullsized":"2024-04-25T12:34:19.783204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782958+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:34:19.782958+0000","last_peered":"2024-04-25T12:34:19.782958+0000","last_clean":"2024-04-25T12:34:19.782958+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:34:19.782958+0000","last_undegraded":"2024-04-25T12:34:19.782958+0000","last_fullsized":"2024-04-25T12:34:19.782958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781589+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:34:19.781589+0000","last_peered":"2024-04-25T12:34:19.781589+0000","last_clean":"2024-04-25T12:34:19.781589+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:34:19.781589+0000","last_undegraded":"2024-04-25T12:34:19.781589+0000","last_fullsized":"2024-04-25T12:34:19.781589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781948+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:34:19.781948+0000","last_peered":"2024-04-25T12:34:19.781948+0000","last_clean":"2024-04-25T12:34:19.781948+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:34:19.781948+0000","last_undegraded":"2024-04-25T12:34:19.781948+0000","last_fullsized":"2024-04-25T12:34:19.781948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981753+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:34:18.981753+0000","last_peered":"2024-04-25T12:34:18.981753+0000","last_clean":"2024-04-25T12:34:18.981753+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:34:18.981753+0000","last_undegraded":"2024-04-25T12:34:18.981753+0000","last_fullsized":"2024-04-25T12:34:18.981753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782248+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:34:19.782248+0000","last_peered":"2024-04-25T12:34:19.782248+0000","last_clean":"2024-04-25T12:34:19.782248+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:34:19.782248+0000","last_undegraded":"2024-04-25T12:34:19.782248+0000","last_fullsized":"2024-04-25T12:34:19.782248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788041+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:34:19.788041+0000","last_peered":"2024-04-25T12:34:19.788041+0000","last_clean":"2024-04-25T12:34:19.788041+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:34:19.788041+0000","last_undegraded":"2024-04-25T12:34:19.788041+0000","last_fullsized":"2024-04-25T12:34:19.788041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783014+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:34:19.783014+0000","last_peered":"2024-04-25T12:34:19.783014+0000","last_clean":"2024-04-25T12:34:19.783014+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:34:19.783014+0000","last_undegraded":"2024-04-25T12:34:19.783014+0000","last_fullsized":"2024-04-25T12:34:19.783014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983538+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:34:18.983538+0000","last_peered":"2024-04-25T12:34:18.983538+0000","last_clean":"2024-04-25T12:34:18.983538+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:34:18.983538+0000","last_undegraded":"2024-04-25T12:34:18.983538+0000","last_fullsized":"2024-04-25T12:34:18.983538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787014+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:34:19.787014+0000","last_peered":"2024-04-25T12:34:19.787014+0000","last_clean":"2024-04-25T12:34:19.787014+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:34:19.787014+0000","last_undegraded":"2024-04-25T12:34:19.787014+0000","last_fullsized":"2024-04-25T12:34:19.787014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783329+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:34:19.783329+0000","last_peered":"2024-04-25T12:34:19.783329+0000","last_clean":"2024-04-25T12:34:19.783329+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:34:19.783329+0000","last_undegraded":"2024-04-25T12:34:19.783329+0000","last_fullsized":"2024-04-25T12:34:19.783329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783247+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:34:19.783247+0000","last_peered":"2024-04-25T12:34:19.783247+0000","last_clean":"2024-04-25T12:34:19.783247+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:34:19.783247+0000","last_undegraded":"2024-04-25T12:34:19.783247+0000","last_fullsized":"2024-04-25T12:34:19.783247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781103+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:34:19.781103+0000","last_peered":"2024-04-25T12:34:19.781103+0000","last_clean":"2024-04-25T12:34:19.781103+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:34:19.781103+0000","last_undegraded":"2024-04-25T12:34:19.781103+0000","last_fullsized":"2024-04-25T12:34:19.781103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980759+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:34:18.980759+0000","last_peered":"2024-04-25T12:34:18.980759+0000","last_clean":"2024-04-25T12:34:18.980759+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:34:18.980759+0000","last_undegraded":"2024-04-25T12:34:18.980759+0000","last_fullsized":"2024-04-25T12:34:18.980759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983863+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:34:18.983863+0000","last_peered":"2024-04-25T12:34:18.983863+0000","last_clean":"2024-04-25T12:34:18.983863+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:34:18.983863+0000","last_undegraded":"2024-04-25T12:34:18.983863+0000","last_fullsized":"2024-04-25T12:34:18.983863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980463+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:34:18.980463+0000","last_peered":"2024-04-25T12:34:18.980463+0000","last_clean":"2024-04-25T12:34:18.980463+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:34:18.980463+0000","last_undegraded":"2024-04-25T12:34:18.980463+0000","last_fullsized":"2024-04-25T12:34:18.980463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789572+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:34:19.789572+0000","last_peered":"2024-04-25T12:34:19.789572+0000","last_clean":"2024-04-25T12:34:19.789572+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:34:19.789572+0000","last_undegraded":"2024-04-25T12:34:19.789572+0000","last_fullsized":"2024-04-25T12:34:19.789572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782068+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:34:19.782068+0000","last_peered":"2024-04-25T12:34:19.782068+0000","last_clean":"2024-04-25T12:34:19.782068+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:34:19.782068+0000","last_undegraded":"2024-04-25T12:34:19.782068+0000","last_fullsized":"2024-04-25T12:34:19.782068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979987+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:34:18.979987+0000","last_peered":"2024-04-25T12:34:18.979987+0000","last_clean":"2024-04-25T12:34:18.979987+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:34:18.979987+0000","last_undegraded":"2024-04-25T12:34:18.979987+0000","last_fullsized":"2024-04-25T12:34:18.979987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783310+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:34:19.783310+0000","last_peered":"2024-04-25T12:34:19.783310+0000","last_clean":"2024-04-25T12:34:19.783310+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:34:19.783310+0000","last_undegraded":"2024-04-25T12:34:19.783310+0000","last_fullsized":"2024-04-25T12:34:19.783310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982842+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:34:18.982842+0000","last_peered":"2024-04-25T12:34:18.982842+0000","last_clean":"2024-04-25T12:34:18.982842+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:34:18.982842+0000","last_undegraded":"2024-04-25T12:34:18.982842+0000","last_fullsized":"2024-04-25T12:34:18.982842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980473+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:34:18.980473+0000","last_peered":"2024-04-25T12:34:18.980473+0000","last_clean":"2024-04-25T12:34:18.980473+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:34:18.980473+0000","last_undegraded":"2024-04-25T12:34:18.980473+0000","last_fullsized":"2024-04-25T12:34:18.980473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781093+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:34:19.781093+0000","last_peered":"2024-04-25T12:34:19.781093+0000","last_clean":"2024-04-25T12:34:19.781093+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:34:19.781093+0000","last_undegraded":"2024-04-25T12:34:19.781093+0000","last_fullsized":"2024-04-25T12:34:19.781093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781452+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:34:19.781452+0000","last_peered":"2024-04-25T12:34:19.781452+0000","last_clean":"2024-04-25T12:34:19.781452+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:34:19.781452+0000","last_undegraded":"2024-04-25T12:34:19.781452+0000","last_fullsized":"2024-04-25T12:34:19.781452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980445+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:34:18.980445+0000","last_peered":"2024-04-25T12:34:18.980445+0000","last_clean":"2024-04-25T12:34:18.980445+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:34:18.980445+0000","last_undegraded":"2024-04-25T12:34:18.980445+0000","last_fullsized":"2024-04-25T12:34:18.980445+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979799+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:34:18.979799+0000","last_peered":"2024-04-25T12:34:18.979799+0000","last_clean":"2024-04-25T12:34:18.979799+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:34:18.979799+0000","last_undegraded":"2024-04-25T12:34:18.979799+0000","last_fullsized":"2024-04-25T12:34:18.979799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982431+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:34:18.982431+0000","last_peered":"2024-04-25T12:34:18.982431+0000","last_clean":"2024-04-25T12:34:18.982431+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:34:18.982431+0000","last_undegraded":"2024-04-25T12:34:18.982431+0000","last_fullsized":"2024-04-25T12:34:18.982431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782825+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:34:19.782825+0000","last_peered":"2024-04-25T12:34:19.782825+0000","last_clean":"2024-04-25T12:34:19.782825+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:34:19.782825+0000","last_undegraded":"2024-04-25T12:34:19.782825+0000","last_fullsized":"2024-04-25T12:34:19.782825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783355+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:34:19.783355+0000","last_peered":"2024-04-25T12:34:19.783355+0000","last_clean":"2024-04-25T12:34:19.783355+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:34:19.783355+0000","last_undegraded":"2024-04-25T12:34:19.783355+0000","last_fullsized":"2024-04-25T12:34:19.783355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983351+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:34:18.983351+0000","last_peered":"2024-04-25T12:34:18.983351+0000","last_clean":"2024-04-25T12:34:18.983351+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:34:18.983351+0000","last_undegraded":"2024-04-25T12:34:18.983351+0000","last_fullsized":"2024-04-25T12:34:18.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787419+0000","last_change":"2024-04-25T12:34:08.560989+0000","last_active":"2024-04-25T12:34:19.787419+0000","last_peered":"2024-04-25T12:34:19.787419+0000","last_clean":"2024-04-25T12:34:19.787419+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:34:19.787419+0000","last_undegraded":"2024-04-25T12:34:19.787419+0000","last_fullsized":"2024-04-25T12:34:19.787419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:49.130809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983747+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:34:18.983747+0000","last_peered":"2024-04-25T12:34:18.983747+0000","last_clean":"2024-04-25T12:34:18.983747+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:34:18.983747+0000","last_undegraded":"2024-04-25T12:34:18.983747+0000","last_fullsized":"2024-04-25T12:34:18.983747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982942+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:34:18.982942+0000","last_peered":"2024-04-25T12:34:18.982942+0000","last_clean":"2024-04-25T12:34:18.982942+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:34:18.982942+0000","last_undegraded":"2024-04-25T12:34:18.982942+0000","last_fullsized":"2024-04-25T12:34:18.982942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980766+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:34:18.980766+0000","last_peered":"2024-04-25T12:34:18.980766+0000","last_clean":"2024-04-25T12:34:18.980766+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:34:18.980766+0000","last_undegraded":"2024-04-25T12:34:18.980766+0000","last_fullsized":"2024-04-25T12:34:18.980766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786888+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:34:19.786888+0000","last_peered":"2024-04-25T12:34:19.786888+0000","last_clean":"2024-04-25T12:34:19.786888+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:34:19.786888+0000","last_undegraded":"2024-04-25T12:34:19.786888+0000","last_fullsized":"2024-04-25T12:34:19.786888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783906+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:34:19.783906+0000","last_peered":"2024-04-25T12:34:19.783906+0000","last_clean":"2024-04-25T12:34:19.783906+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:34:19.783906+0000","last_undegraded":"2024-04-25T12:34:19.783906+0000","last_fullsized":"2024-04-25T12:34:19.783906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782493+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:34:19.782493+0000","last_peered":"2024-04-25T12:34:19.782493+0000","last_clean":"2024-04-25T12:34:19.782493+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:34:19.782493+0000","last_undegraded":"2024-04-25T12:34:19.782493+0000","last_fullsized":"2024-04-25T12:34:19.782493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781393+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:34:19.781393+0000","last_peered":"2024-04-25T12:34:19.781393+0000","last_clean":"2024-04-25T12:34:19.781393+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:34:19.781393+0000","last_undegraded":"2024-04-25T12:34:19.781393+0000","last_fullsized":"2024-04-25T12:34:19.781393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785529+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:34:19.785529+0000","last_peered":"2024-04-25T12:34:19.785529+0000","last_clean":"2024-04-25T12:34:19.785529+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:34:19.785529+0000","last_undegraded":"2024-04-25T12:34:19.785529+0000","last_fullsized":"2024-04-25T12:34:19.785529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783018+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:34:19.783018+0000","last_peered":"2024-04-25T12:34:19.783018+0000","last_clean":"2024-04-25T12:34:19.783018+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:34:19.783018+0000","last_undegraded":"2024-04-25T12:34:19.783018+0000","last_fullsized":"2024-04-25T12:34:19.783018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981441+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:34:18.981441+0000","last_peered":"2024-04-25T12:34:18.981441+0000","last_clean":"2024-04-25T12:34:18.981441+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:34:18.981441+0000","last_undegraded":"2024-04-25T12:34:18.981441+0000","last_fullsized":"2024-04-25T12:34:18.981441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788784+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:34:19.788784+0000","last_peered":"2024-04-25T12:34:19.788784+0000","last_clean":"2024-04-25T12:34:19.788784+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:34:19.788784+0000","last_undegraded":"2024-04-25T12:34:19.788784+0000","last_fullsized":"2024-04-25T12:34:19.788784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785856+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:34:19.785856+0000","last_peered":"2024-04-25T12:34:19.785856+0000","last_clean":"2024-04-25T12:34:19.785856+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:34:19.785856+0000","last_undegraded":"2024-04-25T12:34:19.785856+0000","last_fullsized":"2024-04-25T12:34:19.785856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788439+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:34:19.788439+0000","last_peered":"2024-04-25T12:34:19.788439+0000","last_clean":"2024-04-25T12:34:19.788439+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:34:19.788439+0000","last_undegraded":"2024-04-25T12:34:19.788439+0000","last_fullsized":"2024-04-25T12:34:19.788439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981709+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:34:18.981709+0000","last_peered":"2024-04-25T12:34:18.981709+0000","last_clean":"2024-04-25T12:34:18.981709+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:34:18.981709+0000","last_undegraded":"2024-04-25T12:34:18.981709+0000","last_fullsized":"2024-04-25T12:34:18.981709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781190+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:34:19.781190+0000","last_peered":"2024-04-25T12:34:19.781190+0000","last_clean":"2024-04-25T12:34:19.781190+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:34:19.781190+0000","last_undegraded":"2024-04-25T12:34:19.781190+0000","last_fullsized":"2024-04-25T12:34:19.781190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782909+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:34:19.782909+0000","last_peered":"2024-04-25T12:34:19.782909+0000","last_clean":"2024-04-25T12:34:19.782909+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:34:19.782909+0000","last_undegraded":"2024-04-25T12:34:19.782909+0000","last_fullsized":"2024-04-25T12:34:19.782909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786692+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:34:19.786692+0000","last_peered":"2024-04-25T12:34:19.786692+0000","last_clean":"2024-04-25T12:34:19.786692+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:34:19.786692+0000","last_undegraded":"2024-04-25T12:34:19.786692+0000","last_fullsized":"2024-04-25T12:34:19.786692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782692+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:34:19.782692+0000","last_peered":"2024-04-25T12:34:19.782692+0000","last_clean":"2024-04-25T12:34:19.782692+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:34:19.782692+0000","last_undegraded":"2024-04-25T12:34:19.782692+0000","last_fullsized":"2024-04-25T12:34:19.782692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782809+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:34:19.782809+0000","last_peered":"2024-04-25T12:34:19.782809+0000","last_clean":"2024-04-25T12:34:19.782809+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:34:19.782809+0000","last_undegraded":"2024-04-25T12:34:19.782809+0000","last_fullsized":"2024-04-25T12:34:19.782809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983105+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:34:18.983105+0000","last_peered":"2024-04-25T12:34:18.983105+0000","last_clean":"2024-04-25T12:34:18.983105+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:34:18.983105+0000","last_undegraded":"2024-04-25T12:34:18.983105+0000","last_fullsized":"2024-04-25T12:34:18.983105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982716+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:34:18.982716+0000","last_peered":"2024-04-25T12:34:18.982716+0000","last_clean":"2024-04-25T12:34:18.982716+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:34:18.982716+0000","last_undegraded":"2024-04-25T12:34:18.982716+0000","last_fullsized":"2024-04-25T12:34:18.982716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980674+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:34:18.980674+0000","last_peered":"2024-04-25T12:34:18.980674+0000","last_clean":"2024-04-25T12:34:18.980674+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:34:18.980674+0000","last_undegraded":"2024-04-25T12:34:18.980674+0000","last_fullsized":"2024-04-25T12:34:18.980674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980038+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:34:18.980038+0000","last_peered":"2024-04-25T12:34:18.980038+0000","last_clean":"2024-04-25T12:34:18.980038+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:34:18.980038+0000","last_undegraded":"2024-04-25T12:34:18.980038+0000","last_fullsized":"2024-04-25T12:34:18.980038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782400+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:34:19.782400+0000","last_peered":"2024-04-25T12:34:19.782400+0000","last_clean":"2024-04-25T12:34:19.782400+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:34:19.782400+0000","last_undegraded":"2024-04-25T12:34:19.782400+0000","last_fullsized":"2024-04-25T12:34:19.782400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982809+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:34:18.982809+0000","last_peered":"2024-04-25T12:34:18.982809+0000","last_clean":"2024-04-25T12:34:18.982809+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:34:18.982809+0000","last_undegraded":"2024-04-25T12:34:18.982809+0000","last_fullsized":"2024-04-25T12:34:18.982809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784989+0000","last_change":"2024-04-25T12:34:13.973917+0000","last_active":"2024-04-25T12:34:19.784989+0000","last_peered":"2024-04-25T12:34:19.784989+0000","last_clean":"2024-04-25T12:34:19.784989+0000","last_became_active":"2024-04-25T12:34:13.973640+0000","last_became_peered":"2024-04-25T12:34:13.973640+0000","last_unstale":"2024-04-25T12:34:19.784989+0000","last_undegraded":"2024-04-25T12:34:19.784989+0000","last_fullsized":"2024-04-25T12:34:19.784989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:21.717728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:36.068337+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:34:36.068337+0000","last_peered":"2024-04-25T12:34:36.068337+0000","last_clean":"2024-04-25T12:34:36.068337+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:34:36.068337+0000","last_undegraded":"2024-04-25T12:34:36.068337+0000","last_fullsized":"2024-04-25T12:34:36.068337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783277+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:34:19.783277+0000","last_peered":"2024-04-25T12:34:19.783277+0000","last_clean":"2024-04-25T12:34:19.783277+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:34:19.783277+0000","last_undegraded":"2024-04-25T12:34:19.783277+0000","last_fullsized":"2024-04-25T12:34:19.783277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982241+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:34:18.982241+0000","last_peered":"2024-04-25T12:34:18.982241+0000","last_clean":"2024-04-25T12:34:18.982241+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:34:18.982241+0000","last_undegraded":"2024-04-25T12:34:18.982241+0000","last_fullsized":"2024-04-25T12:34:18.982241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782037+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:34:19.782037+0000","last_peered":"2024-04-25T12:34:19.782037+0000","last_clean":"2024-04-25T12:34:19.782037+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:34:19.782037+0000","last_undegraded":"2024-04-25T12:34:19.782037+0000","last_fullsized":"2024-04-25T12:34:19.782037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788580+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:34:19.788580+0000","last_peered":"2024-04-25T12:34:19.788580+0000","last_clean":"2024-04-25T12:34:19.788580+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:34:19.788580+0000","last_undegraded":"2024-04-25T12:34:19.788580+0000","last_fullsized":"2024-04-25T12:34:19.788580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782777+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:34:19.782777+0000","last_peered":"2024-04-25T12:34:19.782777+0000","last_clean":"2024-04-25T12:34:19.782777+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:34:19.782777+0000","last_undegraded":"2024-04-25T12:34:19.782777+0000","last_fullsized":"2024-04-25T12:34:19.782777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784953+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:34:19.784953+0000","last_peered":"2024-04-25T12:34:19.784953+0000","last_clean":"2024-04-25T12:34:19.784953+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:34:19.784953+0000","last_undegraded":"2024-04-25T12:34:19.784953+0000","last_fullsized":"2024-04-25T12:34:19.784953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780643+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:34:19.780643+0000","last_peered":"2024-04-25T12:34:19.780643+0000","last_clean":"2024-04-25T12:34:19.780643+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:34:19.780643+0000","last_undegraded":"2024-04-25T12:34:19.780643+0000","last_fullsized":"2024-04-25T12:34:19.780643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788514+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:34:19.788514+0000","last_peered":"2024-04-25T12:34:19.788514+0000","last_clean":"2024-04-25T12:34:19.788514+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:34:19.788514+0000","last_undegraded":"2024-04-25T12:34:19.788514+0000","last_fullsized":"2024-04-25T12:34:19.788514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.791187+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:34:19.791187+0000","last_peered":"2024-04-25T12:34:19.791187+0000","last_clean":"2024-04-25T12:34:19.791187+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:34:19.791187+0000","last_undegraded":"2024-04-25T12:34:19.791187+0000","last_fullsized":"2024-04-25T12:34:19.791187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781414+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:34:19.781414+0000","last_peered":"2024-04-25T12:34:19.781414+0000","last_clean":"2024-04-25T12:34:19.781414+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:34:19.781414+0000","last_undegraded":"2024-04-25T12:34:19.781414+0000","last_fullsized":"2024-04-25T12:34:19.781414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782882+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:34:19.782882+0000","last_peered":"2024-04-25T12:34:19.782882+0000","last_clean":"2024-04-25T12:34:19.782882+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:34:19.782882+0000","last_undegraded":"2024-04-25T12:34:19.782882+0000","last_fullsized":"2024-04-25T12:34:19.782882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781481+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:34:19.781481+0000","last_peered":"2024-04-25T12:34:19.781481+0000","last_clean":"2024-04-25T12:34:19.781481+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:34:19.781481+0000","last_undegraded":"2024-04-25T12:34:19.781481+0000","last_fullsized":"2024-04-25T12:34:19.781481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982283+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:34:18.982283+0000","last_peered":"2024-04-25T12:34:18.982283+0000","last_clean":"2024-04-25T12:34:18.982283+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:34:18.982283+0000","last_undegraded":"2024-04-25T12:34:18.982283+0000","last_fullsized":"2024-04-25T12:34:18.982283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983399+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:34:18.983399+0000","last_peered":"2024-04-25T12:34:18.983399+0000","last_clean":"2024-04-25T12:34:18.983399+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:34:18.983399+0000","last_undegraded":"2024-04-25T12:34:18.983399+0000","last_fullsized":"2024-04-25T12:34:18.983399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782668+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:34:19.782668+0000","last_peered":"2024-04-25T12:34:19.782668+0000","last_clean":"2024-04-25T12:34:19.782668+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:34:19.782668+0000","last_undegraded":"2024-04-25T12:34:19.782668+0000","last_fullsized":"2024-04-25T12:34:19.782668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979833+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:34:18.979833+0000","last_peered":"2024-04-25T12:34:18.979833+0000","last_clean":"2024-04-25T12:34:18.979833+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:34:18.979833+0000","last_undegraded":"2024-04-25T12:34:18.979833+0000","last_fullsized":"2024-04-25T12:34:18.979833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982033+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:34:18.982033+0000","last_peered":"2024-04-25T12:34:18.982033+0000","last_clean":"2024-04-25T12:34:18.982033+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:34:18.982033+0000","last_undegraded":"2024-04-25T12:34:18.982033+0000","last_fullsized":"2024-04-25T12:34:18.982033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783137+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:34:19.783137+0000","last_peered":"2024-04-25T12:34:19.783137+0000","last_clean":"2024-04-25T12:34:19.783137+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:34:19.783137+0000","last_undegraded":"2024-04-25T12:34:19.783137+0000","last_fullsized":"2024-04-25T12:34:19.783137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782539+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:34:19.782539+0000","last_peered":"2024-04-25T12:34:19.782539+0000","last_clean":"2024-04-25T12:34:19.782539+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:34:19.782539+0000","last_undegraded":"2024-04-25T12:34:19.782539+0000","last_fullsized":"2024-04-25T12:34:19.782539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781454+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:34:19.781454+0000","last_peered":"2024-04-25T12:34:19.781454+0000","last_clean":"2024-04-25T12:34:19.781454+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:34:19.781454+0000","last_undegraded":"2024-04-25T12:34:19.781454+0000","last_fullsized":"2024-04-25T12:34:19.781454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789165+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:34:19.789165+0000","last_peered":"2024-04-25T12:34:19.789165+0000","last_clean":"2024-04-25T12:34:19.789165+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:34:19.789165+0000","last_undegraded":"2024-04-25T12:34:19.789165+0000","last_fullsized":"2024-04-25T12:34:19.789165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781753+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:34:19.781753+0000","last_peered":"2024-04-25T12:34:19.781753+0000","last_clean":"2024-04-25T12:34:19.781753+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:34:19.781753+0000","last_undegraded":"2024-04-25T12:34:19.781753+0000","last_fullsized":"2024-04-25T12:34:19.781753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785020+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:34:19.785020+0000","last_peered":"2024-04-25T12:34:19.785020+0000","last_clean":"2024-04-25T12:34:19.785020+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:34:19.785020+0000","last_undegraded":"2024-04-25T12:34:19.785020+0000","last_fullsized":"2024-04-25T12:34:19.785020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781340+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:34:19.781340+0000","last_peered":"2024-04-25T12:34:19.781340+0000","last_clean":"2024-04-25T12:34:19.781340+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:34:19.781340+0000","last_undegraded":"2024-04-25T12:34:19.781340+0000","last_fullsized":"2024-04-25T12:34:19.781340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981734+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:34:18.981734+0000","last_peered":"2024-04-25T12:34:18.981734+0000","last_clean":"2024-04-25T12:34:18.981734+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:34:18.981734+0000","last_undegraded":"2024-04-25T12:34:18.981734+0000","last_fullsized":"2024-04-25T12:34:18.981734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781427+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:34:19.781427+0000","last_peered":"2024-04-25T12:34:19.781427+0000","last_clean":"2024-04-25T12:34:19.781427+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:34:19.781427+0000","last_undegraded":"2024-04-25T12:34:19.781427+0000","last_fullsized":"2024-04-25T12:34:19.781427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980630+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:34:18.980630+0000","last_peered":"2024-04-25T12:34:18.980630+0000","last_clean":"2024-04-25T12:34:18.980630+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:34:18.980630+0000","last_undegraded":"2024-04-25T12:34:18.980630+0000","last_fullsized":"2024-04-25T12:34:18.980630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980186+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:34:18.980186+0000","last_peered":"2024-04-25T12:34:18.980186+0000","last_clean":"2024-04-25T12:34:18.980186+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:34:18.980186+0000","last_undegraded":"2024-04-25T12:34:18.980186+0000","last_fullsized":"2024-04-25T12:34:18.980186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788210+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:34:19.788210+0000","last_peered":"2024-04-25T12:34:19.788210+0000","last_clean":"2024-04-25T12:34:19.788210+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:34:19.788210+0000","last_undegraded":"2024-04-25T12:34:19.788210+0000","last_fullsized":"2024-04-25T12:34:19.788210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782446+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:34:19.782446+0000","last_peered":"2024-04-25T12:34:19.782446+0000","last_clean":"2024-04-25T12:34:19.782446+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:34:19.782446+0000","last_undegraded":"2024-04-25T12:34:19.782446+0000","last_fullsized":"2024-04-25T12:34:19.782446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982148+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:34:18.982148+0000","last_peered":"2024-04-25T12:34:18.982148+0000","last_clean":"2024-04-25T12:34:18.982148+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:34:18.982148+0000","last_undegraded":"2024-04-25T12:34:18.982148+0000","last_fullsized":"2024-04-25T12:34:18.982148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781207+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:34:19.781207+0000","last_peered":"2024-04-25T12:34:19.781207+0000","last_clean":"2024-04-25T12:34:19.781207+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:34:19.781207+0000","last_undegraded":"2024-04-25T12:34:19.781207+0000","last_fullsized":"2024-04-25T12:34:19.781207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789184+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:34:19.789184+0000","last_peered":"2024-04-25T12:34:19.789184+0000","last_clean":"2024-04-25T12:34:19.789184+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:34:19.789184+0000","last_undegraded":"2024-04-25T12:34:19.789184+0000","last_fullsized":"2024-04-25T12:34:19.789184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783090+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:34:19.783090+0000","last_peered":"2024-04-25T12:34:19.783090+0000","last_clean":"2024-04-25T12:34:19.783090+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:34:19.783090+0000","last_undegraded":"2024-04-25T12:34:19.783090+0000","last_fullsized":"2024-04-25T12:34:19.783090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783799+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:34:19.783799+0000","last_peered":"2024-04-25T12:34:19.783799+0000","last_clean":"2024-04-25T12:34:19.783799+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:34:19.783799+0000","last_undegraded":"2024-04-25T12:34:19.783799+0000","last_fullsized":"2024-04-25T12:34:19.783799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983704+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:34:18.983704+0000","last_peered":"2024-04-25T12:34:18.983704+0000","last_clean":"2024-04-25T12:34:18.983704+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:34:18.983704+0000","last_undegraded":"2024-04-25T12:34:18.983704+0000","last_fullsized":"2024-04-25T12:34:18.983704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782288+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:34:19.782288+0000","last_peered":"2024-04-25T12:34:19.782288+0000","last_clean":"2024-04-25T12:34:19.782288+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:34:19.782288+0000","last_undegraded":"2024-04-25T12:34:19.782288+0000","last_fullsized":"2024-04-25T12:34:19.782288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983392+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:34:18.983392+0000","last_peered":"2024-04-25T12:34:18.983392+0000","last_clean":"2024-04-25T12:34:18.983392+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:34:18.983392+0000","last_undegraded":"2024-04-25T12:34:18.983392+0000","last_fullsized":"2024-04-25T12:34:18.983392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981957+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:34:18.981957+0000","last_peered":"2024-04-25T12:34:18.981957+0000","last_clean":"2024-04-25T12:34:18.981957+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:34:18.981957+0000","last_undegraded":"2024-04-25T12:34:18.981957+0000","last_fullsized":"2024-04-25T12:34:18.981957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780879+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:34:19.780879+0000","last_peered":"2024-04-25T12:34:19.780879+0000","last_clean":"2024-04-25T12:34:19.780879+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:34:19.780879+0000","last_undegraded":"2024-04-25T12:34:19.780879+0000","last_fullsized":"2024-04-25T12:34:19.780879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979899+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:34:18.979899+0000","last_peered":"2024-04-25T12:34:18.979899+0000","last_clean":"2024-04-25T12:34:18.979899+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:34:18.979899+0000","last_undegraded":"2024-04-25T12:34:18.979899+0000","last_fullsized":"2024-04-25T12:34:18.979899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980556+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:34:18.980556+0000","last_peered":"2024-04-25T12:34:18.980556+0000","last_clean":"2024-04-25T12:34:18.980556+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:34:18.980556+0000","last_undegraded":"2024-04-25T12:34:18.980556+0000","last_fullsized":"2024-04-25T12:34:18.980556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783694+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:34:19.783694+0000","last_peered":"2024-04-25T12:34:19.783694+0000","last_clean":"2024-04-25T12:34:19.783694+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:34:19.783694+0000","last_undegraded":"2024-04-25T12:34:19.783694+0000","last_fullsized":"2024-04-25T12:34:19.783694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784671+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:34:19.784671+0000","last_peered":"2024-04-25T12:34:19.784671+0000","last_clean":"2024-04-25T12:34:19.784671+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:34:19.784671+0000","last_undegraded":"2024-04-25T12:34:19.784671+0000","last_fullsized":"2024-04-25T12:34:19.784671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784485+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:34:19.784485+0000","last_peered":"2024-04-25T12:34:19.784485+0000","last_clean":"2024-04-25T12:34:19.784485+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:34:19.784485+0000","last_undegraded":"2024-04-25T12:34:19.784485+0000","last_fullsized":"2024-04-25T12:34:19.784485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.791100+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:34:19.791100+0000","last_peered":"2024-04-25T12:34:19.791100+0000","last_clean":"2024-04-25T12:34:19.791100+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:34:19.791100+0000","last_undegraded":"2024-04-25T12:34:19.791100+0000","last_fullsized":"2024-04-25T12:34:19.791100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781305+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:34:19.781305+0000","last_peered":"2024-04-25T12:34:19.781305+0000","last_clean":"2024-04-25T12:34:19.781305+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:34:19.781305+0000","last_undegraded":"2024-04-25T12:34:19.781305+0000","last_fullsized":"2024-04-25T12:34:19.781305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782773+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:34:19.782773+0000","last_peered":"2024-04-25T12:34:19.782773+0000","last_clean":"2024-04-25T12:34:19.782773+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:34:19.782773+0000","last_undegraded":"2024-04-25T12:34:19.782773+0000","last_fullsized":"2024-04-25T12:34:19.782773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786973+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:34:19.786973+0000","last_peered":"2024-04-25T12:34:19.786973+0000","last_clean":"2024-04-25T12:34:19.786973+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:34:19.786973+0000","last_undegraded":"2024-04-25T12:34:19.786973+0000","last_fullsized":"2024-04-25T12:34:19.786973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783422+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:34:19.783422+0000","last_peered":"2024-04-25T12:34:19.783422+0000","last_clean":"2024-04-25T12:34:19.783422+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:34:19.783422+0000","last_undegraded":"2024-04-25T12:34:19.783422+0000","last_fullsized":"2024-04-25T12:34:19.783422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788118+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:34:19.788118+0000","last_peered":"2024-04-25T12:34:19.788118+0000","last_clean":"2024-04-25T12:34:19.788118+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:34:19.788118+0000","last_undegraded":"2024-04-25T12:34:19.788118+0000","last_fullsized":"2024-04-25T12:34:19.788118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782309+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:34:19.782309+0000","last_peered":"2024-04-25T12:34:19.782309+0000","last_clean":"2024-04-25T12:34:19.782309+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:34:19.782309+0000","last_undegraded":"2024-04-25T12:34:19.782309+0000","last_fullsized":"2024-04-25T12:34:19.782309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782643+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:34:19.782643+0000","last_peered":"2024-04-25T12:34:19.782643+0000","last_clean":"2024-04-25T12:34:19.782643+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:34:19.782643+0000","last_undegraded":"2024-04-25T12:34:19.782643+0000","last_fullsized":"2024-04-25T12:34:19.782643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785344+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:34:19.785344+0000","last_peered":"2024-04-25T12:34:19.785344+0000","last_clean":"2024-04-25T12:34:19.785344+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:34:19.785344+0000","last_undegraded":"2024-04-25T12:34:19.785344+0000","last_fullsized":"2024-04-25T12:34:19.785344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":103515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":693393,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":693371,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:34:48.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:34:49.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.064+0000 7ff007200640 1 Processor -- start 2024-04-25T12:34:49.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.064+0000 7ff007200640 1 -- start start 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff007200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0080c9080 0x7ff0081546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff007200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0080c82b0 0x7ff00815e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff007200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff00815a330 con 0x7ff00815e590 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff007200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff008073b10 con 0x7ff0080c9080 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff007200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff008073df0 con 0x7ff0080c82b0 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff005e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0080c9080 0x7ff0081546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff006800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff00815e590 0x7ff00806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55978/0 (socket says 172.21.15.123:55978) 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.066+0000 7ff006800640 1 -- 172.21.15.123:0/3205948744 learned_addr learned my addr 172.21.15.123:0/3205948744 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:49.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.067+0000 7fefffe00640 1 --2- 172.21.15.123:0/3205948744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0080c82b0 0x7ff00815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.067+0000 7ff005e00640 1 -- 172.21.15.123:0/3205948744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0080c82b0 msgr2=0x7ff00815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:49.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.067+0000 7ff005e00640 1 --2- 172.21.15.123:0/3205948744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0080c82b0 0x7ff00815e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.067+0000 7ff005e00640 1 -- 172.21.15.123:0/3205948744 >> v1:172.21.15.123:6789/0 conn(0x7ff00815e590 legacy=0x7ff00806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.067+0000 7ff005e00640 1 -- 172.21.15.123:0/3205948744 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0080740d0 con 0x7ff0080c9080 2024-04-25T12:34:49.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.067+0000 7ff005400640 1 -- 172.21.15.123:0/3205948744 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451627827 0 0) 0x7ff00815a330 con 0x7ff00815e590 2024-04-25T12:34:49.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.068+0000 7ff005e00640 1 --2- 172.21.15.123:0/3205948744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c9080 0x7ff0081546d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff00815a500 tx=0x7feff802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa1272ebba2d318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.068+0000 7ff005400640 1 -- 172.21.15.123:0/3205948744 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8037070 con 0x7ff0080c9080 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.068+0000 7ff005400640 1 -- 172.21.15.123:0/3205948744 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff8032440 con 0x7ff0080c9080 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.068+0000 7ff005400640 1 -- 172.21.15.123:0/3205948744 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8032810 con 0x7ff0080c9080 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 -- 172.21.15.123:0/3205948744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c9080 msgr2=0x7ff0081546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 --2- 172.21.15.123:0/3205948744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c9080 0x7ff0081546d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff00815a500 tx=0x7feff802a180 comp rx=0 tx=0).stop 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 -- 172.21.15.123:0/3205948744 shutdown_connections 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 --2- 172.21.15.123:0/3205948744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0080c82b0 0x7ff00815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 --2- 172.21.15.123:0/3205948744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c9080 0x7ff0081546d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 -- 172.21.15.123:0/3205948744 >> 172.21.15.123:0/3205948744 conn(0x7ff0080cd600 msgr2=0x7ff0080cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 -- 172.21.15.123:0/3205948744 shutdown_connections 2024-04-25T12:34:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.069+0000 7ff007200640 1 -- 172.21.15.123:0/3205948744 wait complete. 2024-04-25T12:34:49.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.072+0000 7ff007200640 1 Processor -- start 2024-04-25T12:34:49.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.072+0000 7ff007200640 1 -- start start 2024-04-25T12:34:49.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.073+0000 7ff007200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c82b0 0x7ff0080b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.073+0000 7ff007200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0080c9080 0x7ff008155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.073+0000 7ff007200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff00815e590 0x7ff008150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.073+0000 7ff007200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff008073760 con 0x7ff0080c82b0 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.073+0000 7ff007200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff00815db30 con 0x7ff0080c9080 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.073+0000 7ff007200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff0080b1d70 con 0x7ff00815e590 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c82b0 0x7ff0080b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c82b0 0x7ff0080b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35268/0 (socket says 172.21.15.123:35268) 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 -- 172.21.15.123:0/2399276460 learned_addr learned my addr 172.21.15.123:0/2399276460 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7fefffe00640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0080c9080 0x7ff008155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff006800640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff00815e590 0x7ff008150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 -- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff00815e590 msgr2=0x7ff008150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff00815e590 0x7ff008150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 -- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0080c9080 msgr2=0x7ff008155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0080c9080 0x7ff008155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 -- 172.21.15.123:0/2399276460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0080b18a0 con 0x7ff0080c82b0 2024-04-25T12:34:49.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.074+0000 7ff005e00640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c82b0 0x7ff0080b1360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:49.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.075+0000 7feffe000640 1 -- 172.21.15.123:0/2399276460 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8037050 con 0x7ff0080c82b0 2024-04-25T12:34:49.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.075+0000 7feffe000640 1 -- 172.21.15.123:0/2399276460 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff8038040 con 0x7ff0080c82b0 2024-04-25T12:34:49.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.075+0000 7feffe000640 1 -- 172.21.15.123:0/2399276460 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8004470 con 0x7ff0080c82b0 2024-04-25T12:34:49.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.075+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0081f6610 con 0x7ff0080c82b0 2024-04-25T12:34:49.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.075+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0080afb00 con 0x7ff0080c82b0 2024-04-25T12:34:49.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.076+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00806f950 con 0x7ff0080c82b0 2024-04-25T12:34:49.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.077+0000 7feffe000640 1 -- 172.21.15.123:0/2399276460 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feff8003610 con 0x7ff0080c82b0 2024-04-25T12:34:49.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.078+0000 7feffe000640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefe40744e0 0x7fefe40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.078+0000 7feffe000640 1 -- 172.21.15.123:0/2399276460 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7feff80b5f40 con 0x7ff0080c82b0 2024-04-25T12:34:49.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.079+0000 7fefffe00640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefe40744e0 0x7fefe40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.080+0000 7fefffe00640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefe40744e0 0x7fefe40769f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff008159b60 tx=0x7feff0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:49.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.083+0000 7feffe000640 1 -- 172.21.15.123:0/2399276460 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff00806f950 con 0x7ff0080c82b0 2024-04-25T12:34:49.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.287+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0080b3ad0 con 0x7fefe40744e0 2024-04-25T12:34:49.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.297+0000 7feffe000640 1 -- 172.21.15.123:0/2399276460 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332599 (secure 0 0 0) 0x7ff0080b3ad0 con 0x7fefe40744e0 2024-04-25T12:34:49.300 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:49.304 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefe40744e0 msgr2=0x7fefe40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefe40744e0 0x7fefe40769f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff008159b60 tx=0x7feff0009190 comp rx=0 tx=0).stop 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c82b0 msgr2=0x7ff0080b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c82b0 0x7ff0080b1360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff80046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 shutdown_connections 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefe40744e0 0x7fefe40769f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff00815e590 0x7ff008150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0080c9080 0x7ff008155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 --2- 172.21.15.123:0/2399276460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0080c82b0 0x7ff0080b1360 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.304+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 >> 172.21.15.123:0/2399276460 conn(0x7ff0080cd600 msgr2=0x7ff0080c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:49.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.305+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 shutdown_connections 2024-04-25T12:34:49.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.305+0000 7ff007200640 1 -- 172.21.15.123:0/2399276460 wait complete. 2024-04-25T12:34:49.330 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T12:34:48.800172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218376,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756344,"statfs":{"total":773094113280,"available":772870496256,"internally_reserved":0,"allocated":2260992,"data_stored":2013480,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003705"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783414+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:34:19.783414+0000","last_peered":"2024-04-25T12:34:19.783414+0000","last_clean":"2024-04-25T12:34:19.783414+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:34:19.783414+0000","last_undegraded":"2024-04-25T12:34:19.783414+0000","last_fullsized":"2024-04-25T12:34:19.783414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782890+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:34:19.782890+0000","last_peered":"2024-04-25T12:34:19.782890+0000","last_clean":"2024-04-25T12:34:19.782890+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:34:19.782890+0000","last_undegraded":"2024-04-25T12:34:19.782890+0000","last_fullsized":"2024-04-25T12:34:19.782890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780793+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:34:19.780793+0000","last_peered":"2024-04-25T12:34:19.780793+0000","last_clean":"2024-04-25T12:34:19.780793+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:34:19.780793+0000","last_undegraded":"2024-04-25T12:34:19.780793+0000","last_fullsized":"2024-04-25T12:34:19.780793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782997+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:34:19.782997+0000","last_peered":"2024-04-25T12:34:19.782997+0000","last_clean":"2024-04-25T12:34:19.782997+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:34:19.782997+0000","last_undegraded":"2024-04-25T12:34:19.782997+0000","last_fullsized":"2024-04-25T12:34:19.782997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782606+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:34:19.782606+0000","last_peered":"2024-04-25T12:34:19.782606+0000","last_clean":"2024-04-25T12:34:19.782606+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:34:19.782606+0000","last_undegraded":"2024-04-25T12:34:19.782606+0000","last_fullsized":"2024-04-25T12:34:19.782606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784500+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:34:19.784500+0000","last_peered":"2024-04-25T12:34:19.784500+0000","last_clean":"2024-04-25T12:34:19.784500+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:34:19.784500+0000","last_undegraded":"2024-04-25T12:34:19.784500+0000","last_fullsized":"2024-04-25T12:34:19.784500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785563+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:34:19.785563+0000","last_peered":"2024-04-25T12:34:19.785563+0000","last_clean":"2024-04-25T12:34:19.785563+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:34:19.785563+0000","last_undegraded":"2024-04-25T12:34:19.785563+0000","last_fullsized":"2024-04-25T12:34:19.785563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.790406+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:34:19.790406+0000","last_peered":"2024-04-25T12:34:19.790406+0000","last_clean":"2024-04-25T12:34:19.790406+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:34:19.790406+0000","last_undegraded":"2024-04-25T12:34:19.790406+0000","last_fullsized":"2024-04-25T12:34:19.790406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788036+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:34:19.788036+0000","last_peered":"2024-04-25T12:34:19.788036+0000","last_clean":"2024-04-25T12:34:19.788036+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:34:19.788036+0000","last_undegraded":"2024-04-25T12:34:19.788036+0000","last_fullsized":"2024-04-25T12:34:19.788036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784075+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:34:19.784075+0000","last_peered":"2024-04-25T12:34:19.784075+0000","last_clean":"2024-04-25T12:34:19.784075+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:34:19.784075+0000","last_undegraded":"2024-04-25T12:34:19.784075+0000","last_fullsized":"2024-04-25T12:34:19.784075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982844+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:34:18.982844+0000","last_peered":"2024-04-25T12:34:18.982844+0000","last_clean":"2024-04-25T12:34:18.982844+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:34:18.982844+0000","last_undegraded":"2024-04-25T12:34:18.982844+0000","last_fullsized":"2024-04-25T12:34:18.982844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786691+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:34:19.786691+0000","last_peered":"2024-04-25T12:34:19.786691+0000","last_clean":"2024-04-25T12:34:19.786691+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:34:19.786691+0000","last_undegraded":"2024-04-25T12:34:19.786691+0000","last_fullsized":"2024-04-25T12:34:19.786691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982590+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:34:18.982590+0000","last_peered":"2024-04-25T12:34:18.982590+0000","last_clean":"2024-04-25T12:34:18.982590+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:34:18.982590+0000","last_undegraded":"2024-04-25T12:34:18.982590+0000","last_fullsized":"2024-04-25T12:34:18.982590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983302+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:34:18.983302+0000","last_peered":"2024-04-25T12:34:18.983302+0000","last_clean":"2024-04-25T12:34:18.983302+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:34:18.983302+0000","last_undegraded":"2024-04-25T12:34:18.983302+0000","last_fullsized":"2024-04-25T12:34:18.983302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982528+0000","last_change":"2024-04-25T12:34:18.007956+0000","last_active":"2024-04-25T12:34:18.982528+0000","last_peered":"2024-04-25T12:34:18.982528+0000","last_clean":"2024-04-25T12:34:18.982528+0000","last_became_active":"2024-04-25T12:34:18.007500+0000","last_became_peered":"2024-04-25T12:34:18.007500+0000","last_unstale":"2024-04-25T12:34:18.982528+0000","last_undegraded":"2024-04-25T12:34:18.982528+0000","last_fullsized":"2024-04-25T12:34:18.982528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:56.400953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782265+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:34:19.782265+0000","last_peered":"2024-04-25T12:34:19.782265+0000","last_clean":"2024-04-25T12:34:19.782265+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:34:19.782265+0000","last_undegraded":"2024-04-25T12:34:19.782265+0000","last_fullsized":"2024-04-25T12:34:19.782265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980152+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:34:18.980152+0000","last_peered":"2024-04-25T12:34:18.980152+0000","last_clean":"2024-04-25T12:34:18.980152+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:34:18.980152+0000","last_undegraded":"2024-04-25T12:34:18.980152+0000","last_fullsized":"2024-04-25T12:34:18.980152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783204+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:34:19.783204+0000","last_peered":"2024-04-25T12:34:19.783204+0000","last_clean":"2024-04-25T12:34:19.783204+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:34:19.783204+0000","last_undegraded":"2024-04-25T12:34:19.783204+0000","last_fullsized":"2024-04-25T12:34:19.783204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782958+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:34:19.782958+0000","last_peered":"2024-04-25T12:34:19.782958+0000","last_clean":"2024-04-25T12:34:19.782958+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:34:19.782958+0000","last_undegraded":"2024-04-25T12:34:19.782958+0000","last_fullsized":"2024-04-25T12:34:19.782958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781589+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:34:19.781589+0000","last_peered":"2024-04-25T12:34:19.781589+0000","last_clean":"2024-04-25T12:34:19.781589+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:34:19.781589+0000","last_undegraded":"2024-04-25T12:34:19.781589+0000","last_fullsized":"2024-04-25T12:34:19.781589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781948+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:34:19.781948+0000","last_peered":"2024-04-25T12:34:19.781948+0000","last_clean":"2024-04-25T12:34:19.781948+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:34:19.781948+0000","last_undegraded":"2024-04-25T12:34:19.781948+0000","last_fullsized":"2024-04-25T12:34:19.781948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981753+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:34:18.981753+0000","last_peered":"2024-04-25T12:34:18.981753+0000","last_clean":"2024-04-25T12:34:18.981753+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:34:18.981753+0000","last_undegraded":"2024-04-25T12:34:18.981753+0000","last_fullsized":"2024-04-25T12:34:18.981753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782248+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:34:19.782248+0000","last_peered":"2024-04-25T12:34:19.782248+0000","last_clean":"2024-04-25T12:34:19.782248+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:34:19.782248+0000","last_undegraded":"2024-04-25T12:34:19.782248+0000","last_fullsized":"2024-04-25T12:34:19.782248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788041+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:34:19.788041+0000","last_peered":"2024-04-25T12:34:19.788041+0000","last_clean":"2024-04-25T12:34:19.788041+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:34:19.788041+0000","last_undegraded":"2024-04-25T12:34:19.788041+0000","last_fullsized":"2024-04-25T12:34:19.788041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783014+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:34:19.783014+0000","last_peered":"2024-04-25T12:34:19.783014+0000","last_clean":"2024-04-25T12:34:19.783014+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:34:19.783014+0000","last_undegraded":"2024-04-25T12:34:19.783014+0000","last_fullsized":"2024-04-25T12:34:19.783014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983538+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:34:18.983538+0000","last_peered":"2024-04-25T12:34:18.983538+0000","last_clean":"2024-04-25T12:34:18.983538+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:34:18.983538+0000","last_undegraded":"2024-04-25T12:34:18.983538+0000","last_fullsized":"2024-04-25T12:34:18.983538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787014+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:34:19.787014+0000","last_peered":"2024-04-25T12:34:19.787014+0000","last_clean":"2024-04-25T12:34:19.787014+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:34:19.787014+0000","last_undegraded":"2024-04-25T12:34:19.787014+0000","last_fullsized":"2024-04-25T12:34:19.787014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783329+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:34:19.783329+0000","last_peered":"2024-04-25T12:34:19.783329+0000","last_clean":"2024-04-25T12:34:19.783329+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:34:19.783329+0000","last_undegraded":"2024-04-25T12:34:19.783329+0000","last_fullsized":"2024-04-25T12:34:19.783329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783247+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:34:19.783247+0000","last_peered":"2024-04-25T12:34:19.783247+0000","last_clean":"2024-04-25T12:34:19.783247+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:34:19.783247+0000","last_undegraded":"2024-04-25T12:34:19.783247+0000","last_fullsized":"2024-04-25T12:34:19.783247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781103+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:34:19.781103+0000","last_peered":"2024-04-25T12:34:19.781103+0000","last_clean":"2024-04-25T12:34:19.781103+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:34:19.781103+0000","last_undegraded":"2024-04-25T12:34:19.781103+0000","last_fullsized":"2024-04-25T12:34:19.781103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980759+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:34:18.980759+0000","last_peered":"2024-04-25T12:34:18.980759+0000","last_clean":"2024-04-25T12:34:18.980759+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:34:18.980759+0000","last_undegraded":"2024-04-25T12:34:18.980759+0000","last_fullsized":"2024-04-25T12:34:18.980759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983863+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:34:18.983863+0000","last_peered":"2024-04-25T12:34:18.983863+0000","last_clean":"2024-04-25T12:34:18.983863+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:34:18.983863+0000","last_undegraded":"2024-04-25T12:34:18.983863+0000","last_fullsized":"2024-04-25T12:34:18.983863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980463+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:34:18.980463+0000","last_peered":"2024-04-25T12:34:18.980463+0000","last_clean":"2024-04-25T12:34:18.980463+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:34:18.980463+0000","last_undegraded":"2024-04-25T12:34:18.980463+0000","last_fullsized":"2024-04-25T12:34:18.980463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789572+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:34:19.789572+0000","last_peered":"2024-04-25T12:34:19.789572+0000","last_clean":"2024-04-25T12:34:19.789572+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:34:19.789572+0000","last_undegraded":"2024-04-25T12:34:19.789572+0000","last_fullsized":"2024-04-25T12:34:19.789572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782068+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:34:19.782068+0000","last_peered":"2024-04-25T12:34:19.782068+0000","last_clean":"2024-04-25T12:34:19.782068+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:34:19.782068+0000","last_undegraded":"2024-04-25T12:34:19.782068+0000","last_fullsized":"2024-04-25T12:34:19.782068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979987+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:34:18.979987+0000","last_peered":"2024-04-25T12:34:18.979987+0000","last_clean":"2024-04-25T12:34:18.979987+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:34:18.979987+0000","last_undegraded":"2024-04-25T12:34:18.979987+0000","last_fullsized":"2024-04-25T12:34:18.979987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783310+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:34:19.783310+0000","last_peered":"2024-04-25T12:34:19.783310+0000","last_clean":"2024-04-25T12:34:19.783310+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:34:19.783310+0000","last_undegraded":"2024-04-25T12:34:19.783310+0000","last_fullsized":"2024-04-25T12:34:19.783310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982842+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:34:18.982842+0000","last_peered":"2024-04-25T12:34:18.982842+0000","last_clean":"2024-04-25T12:34:18.982842+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:34:18.982842+0000","last_undegraded":"2024-04-25T12:34:18.982842+0000","last_fullsized":"2024-04-25T12:34:18.982842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980473+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:34:18.980473+0000","last_peered":"2024-04-25T12:34:18.980473+0000","last_clean":"2024-04-25T12:34:18.980473+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:34:18.980473+0000","last_undegraded":"2024-04-25T12:34:18.980473+0000","last_fullsized":"2024-04-25T12:34:18.980473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781093+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:34:19.781093+0000","last_peered":"2024-04-25T12:34:19.781093+0000","last_clean":"2024-04-25T12:34:19.781093+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:34:19.781093+0000","last_undegraded":"2024-04-25T12:34:19.781093+0000","last_fullsized":"2024-04-25T12:34:19.781093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781452+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:34:19.781452+0000","last_peered":"2024-04-25T12:34:19.781452+0000","last_clean":"2024-04-25T12:34:19.781452+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:34:19.781452+0000","last_undegraded":"2024-04-25T12:34:19.781452+0000","last_fullsized":"2024-04-25T12:34:19.781452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980445+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:34:18.980445+0000","last_peered":"2024-04-25T12:34:18.980445+0000","last_clean":"2024-04-25T12:34:18.980445+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:34:18.980445+0000","last_undegraded":"2024-04-25T12:34:18.980445+0000","last_fullsized":"2024-04-25T12:34:18.980445+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979799+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:34:18.979799+0000","last_peered":"2024-04-25T12:34:18.979799+0000","last_clean":"2024-04-25T12:34:18.979799+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:34:18.979799+0000","last_undegraded":"2024-04-25T12:34:18.979799+0000","last_fullsized":"2024-04-25T12:34:18.979799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982431+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:34:18.982431+0000","last_peered":"2024-04-25T12:34:18.982431+0000","last_clean":"2024-04-25T12:34:18.982431+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:34:18.982431+0000","last_undegraded":"2024-04-25T12:34:18.982431+0000","last_fullsized":"2024-04-25T12:34:18.982431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782825+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:34:19.782825+0000","last_peered":"2024-04-25T12:34:19.782825+0000","last_clean":"2024-04-25T12:34:19.782825+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:34:19.782825+0000","last_undegraded":"2024-04-25T12:34:19.782825+0000","last_fullsized":"2024-04-25T12:34:19.782825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783355+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:34:19.783355+0000","last_peered":"2024-04-25T12:34:19.783355+0000","last_clean":"2024-04-25T12:34:19.783355+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:34:19.783355+0000","last_undegraded":"2024-04-25T12:34:19.783355+0000","last_fullsized":"2024-04-25T12:34:19.783355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983351+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:34:18.983351+0000","last_peered":"2024-04-25T12:34:18.983351+0000","last_clean":"2024-04-25T12:34:18.983351+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:34:18.983351+0000","last_undegraded":"2024-04-25T12:34:18.983351+0000","last_fullsized":"2024-04-25T12:34:18.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787419+0000","last_change":"2024-04-25T12:34:08.560989+0000","last_active":"2024-04-25T12:34:19.787419+0000","last_peered":"2024-04-25T12:34:19.787419+0000","last_clean":"2024-04-25T12:34:19.787419+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:34:19.787419+0000","last_undegraded":"2024-04-25T12:34:19.787419+0000","last_fullsized":"2024-04-25T12:34:19.787419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:49.130809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983747+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:34:18.983747+0000","last_peered":"2024-04-25T12:34:18.983747+0000","last_clean":"2024-04-25T12:34:18.983747+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:34:18.983747+0000","last_undegraded":"2024-04-25T12:34:18.983747+0000","last_fullsized":"2024-04-25T12:34:18.983747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982942+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:34:18.982942+0000","last_peered":"2024-04-25T12:34:18.982942+0000","last_clean":"2024-04-25T12:34:18.982942+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:34:18.982942+0000","last_undegraded":"2024-04-25T12:34:18.982942+0000","last_fullsized":"2024-04-25T12:34:18.982942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980766+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:34:18.980766+0000","last_peered":"2024-04-25T12:34:18.980766+0000","last_clean":"2024-04-25T12:34:18.980766+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:34:18.980766+0000","last_undegraded":"2024-04-25T12:34:18.980766+0000","last_fullsized":"2024-04-25T12:34:18.980766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786888+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:34:19.786888+0000","last_peered":"2024-04-25T12:34:19.786888+0000","last_clean":"2024-04-25T12:34:19.786888+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:34:19.786888+0000","last_undegraded":"2024-04-25T12:34:19.786888+0000","last_fullsized":"2024-04-25T12:34:19.786888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783906+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:34:19.783906+0000","last_peered":"2024-04-25T12:34:19.783906+0000","last_clean":"2024-04-25T12:34:19.783906+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:34:19.783906+0000","last_undegraded":"2024-04-25T12:34:19.783906+0000","last_fullsized":"2024-04-25T12:34:19.783906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782493+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:34:19.782493+0000","last_peered":"2024-04-25T12:34:19.782493+0000","last_clean":"2024-04-25T12:34:19.782493+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:34:19.782493+0000","last_undegraded":"2024-04-25T12:34:19.782493+0000","last_fullsized":"2024-04-25T12:34:19.782493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781393+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:34:19.781393+0000","last_peered":"2024-04-25T12:34:19.781393+0000","last_clean":"2024-04-25T12:34:19.781393+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:34:19.781393+0000","last_undegraded":"2024-04-25T12:34:19.781393+0000","last_fullsized":"2024-04-25T12:34:19.781393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785529+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:34:19.785529+0000","last_peered":"2024-04-25T12:34:19.785529+0000","last_clean":"2024-04-25T12:34:19.785529+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:34:19.785529+0000","last_undegraded":"2024-04-25T12:34:19.785529+0000","last_fullsized":"2024-04-25T12:34:19.785529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783018+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:34:19.783018+0000","last_peered":"2024-04-25T12:34:19.783018+0000","last_clean":"2024-04-25T12:34:19.783018+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:34:19.783018+0000","last_undegraded":"2024-04-25T12:34:19.783018+0000","last_fullsized":"2024-04-25T12:34:19.783018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981441+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:34:18.981441+0000","last_peered":"2024-04-25T12:34:18.981441+0000","last_clean":"2024-04-25T12:34:18.981441+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:34:18.981441+0000","last_undegraded":"2024-04-25T12:34:18.981441+0000","last_fullsized":"2024-04-25T12:34:18.981441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788784+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:34:19.788784+0000","last_peered":"2024-04-25T12:34:19.788784+0000","last_clean":"2024-04-25T12:34:19.788784+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:34:19.788784+0000","last_undegraded":"2024-04-25T12:34:19.788784+0000","last_fullsized":"2024-04-25T12:34:19.788784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785856+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:34:19.785856+0000","last_peered":"2024-04-25T12:34:19.785856+0000","last_clean":"2024-04-25T12:34:19.785856+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:34:19.785856+0000","last_undegraded":"2024-04-25T12:34:19.785856+0000","last_fullsized":"2024-04-25T12:34:19.785856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788439+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:34:19.788439+0000","last_peered":"2024-04-25T12:34:19.788439+0000","last_clean":"2024-04-25T12:34:19.788439+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:34:19.788439+0000","last_undegraded":"2024-04-25T12:34:19.788439+0000","last_fullsized":"2024-04-25T12:34:19.788439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981709+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:34:18.981709+0000","last_peered":"2024-04-25T12:34:18.981709+0000","last_clean":"2024-04-25T12:34:18.981709+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:34:18.981709+0000","last_undegraded":"2024-04-25T12:34:18.981709+0000","last_fullsized":"2024-04-25T12:34:18.981709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781190+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:34:19.781190+0000","last_peered":"2024-04-25T12:34:19.781190+0000","last_clean":"2024-04-25T12:34:19.781190+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:34:19.781190+0000","last_undegraded":"2024-04-25T12:34:19.781190+0000","last_fullsized":"2024-04-25T12:34:19.781190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782909+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:34:19.782909+0000","last_peered":"2024-04-25T12:34:19.782909+0000","last_clean":"2024-04-25T12:34:19.782909+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:34:19.782909+0000","last_undegraded":"2024-04-25T12:34:19.782909+0000","last_fullsized":"2024-04-25T12:34:19.782909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786692+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:34:19.786692+0000","last_peered":"2024-04-25T12:34:19.786692+0000","last_clean":"2024-04-25T12:34:19.786692+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:34:19.786692+0000","last_undegraded":"2024-04-25T12:34:19.786692+0000","last_fullsized":"2024-04-25T12:34:19.786692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782692+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:34:19.782692+0000","last_peered":"2024-04-25T12:34:19.782692+0000","last_clean":"2024-04-25T12:34:19.782692+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:34:19.782692+0000","last_undegraded":"2024-04-25T12:34:19.782692+0000","last_fullsized":"2024-04-25T12:34:19.782692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782809+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:34:19.782809+0000","last_peered":"2024-04-25T12:34:19.782809+0000","last_clean":"2024-04-25T12:34:19.782809+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:34:19.782809+0000","last_undegraded":"2024-04-25T12:34:19.782809+0000","last_fullsized":"2024-04-25T12:34:19.782809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983105+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:34:18.983105+0000","last_peered":"2024-04-25T12:34:18.983105+0000","last_clean":"2024-04-25T12:34:18.983105+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:34:18.983105+0000","last_undegraded":"2024-04-25T12:34:18.983105+0000","last_fullsized":"2024-04-25T12:34:18.983105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982716+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:34:18.982716+0000","last_peered":"2024-04-25T12:34:18.982716+0000","last_clean":"2024-04-25T12:34:18.982716+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:34:18.982716+0000","last_undegraded":"2024-04-25T12:34:18.982716+0000","last_fullsized":"2024-04-25T12:34:18.982716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980674+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:34:18.980674+0000","last_peered":"2024-04-25T12:34:18.980674+0000","last_clean":"2024-04-25T12:34:18.980674+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:34:18.980674+0000","last_undegraded":"2024-04-25T12:34:18.980674+0000","last_fullsized":"2024-04-25T12:34:18.980674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980038+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:34:18.980038+0000","last_peered":"2024-04-25T12:34:18.980038+0000","last_clean":"2024-04-25T12:34:18.980038+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:34:18.980038+0000","last_undegraded":"2024-04-25T12:34:18.980038+0000","last_fullsized":"2024-04-25T12:34:18.980038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782400+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:34:19.782400+0000","last_peered":"2024-04-25T12:34:19.782400+0000","last_clean":"2024-04-25T12:34:19.782400+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:34:19.782400+0000","last_undegraded":"2024-04-25T12:34:19.782400+0000","last_fullsized":"2024-04-25T12:34:19.782400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982809+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:34:18.982809+0000","last_peered":"2024-04-25T12:34:18.982809+0000","last_clean":"2024-04-25T12:34:18.982809+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:34:18.982809+0000","last_undegraded":"2024-04-25T12:34:18.982809+0000","last_fullsized":"2024-04-25T12:34:18.982809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784989+0000","last_change":"2024-04-25T12:34:13.973917+0000","last_active":"2024-04-25T12:34:19.784989+0000","last_peered":"2024-04-25T12:34:19.784989+0000","last_clean":"2024-04-25T12:34:19.784989+0000","last_became_active":"2024-04-25T12:34:13.973640+0000","last_became_peered":"2024-04-25T12:34:13.973640+0000","last_unstale":"2024-04-25T12:34:19.784989+0000","last_undegraded":"2024-04-25T12:34:19.784989+0000","last_fullsized":"2024-04-25T12:34:19.784989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:21.717728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:36.068337+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:34:36.068337+0000","last_peered":"2024-04-25T12:34:36.068337+0000","last_clean":"2024-04-25T12:34:36.068337+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:34:36.068337+0000","last_undegraded":"2024-04-25T12:34:36.068337+0000","last_fullsized":"2024-04-25T12:34:36.068337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783277+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:34:19.783277+0000","last_peered":"2024-04-25T12:34:19.783277+0000","last_clean":"2024-04-25T12:34:19.783277+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:34:19.783277+0000","last_undegraded":"2024-04-25T12:34:19.783277+0000","last_fullsized":"2024-04-25T12:34:19.783277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982241+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:34:18.982241+0000","last_peered":"2024-04-25T12:34:18.982241+0000","last_clean":"2024-04-25T12:34:18.982241+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:34:18.982241+0000","last_undegraded":"2024-04-25T12:34:18.982241+0000","last_fullsized":"2024-04-25T12:34:18.982241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782037+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:34:19.782037+0000","last_peered":"2024-04-25T12:34:19.782037+0000","last_clean":"2024-04-25T12:34:19.782037+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:34:19.782037+0000","last_undegraded":"2024-04-25T12:34:19.782037+0000","last_fullsized":"2024-04-25T12:34:19.782037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788580+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:34:19.788580+0000","last_peered":"2024-04-25T12:34:19.788580+0000","last_clean":"2024-04-25T12:34:19.788580+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:34:19.788580+0000","last_undegraded":"2024-04-25T12:34:19.788580+0000","last_fullsized":"2024-04-25T12:34:19.788580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782777+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:34:19.782777+0000","last_peered":"2024-04-25T12:34:19.782777+0000","last_clean":"2024-04-25T12:34:19.782777+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:34:19.782777+0000","last_undegraded":"2024-04-25T12:34:19.782777+0000","last_fullsized":"2024-04-25T12:34:19.782777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784953+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:34:19.784953+0000","last_peered":"2024-04-25T12:34:19.784953+0000","last_clean":"2024-04-25T12:34:19.784953+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:34:19.784953+0000","last_undegraded":"2024-04-25T12:34:19.784953+0000","last_fullsized":"2024-04-25T12:34:19.784953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780643+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:34:19.780643+0000","last_peered":"2024-04-25T12:34:19.780643+0000","last_clean":"2024-04-25T12:34:19.780643+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:34:19.780643+0000","last_undegraded":"2024-04-25T12:34:19.780643+0000","last_fullsized":"2024-04-25T12:34:19.780643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788514+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:34:19.788514+0000","last_peered":"2024-04-25T12:34:19.788514+0000","last_clean":"2024-04-25T12:34:19.788514+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:34:19.788514+0000","last_undegraded":"2024-04-25T12:34:19.788514+0000","last_fullsized":"2024-04-25T12:34:19.788514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.791187+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:34:19.791187+0000","last_peered":"2024-04-25T12:34:19.791187+0000","last_clean":"2024-04-25T12:34:19.791187+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:34:19.791187+0000","last_undegraded":"2024-04-25T12:34:19.791187+0000","last_fullsized":"2024-04-25T12:34:19.791187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781414+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:34:19.781414+0000","last_peered":"2024-04-25T12:34:19.781414+0000","last_clean":"2024-04-25T12:34:19.781414+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:34:19.781414+0000","last_undegraded":"2024-04-25T12:34:19.781414+0000","last_fullsized":"2024-04-25T12:34:19.781414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782882+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:34:19.782882+0000","last_peered":"2024-04-25T12:34:19.782882+0000","last_clean":"2024-04-25T12:34:19.782882+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:34:19.782882+0000","last_undegraded":"2024-04-25T12:34:19.782882+0000","last_fullsized":"2024-04-25T12:34:19.782882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781481+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:34:19.781481+0000","last_peered":"2024-04-25T12:34:19.781481+0000","last_clean":"2024-04-25T12:34:19.781481+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:34:19.781481+0000","last_undegraded":"2024-04-25T12:34:19.781481+0000","last_fullsized":"2024-04-25T12:34:19.781481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982283+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:34:18.982283+0000","last_peered":"2024-04-25T12:34:18.982283+0000","last_clean":"2024-04-25T12:34:18.982283+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:34:18.982283+0000","last_undegraded":"2024-04-25T12:34:18.982283+0000","last_fullsized":"2024-04-25T12:34:18.982283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983399+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:34:18.983399+0000","last_peered":"2024-04-25T12:34:18.983399+0000","last_clean":"2024-04-25T12:34:18.983399+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:34:18.983399+0000","last_undegraded":"2024-04-25T12:34:18.983399+0000","last_fullsized":"2024-04-25T12:34:18.983399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782668+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:34:19.782668+0000","last_peered":"2024-04-25T12:34:19.782668+0000","last_clean":"2024-04-25T12:34:19.782668+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:34:19.782668+0000","last_undegraded":"2024-04-25T12:34:19.782668+0000","last_fullsized":"2024-04-25T12:34:19.782668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979833+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:34:18.979833+0000","last_peered":"2024-04-25T12:34:18.979833+0000","last_clean":"2024-04-25T12:34:18.979833+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:34:18.979833+0000","last_undegraded":"2024-04-25T12:34:18.979833+0000","last_fullsized":"2024-04-25T12:34:18.979833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982033+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:34:18.982033+0000","last_peered":"2024-04-25T12:34:18.982033+0000","last_clean":"2024-04-25T12:34:18.982033+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:34:18.982033+0000","last_undegraded":"2024-04-25T12:34:18.982033+0000","last_fullsized":"2024-04-25T12:34:18.982033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783137+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:34:19.783137+0000","last_peered":"2024-04-25T12:34:19.783137+0000","last_clean":"2024-04-25T12:34:19.783137+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:34:19.783137+0000","last_undegraded":"2024-04-25T12:34:19.783137+0000","last_fullsized":"2024-04-25T12:34:19.783137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782539+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:34:19.782539+0000","last_peered":"2024-04-25T12:34:19.782539+0000","last_clean":"2024-04-25T12:34:19.782539+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:34:19.782539+0000","last_undegraded":"2024-04-25T12:34:19.782539+0000","last_fullsized":"2024-04-25T12:34:19.782539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781454+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:34:19.781454+0000","last_peered":"2024-04-25T12:34:19.781454+0000","last_clean":"2024-04-25T12:34:19.781454+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:34:19.781454+0000","last_undegraded":"2024-04-25T12:34:19.781454+0000","last_fullsized":"2024-04-25T12:34:19.781454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789165+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:34:19.789165+0000","last_peered":"2024-04-25T12:34:19.789165+0000","last_clean":"2024-04-25T12:34:19.789165+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:34:19.789165+0000","last_undegraded":"2024-04-25T12:34:19.789165+0000","last_fullsized":"2024-04-25T12:34:19.789165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781753+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:34:19.781753+0000","last_peered":"2024-04-25T12:34:19.781753+0000","last_clean":"2024-04-25T12:34:19.781753+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:34:19.781753+0000","last_undegraded":"2024-04-25T12:34:19.781753+0000","last_fullsized":"2024-04-25T12:34:19.781753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785020+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:34:19.785020+0000","last_peered":"2024-04-25T12:34:19.785020+0000","last_clean":"2024-04-25T12:34:19.785020+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:34:19.785020+0000","last_undegraded":"2024-04-25T12:34:19.785020+0000","last_fullsized":"2024-04-25T12:34:19.785020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781340+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:34:19.781340+0000","last_peered":"2024-04-25T12:34:19.781340+0000","last_clean":"2024-04-25T12:34:19.781340+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:34:19.781340+0000","last_undegraded":"2024-04-25T12:34:19.781340+0000","last_fullsized":"2024-04-25T12:34:19.781340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981734+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:34:18.981734+0000","last_peered":"2024-04-25T12:34:18.981734+0000","last_clean":"2024-04-25T12:34:18.981734+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:34:18.981734+0000","last_undegraded":"2024-04-25T12:34:18.981734+0000","last_fullsized":"2024-04-25T12:34:18.981734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781427+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:34:19.781427+0000","last_peered":"2024-04-25T12:34:19.781427+0000","last_clean":"2024-04-25T12:34:19.781427+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:34:19.781427+0000","last_undegraded":"2024-04-25T12:34:19.781427+0000","last_fullsized":"2024-04-25T12:34:19.781427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980630+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:34:18.980630+0000","last_peered":"2024-04-25T12:34:18.980630+0000","last_clean":"2024-04-25T12:34:18.980630+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:34:18.980630+0000","last_undegraded":"2024-04-25T12:34:18.980630+0000","last_fullsized":"2024-04-25T12:34:18.980630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980186+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:34:18.980186+0000","last_peered":"2024-04-25T12:34:18.980186+0000","last_clean":"2024-04-25T12:34:18.980186+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:34:18.980186+0000","last_undegraded":"2024-04-25T12:34:18.980186+0000","last_fullsized":"2024-04-25T12:34:18.980186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788210+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:34:19.788210+0000","last_peered":"2024-04-25T12:34:19.788210+0000","last_clean":"2024-04-25T12:34:19.788210+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:34:19.788210+0000","last_undegraded":"2024-04-25T12:34:19.788210+0000","last_fullsized":"2024-04-25T12:34:19.788210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782446+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:34:19.782446+0000","last_peered":"2024-04-25T12:34:19.782446+0000","last_clean":"2024-04-25T12:34:19.782446+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:34:19.782446+0000","last_undegraded":"2024-04-25T12:34:19.782446+0000","last_fullsized":"2024-04-25T12:34:19.782446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982148+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:34:18.982148+0000","last_peered":"2024-04-25T12:34:18.982148+0000","last_clean":"2024-04-25T12:34:18.982148+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:34:18.982148+0000","last_undegraded":"2024-04-25T12:34:18.982148+0000","last_fullsized":"2024-04-25T12:34:18.982148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781207+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:34:19.781207+0000","last_peered":"2024-04-25T12:34:19.781207+0000","last_clean":"2024-04-25T12:34:19.781207+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:34:19.781207+0000","last_undegraded":"2024-04-25T12:34:19.781207+0000","last_fullsized":"2024-04-25T12:34:19.781207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789184+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:34:19.789184+0000","last_peered":"2024-04-25T12:34:19.789184+0000","last_clean":"2024-04-25T12:34:19.789184+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:34:19.789184+0000","last_undegraded":"2024-04-25T12:34:19.789184+0000","last_fullsized":"2024-04-25T12:34:19.789184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783090+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:34:19.783090+0000","last_peered":"2024-04-25T12:34:19.783090+0000","last_clean":"2024-04-25T12:34:19.783090+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:34:19.783090+0000","last_undegraded":"2024-04-25T12:34:19.783090+0000","last_fullsized":"2024-04-25T12:34:19.783090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783799+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:34:19.783799+0000","last_peered":"2024-04-25T12:34:19.783799+0000","last_clean":"2024-04-25T12:34:19.783799+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:34:19.783799+0000","last_undegraded":"2024-04-25T12:34:19.783799+0000","last_fullsized":"2024-04-25T12:34:19.783799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983704+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:34:18.983704+0000","last_peered":"2024-04-25T12:34:18.983704+0000","last_clean":"2024-04-25T12:34:18.983704+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:34:18.983704+0000","last_undegraded":"2024-04-25T12:34:18.983704+0000","last_fullsized":"2024-04-25T12:34:18.983704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782288+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:34:19.782288+0000","last_peered":"2024-04-25T12:34:19.782288+0000","last_clean":"2024-04-25T12:34:19.782288+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:34:19.782288+0000","last_undegraded":"2024-04-25T12:34:19.782288+0000","last_fullsized":"2024-04-25T12:34:19.782288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983392+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:34:18.983392+0000","last_peered":"2024-04-25T12:34:18.983392+0000","last_clean":"2024-04-25T12:34:18.983392+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:34:18.983392+0000","last_undegraded":"2024-04-25T12:34:18.983392+0000","last_fullsized":"2024-04-25T12:34:18.983392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981957+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:34:18.981957+0000","last_peered":"2024-04-25T12:34:18.981957+0000","last_clean":"2024-04-25T12:34:18.981957+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:34:18.981957+0000","last_undegraded":"2024-04-25T12:34:18.981957+0000","last_fullsized":"2024-04-25T12:34:18.981957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780879+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:34:19.780879+0000","last_peered":"2024-04-25T12:34:19.780879+0000","last_clean":"2024-04-25T12:34:19.780879+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:34:19.780879+0000","last_undegraded":"2024-04-25T12:34:19.780879+0000","last_fullsized":"2024-04-25T12:34:19.780879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979899+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:34:18.979899+0000","last_peered":"2024-04-25T12:34:18.979899+0000","last_clean":"2024-04-25T12:34:18.979899+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:34:18.979899+0000","last_undegraded":"2024-04-25T12:34:18.979899+0000","last_fullsized":"2024-04-25T12:34:18.979899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980556+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:34:18.980556+0000","last_peered":"2024-04-25T12:34:18.980556+0000","last_clean":"2024-04-25T12:34:18.980556+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:34:18.980556+0000","last_undegraded":"2024-04-25T12:34:18.980556+0000","last_fullsized":"2024-04-25T12:34:18.980556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783694+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:34:19.783694+0000","last_peered":"2024-04-25T12:34:19.783694+0000","last_clean":"2024-04-25T12:34:19.783694+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:34:19.783694+0000","last_undegraded":"2024-04-25T12:34:19.783694+0000","last_fullsized":"2024-04-25T12:34:19.783694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784671+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:34:19.784671+0000","last_peered":"2024-04-25T12:34:19.784671+0000","last_clean":"2024-04-25T12:34:19.784671+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:34:19.784671+0000","last_undegraded":"2024-04-25T12:34:19.784671+0000","last_fullsized":"2024-04-25T12:34:19.784671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784485+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:34:19.784485+0000","last_peered":"2024-04-25T12:34:19.784485+0000","last_clean":"2024-04-25T12:34:19.784485+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:34:19.784485+0000","last_undegraded":"2024-04-25T12:34:19.784485+0000","last_fullsized":"2024-04-25T12:34:19.784485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.791100+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:34:19.791100+0000","last_peered":"2024-04-25T12:34:19.791100+0000","last_clean":"2024-04-25T12:34:19.791100+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:34:19.791100+0000","last_undegraded":"2024-04-25T12:34:19.791100+0000","last_fullsized":"2024-04-25T12:34:19.791100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781305+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:34:19.781305+0000","last_peered":"2024-04-25T12:34:19.781305+0000","last_clean":"2024-04-25T12:34:19.781305+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:34:19.781305+0000","last_undegraded":"2024-04-25T12:34:19.781305+0000","last_fullsized":"2024-04-25T12:34:19.781305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782773+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:34:19.782773+0000","last_peered":"2024-04-25T12:34:19.782773+0000","last_clean":"2024-04-25T12:34:19.782773+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:34:19.782773+0000","last_undegraded":"2024-04-25T12:34:19.782773+0000","last_fullsized":"2024-04-25T12:34:19.782773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786973+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:34:19.786973+0000","last_peered":"2024-04-25T12:34:19.786973+0000","last_clean":"2024-04-25T12:34:19.786973+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:34:19.786973+0000","last_undegraded":"2024-04-25T12:34:19.786973+0000","last_fullsized":"2024-04-25T12:34:19.786973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783422+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:34:19.783422+0000","last_peered":"2024-04-25T12:34:19.783422+0000","last_clean":"2024-04-25T12:34:19.783422+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:34:19.783422+0000","last_undegraded":"2024-04-25T12:34:19.783422+0000","last_fullsized":"2024-04-25T12:34:19.783422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788118+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:34:19.788118+0000","last_peered":"2024-04-25T12:34:19.788118+0000","last_clean":"2024-04-25T12:34:19.788118+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:34:19.788118+0000","last_undegraded":"2024-04-25T12:34:19.788118+0000","last_fullsized":"2024-04-25T12:34:19.788118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782309+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:34:19.782309+0000","last_peered":"2024-04-25T12:34:19.782309+0000","last_clean":"2024-04-25T12:34:19.782309+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:34:19.782309+0000","last_undegraded":"2024-04-25T12:34:19.782309+0000","last_fullsized":"2024-04-25T12:34:19.782309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782643+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:34:19.782643+0000","last_peered":"2024-04-25T12:34:19.782643+0000","last_clean":"2024-04-25T12:34:19.782643+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:34:19.782643+0000","last_undegraded":"2024-04-25T12:34:19.782643+0000","last_fullsized":"2024-04-25T12:34:19.782643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785344+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:34:19.785344+0000","last_peered":"2024-04-25T12:34:19.785344+0000","last_clean":"2024-04-25T12:34:19.785344+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:34:19.785344+0000","last_undegraded":"2024-04-25T12:34:19.785344+0000","last_fullsized":"2024-04-25T12:34:19.785344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":103515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":693393,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":693371,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:34:49.335 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T12:34:49.335 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T12:34:49.335 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T12:34:49.336 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T12:34:49.336 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T12:34:49.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.646+0000 7fd943400640 1 Processor -- start 2024-04-25T12:34:49.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.646+0000 7fd943400640 1 -- start start 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.648+0000 7fd943400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd93c0c4d50 0x7fd93c154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd943400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd93c15afa0 0x7fd93c0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd943400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93c15b3b0 con 0x7fd93c074870 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd943400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd93c0c61f0 con 0x7fd93c15afa0 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd943400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd93c0c87d0 con 0x7fd93c0c4d50 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd941400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd93c15afa0 0x7fd93c0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd941400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd93c15afa0 0x7fd93c0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35284/0 (socket says 172.21.15.123:35284) 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd941400640 1 -- 172.21.15.123:0/3840681284 learned_addr learned my addr 172.21.15.123:0/3840681284 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd940a00640 1 --2- 172.21.15.123:0/3840681284 >> v2:172.21.15.153:3300/0 conn(0x7fd93c0c4d50 0x7fd93c154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd941400640 1 -- 172.21.15.123:0/3840681284 >> v2:172.21.15.153:3300/0 conn(0x7fd93c0c4d50 msgr2=0x7fd93c154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd941400640 1 --2- 172.21.15.123:0/3840681284 >> v2:172.21.15.153:3300/0 conn(0x7fd93c0c4d50 0x7fd93c154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd941400640 1 -- 172.21.15.123:0/3840681284 >> v1:172.21.15.153:6789/0 conn(0x7fd93c074870 legacy=0x7fd93c0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.649+0000 7fd93be00640 1 -- 172.21.15.123:0/3840681284 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918613759 0 0) 0x7fd93c15b3b0 con 0x7fd93c074870 2024-04-25T12:34:49.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.650+0000 7fd941400640 1 -- 172.21.15.123:0/3840681284 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c073ce0 con 0x7fd93c15afa0 2024-04-25T12:34:49.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.650+0000 7fd941400640 1 --2- 172.21.15.123:0/3840681284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c15afa0 0x7fd93c0cda90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0ce6d0 tx=0x7fd93402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a8df60ed1fef61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:49.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.651+0000 7fd93be00640 1 -- 172.21.15.123:0/3840681284 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934036070 con 0x7fd93c15afa0 2024-04-25T12:34:49.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.651+0000 7fd93be00640 1 -- 172.21.15.123:0/3840681284 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd934031600 con 0x7fd93c15afa0 2024-04-25T12:34:49.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.651+0000 7fd93be00640 1 -- 172.21.15.123:0/3840681284 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340319f0 con 0x7fd93c15afa0 2024-04-25T12:34:49.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 -- 172.21.15.123:0/3840681284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c15afa0 msgr2=0x7fd93c0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 --2- 172.21.15.123:0/3840681284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c15afa0 0x7fd93c0cda90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0ce6d0 tx=0x7fd93402a7c0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 -- 172.21.15.123:0/3840681284 shutdown_connections 2024-04-25T12:34:49.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 --2- 172.21.15.123:0/3840681284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c15afa0 0x7fd93c0cda90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 --2- 172.21.15.123:0/3840681284 >> v2:172.21.15.153:3300/0 conn(0x7fd93c0c4d50 0x7fd93c154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 -- 172.21.15.123:0/3840681284 >> 172.21.15.123:0/3840681284 conn(0x7fd93c06c750 msgr2=0x7fd93c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:49.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 -- 172.21.15.123:0/3840681284 shutdown_connections 2024-04-25T12:34:49.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.652+0000 7fd943400640 1 -- 172.21.15.123:0/3840681284 wait complete. 2024-04-25T12:34:49.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.654+0000 7fd943400640 1 Processor -- start 2024-04-25T12:34:49.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.654+0000 7fd943400640 1 -- start start 2024-04-25T12:34:49.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd943400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd93c074870 0x7fd93c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd943400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c0c4d50 0x7fd93c153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd943400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd93c15afa0 0x7fd93c0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd943400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c0c8460 con 0x7fd93c0c4d50 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd943400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c0742a0 con 0x7fd93c074870 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd943400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd93c0c9370 con 0x7fd93c15afa0 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd940a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd93c074870 0x7fd93c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd941400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd93c15afa0 0x7fd93c0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd933e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c0c4d50 0x7fd93c153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd940a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd93c074870 0x7fd93c159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55256/0 (socket says 172.21.15.123:55256) 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd940a00640 1 -- 172.21.15.123:0/49032690 learned_addr learned my addr 172.21.15.123:0/49032690 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:49.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.656+0000 7fd941400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd93c15afa0 0x7fd93c0ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35894/0 (socket says 172.21.15.123:35894) 2024-04-25T12:34:49.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd933e00640 1 -- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd93c15afa0 msgr2=0x7fd93c0ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd933e00640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd93c15afa0 0x7fd93c0ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd933e00640 1 -- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd93c074870 msgr2=0x7fd93c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd933e00640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd93c074870 0x7fd93c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd933e00640 1 -- 172.21.15.123:0/49032690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c1f6400 con 0x7fd93c0c4d50 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd940a00640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd93c074870 0x7fd93c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd941400640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd93c15afa0 0x7fd93c0ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd933e00640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c0c4d50 0x7fd93c153210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c410 tx=0x7fd92400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd939600640 1 -- 172.21.15.123:0/49032690 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924017070 con 0x7fd93c0c4d50 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd939600640 1 -- 172.21.15.123:0/49032690 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9240035f0 con 0x7fd93c0c4d50 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c0b21e0 con 0x7fd93c0c4d50 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd939600640 1 -- 172.21.15.123:0/49032690 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240039e0 con 0x7fd93c0c4d50 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.657+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c1f6670 con 0x7fd93c0c4d50 2024-04-25T12:34:49.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.659+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93c15a630 con 0x7fd93c0c4d50 2024-04-25T12:34:49.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.660+0000 7fd939600640 1 -- 172.21.15.123:0/49032690 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd92401a070 con 0x7fd93c0c4d50 2024-04-25T12:34:49.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.661+0000 7fd939600640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9180744e0 0x7fd9180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:49.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.661+0000 7fd939600640 1 -- 172.21.15.123:0/49032690 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd9240946d0 con 0x7fd93c0c4d50 2024-04-25T12:34:49.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.661+0000 7fd940a00640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9180744e0 0x7fd9180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:49.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.662+0000 7fd940a00640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9180744e0 0x7fd9180769f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009f50 tx=0x7fd92c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:49.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.666+0000 7fd939600640 1 -- 172.21.15.123:0/49032690 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd92405e6b0 con 0x7fd93c0c4d50 2024-04-25T12:34:49.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.908+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd93c076700 con 0x7fd93c0c4d50 2024-04-25T12:34:49.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.917+0000 7fd939600640 1 -- 172.21.15.123:0/49032690 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186749 (secure 0 0 0) 0x7fd9240629b0 con 0x7fd93c0c4d50 2024-04-25T12:34:49.920 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:49.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.921+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9180744e0 msgr2=0x7fd9180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9180744e0 0x7fd9180769f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009f50 tx=0x7fd92c009190 comp rx=0 tx=0).stop 2024-04-25T12:34:49.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c0c4d50 msgr2=0x7fd93c153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c0c4d50 0x7fd93c153210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c410 tx=0x7fd92400c810 comp rx=0 tx=0).stop 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 shutdown_connections 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9180744e0 0x7fd9180769f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd93c15afa0 0x7fd93c0ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93c0c4d50 0x7fd93c153210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 --2- 172.21.15.123:0/49032690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd93c074870 0x7fd93c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 >> 172.21.15.123:0/49032690 conn(0x7fd93c06c750 msgr2=0x7fd93c0c7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 shutdown_connections 2024-04-25T12:34:49.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:49.922+0000 7fd943400640 1 -- 172.21.15.123:0/49032690 wait complete. 2024-04-25T12:34:49.946 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2246116237},{"type":"v1","addr":"172.21.15.153:6801","nonce":2246116237}]},"active_addr":"172.21.15.153:6801/2246116237","active_change":"2024-04-25T12:33:58.779146+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2653746816}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2365170275}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":610976571}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2276010661}]}]} 2024-04-25T12:34:49.950 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T12:34:49.950 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T12:34:49.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:50.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.215+0000 7f51b3400640 1 Processor -- start 2024-04-25T12:34:50.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.215+0000 7f51b3400640 1 -- start start 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51b3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51ac074870 0x7f51ac154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51b3400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac15afa0 0x7f51ac0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51b3400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51ac15b3b0 con 0x7f51ac0c4d50 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51b3400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f51ac0c61f0 con 0x7f51ac074870 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51b3400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f51ac0c87d0 con 0x7f51ac15afa0 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51abe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51ac074870 0x7f51ac154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51abe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51ac074870 0x7f51ac154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35302/0 (socket says 172.21.15.123:35302) 2024-04-25T12:34:50.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51abe00640 1 -- 172.21.15.123:0/36561402 learned_addr learned my addr 172.21.15.123:0/36561402 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:50.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.217+0000 7f51b1400640 1 --2- 172.21.15.123:0/36561402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac15afa0 0x7f51ac0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.218+0000 7f51abe00640 1 -- 172.21.15.123:0/36561402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac15afa0 msgr2=0x7f51ac0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:50.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.218+0000 7f51abe00640 1 --2- 172.21.15.123:0/36561402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac15afa0 0x7f51ac0cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.218+0000 7f51ab400640 1 -- 172.21.15.123:0/36561402 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342876883 0 0) 0x7f51ac15b3b0 con 0x7f51ac0c4d50 2024-04-25T12:34:50.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.218+0000 7f51abe00640 1 -- 172.21.15.123:0/36561402 >> v1:172.21.15.153:6789/0 conn(0x7f51ac0c4d50 legacy=0x7f51ac0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.218+0000 7f51abe00640 1 -- 172.21.15.123:0/36561402 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac073ce0 con 0x7f51ac074870 2024-04-25T12:34:50.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.219+0000 7f51abe00640 1 --2- 172.21.15.123:0/36561402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac154890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5198009870 tx=0x7f519802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8c137c9a7a6c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:50.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.219+0000 7f51ab400640 1 -- 172.21.15.123:0/36561402 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198037070 con 0x7f51ac074870 2024-04-25T12:34:50.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.219+0000 7f51ab400640 1 -- 172.21.15.123:0/36561402 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519802ae50 con 0x7f51ac074870 2024-04-25T12:34:50.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.219+0000 7f51ab400640 1 -- 172.21.15.123:0/36561402 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198032860 con 0x7f51ac074870 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 -- 172.21.15.123:0/36561402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 msgr2=0x7f51ac154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 --2- 172.21.15.123:0/36561402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac154890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5198009870 tx=0x7f519802a300 comp rx=0 tx=0).stop 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 -- 172.21.15.123:0/36561402 shutdown_connections 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 --2- 172.21.15.123:0/36561402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac15afa0 0x7f51ac0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 --2- 172.21.15.123:0/36561402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac154890 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 -- 172.21.15.123:0/36561402 >> 172.21.15.123:0/36561402 conn(0x7f51ac06c750 msgr2=0x7f51ac06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 -- 172.21.15.123:0/36561402 shutdown_connections 2024-04-25T12:34:50.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.220+0000 7f51b3400640 1 -- 172.21.15.123:0/36561402 wait complete. 2024-04-25T12:34:50.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.224+0000 7f51b3400640 1 Processor -- start 2024-04-25T12:34:50.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.224+0000 7f51b3400640 1 -- start start 2024-04-25T12:34:50.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac0c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b3400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac0c4d50 0x7f51ac1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51ac15afa0 0x7f51ac0c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b3400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac0c8460 con 0x7f51ac074870 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b3400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac0742a0 con 0x7f51ac15afa0 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b3400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f51ac0c9370 con 0x7f51ac0c4d50 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac0c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51abe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac0c4d50 0x7f51ac1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51ac15afa0 0x7f51ac0c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac0c44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35316/0 (socket says 172.21.15.123:35316) 2024-04-25T12:34:50.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.227+0000 7f51b0a00640 1 -- 172.21.15.123:0/2491264688 learned_addr learned my addr 172.21.15.123:0/2491264688 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:50.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51abe00640 1 -- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51ac15afa0 msgr2=0x7f51ac0c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51abe00640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51ac15afa0 0x7f51ac0c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51abe00640 1 -- 172.21.15.123:0/2491264688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 msgr2=0x7f51ac0c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51abe00640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac0c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51abe00640 1 -- 172.21.15.123:0/2491264688 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac0cc710 con 0x7f51ac0c4d50 2024-04-25T12:34:50.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51b0a00640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac0c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51b1400640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51ac15afa0 0x7f51ac0c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51abe00640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac0c4d50 0x7f51ac1590a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0ce700 tx=0x7f51980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51a8c00640 1 -- 172.21.15.123:0/2491264688 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198037040 con 0x7f51ac0c4d50 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51a8c00640 1 -- 172.21.15.123:0/2491264688 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5198004230 con 0x7f51ac0c4d50 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51a8c00640 1 -- 172.21.15.123:0/2491264688 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198003610 con 0x7f51ac0c4d50 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.228+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac1595e0 con 0x7f51ac0c4d50 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.229+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac0ccc60 con 0x7f51ac0c4d50 2024-04-25T12:34:50.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.230+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ac076700 con 0x7f51ac0c4d50 2024-04-25T12:34:50.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.231+0000 7f51a8c00640 1 -- 172.21.15.123:0/2491264688 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f519803a070 con 0x7f51ac0c4d50 2024-04-25T12:34:50.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.232+0000 7f51a8c00640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f518c0744b0 0x7f518c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.232+0000 7f51a8c00640 1 -- 172.21.15.123:0/2491264688 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f51980b5a80 con 0x7f51ac0c4d50 2024-04-25T12:34:50.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.232+0000 7f51b0a00640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f518c0744b0 0x7f518c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.233+0000 7f51b0a00640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f518c0744b0 0x7f518c0769c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f60 tx=0x7f51a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:50.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.237+0000 7f51a8c00640 1 -- 172.21.15.123:0/2491264688 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51980b7e20 con 0x7f51ac0c4d50 2024-04-25T12:34:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.436+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51ac069920 con 0x7f51ac0c4d50 2024-04-25T12:34:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.438+0000 7f51a8c00640 1 -- 172.21.15.123:0/2491264688 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7f5198041020 con 0x7f51ac0c4d50 2024-04-25T12:34:50.439 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:50.439 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f518c0744b0 msgr2=0x7f518c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f518c0744b0 0x7f518c0769c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f60 tx=0x7f51a0009040 comp rx=0 tx=0).stop 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac0c4d50 msgr2=0x7f51ac1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac0c4d50 0x7f51ac1590a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0ce700 tx=0x7f51980046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 shutdown_connections 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f518c0744b0 0x7f518c0769c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51ac15afa0 0x7f51ac0c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51ac0c4d50 0x7f51ac1590a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 --2- 172.21.15.123:0/2491264688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ac074870 0x7f51ac0c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 >> 172.21.15.123:0/2491264688 conn(0x7f51ac06c750 msgr2=0x7f51ac06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 shutdown_connections 2024-04-25T12:34:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.441+0000 7f51b3400640 1 -- 172.21.15.123:0/2491264688 wait complete. 2024-04-25T12:34:50.469 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T12:34:50.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:50.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.731+0000 7fbbba400640 1 Processor -- start 2024-04-25T12:34:50.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.732+0000 7fbbba400640 1 -- start start 2024-04-25T12:34:50.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbba400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbba400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbb4074870 0x7fbbb415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbba400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbbb40c8c60 0x7fbbb40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbba400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb40c61f0 con 0x7fbbb4074870 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbba400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb40c87d0 con 0x7fbbb40c8c60 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbba400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb4073ce0 con 0x7fbbb40c4d50 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbabe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbb4074870 0x7fbbb415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbabe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbb4074870 0x7fbbb415afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35326/0 (socket says 172.21.15.123:35326) 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbabe00640 1 -- 172.21.15.123:0/1866993066 learned_addr learned my addr 172.21.15.123:0/1866993066 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbb3400640 1 --2- 172.21.15.123:0/1866993066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.734+0000 7fbbb3e00640 1 --2- 172.21.15.123:0/1866993066 >> v2:172.21.15.153:3300/0 conn(0x7fbbb40c8c60 0x7fbbb40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.735+0000 7fbbabe00640 1 -- 172.21.15.123:0/1866993066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 msgr2=0x7fbbb4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:50.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.735+0000 7fbbabe00640 1 --2- 172.21.15.123:0/1866993066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.735+0000 7fbbabe00640 1 -- 172.21.15.123:0/1866993066 >> v2:172.21.15.153:3300/0 conn(0x7fbbb40c8c60 msgr2=0x7fbbb40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:50.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.735+0000 7fbbabe00640 1 --2- 172.21.15.123:0/1866993066 >> v2:172.21.15.153:3300/0 conn(0x7fbbb40c8c60 0x7fbbb40c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.735+0000 7fbbabe00640 1 -- 172.21.15.123:0/1866993066 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb4073fc0 con 0x7fbbb4074870 2024-04-25T12:34:50.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.736+0000 7fbbabe00640 1 --2- 172.21.15.123:0/1866993066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb4074870 0x7fbbb415afa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbb406eda0 tx=0x7fbb9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376900043f5d0af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:50.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.736+0000 7fbbb2a00640 1 -- 172.21.15.123:0/1866993066 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c037070 con 0x7fbbb4074870 2024-04-25T12:34:50.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.736+0000 7fbbb2a00640 1 -- 172.21.15.123:0/1866993066 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9c032440 con 0x7fbbb4074870 2024-04-25T12:34:50.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.736+0000 7fbbb2a00640 1 -- 172.21.15.123:0/1866993066 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c032810 con 0x7fbbb4074870 2024-04-25T12:34:50.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 -- 172.21.15.123:0/1866993066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb4074870 msgr2=0x7fbbb415afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 --2- 172.21.15.123:0/1866993066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb4074870 0x7fbbb415afa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbb406eda0 tx=0x7fbb9c02a270 comp rx=0 tx=0).stop 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 -- 172.21.15.123:0/1866993066 shutdown_connections 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 --2- 172.21.15.123:0/1866993066 >> v2:172.21.15.153:3300/0 conn(0x7fbbb40c8c60 0x7fbbb40c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 --2- 172.21.15.123:0/1866993066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb4074870 0x7fbbb415afa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 --2- 172.21.15.123:0/1866993066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 -- 172.21.15.123:0/1866993066 >> 172.21.15.123:0/1866993066 conn(0x7fbbb406c750 msgr2=0x7fbbb406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 -- 172.21.15.123:0/1866993066 shutdown_connections 2024-04-25T12:34:50.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.737+0000 7fbbba400640 1 -- 172.21.15.123:0/1866993066 wait complete. 2024-04-25T12:34:50.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.739+0000 7fbbba400640 1 Processor -- start 2024-04-25T12:34:50.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.739+0000 7fbbba400640 1 -- start start 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbba400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbb4074870 0x7fbbb41f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbba400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb414c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbba400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb40c8c60 0x7fbbb40c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbba400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb40cda90 con 0x7fbbb40c8c60 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbba400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb40c8460 con 0x7fbbb4074870 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbba400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb40cdc10 con 0x7fbbb40c4d50 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb40c8c60 0x7fbbb40c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb40c8c60 0x7fbbb40c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35342/0 (socket says 172.21.15.123:35342) 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 -- 172.21.15.123:0/3944375778 learned_addr learned my addr 172.21.15.123:0/3944375778 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:50.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbb4074870 0x7fbbb41f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbabe00640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb414c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 -- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 msgr2=0x7fbbb414c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb414c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 -- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbb4074870 msgr2=0x7fbbb41f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbb4074870 0x7fbbb41f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.741+0000 7fbbb3e00640 1 -- 172.21.15.123:0/3944375778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb41f7240 con 0x7fbbb40c8c60 2024-04-25T12:34:50.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.742+0000 7fbbb3e00640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb40c8c60 0x7fbbb40c3de0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00efc0 tx=0x7fbbac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:50.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.742+0000 7fbbab400640 1 -- 172.21.15.123:0/3944375778 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac018020 con 0x7fbbb40c8c60 2024-04-25T12:34:50.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.742+0000 7fbbab400640 1 -- 172.21.15.123:0/3944375778 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac014480 con 0x7fbbb40c8c60 2024-04-25T12:34:50.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.742+0000 7fbbab400640 1 -- 172.21.15.123:0/3944375778 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac009e60 con 0x7fbbb40c8c60 2024-04-25T12:34:50.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.742+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb4151dc0 con 0x7fbbb40c8c60 2024-04-25T12:34:50.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.742+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb40cd060 con 0x7fbbb40c8c60 2024-04-25T12:34:50.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.743+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb4076700 con 0x7fbbb40c8c60 2024-04-25T12:34:50.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.746+0000 7fbbab400640 1 -- 172.21.15.123:0/3944375778 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbbac005d10 con 0x7fbbb40c8c60 2024-04-25T12:34:50.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.746+0000 7fbbab400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbb900744e0 0x7fbb900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:50.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.747+0000 7fbbab400640 1 -- 172.21.15.123:0/3944375778 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbbac096190 con 0x7fbbb40c8c60 2024-04-25T12:34:50.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.747+0000 7fbbb3400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbb900744e0 0x7fbb900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:50.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.748+0000 7fbbb3400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbb900744e0 0x7fbb900769f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009aa0 tx=0x7fbba4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:50.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.751+0000 7fbbab400640 1 -- 172.21.15.123:0/3944375778 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbbac022030 con 0x7fbbb40c8c60 2024-04-25T12:34:50.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.953+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbb40b3610 con 0x7fbbb40c8c60 2024-04-25T12:34:50.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.956+0000 7fbbab400640 1 -- 172.21.15.123:0/3944375778 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fbbac010040 con 0x7fbbb40c8c60 2024-04-25T12:34:50.957 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:50.957 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:50.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.958+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbb900744e0 msgr2=0x7fbb900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.958+0000 7fbbba400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbb900744e0 0x7fbb900769f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009aa0 tx=0x7fbba4009190 comp rx=0 tx=0).stop 2024-04-25T12:34:50.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb40c8c60 msgr2=0x7fbbb40c3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:50.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb40c8c60 0x7fbbb40c3de0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00efc0 tx=0x7fbbac00c490 comp rx=0 tx=0).stop 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 shutdown_connections 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbb900744e0 0x7fbb900769f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb40c8c60 0x7fbbb40c3de0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbbb40c4d50 0x7fbbb414c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 --2- 172.21.15.123:0/3944375778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbb4074870 0x7fbbb41f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 >> 172.21.15.123:0/3944375778 conn(0x7fbbb406c750 msgr2=0x7fbbb406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 shutdown_connections 2024-04-25T12:34:50.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:50.959+0000 7fbbba400640 1 -- 172.21.15.123:0/3944375778 wait complete. 2024-04-25T12:34:50.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T12:34:50.985 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T12:34:50.985 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T12:34:50.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T12:34:50.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T12:34:50.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T12:34:50.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T12:34:50.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T12:34:51.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.407+0000 7f0b22c00640 1 Processor -- start 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.407+0000 7f0b22c00640 1 -- start start 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.410+0000 7f0b22c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b140334e0 0x7f0b14006a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.410+0000 7f0b22c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0b14045e70 0x7f0b140d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.411+0000 7f0b1be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b140334e0 0x7f0b14006a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.411+0000 7f0b1be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b140334e0 0x7f0b14006a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35954/0 (socket says 172.21.15.123:35954) 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.411+0000 7f0b1be00640 1 -- 172.21.15.123:0/1602065223 learned_addr learned my addr 172.21.15.123:0/1602065223 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.411+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b14047540 con 0x7f0b14050ed0 2024-04-25T12:34:51.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.411+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0b140d5c30 con 0x7f0b14045e70 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.412+0000 7f0b1b400640 1 --2- 172.21.15.123:0/1602065223 >> v2:172.21.15.153:3300/0 conn(0x7f0b14045e70 0x7f0b140d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.412+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0b14034fe0 con 0x7f0b140334e0 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.412+0000 7f0b1aa00640 1 -- 172.21.15.123:0/1602065223 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081822062 0 0) 0x7f0b14047540 con 0x7f0b14050ed0 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.413+0000 7f0b1aa00640 1 -- 172.21.15.123:0/1602065223 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b04003520 con 0x7f0b14050ed0 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.413+0000 7f0b1aa00640 1 -- 172.21.15.123:0/1602065223 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4135294224 0 0) 0x7f0b04003520 con 0x7f0b14050ed0 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.413+0000 7f0b1be00640 1 -- 172.21.15.123:0/1602065223 >> v2:172.21.15.153:3300/0 conn(0x7f0b14045e70 msgr2=0x7f0b140d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.413+0000 7f0b1be00640 1 --2- 172.21.15.123:0/1602065223 >> v2:172.21.15.153:3300/0 conn(0x7f0b14045e70 0x7f0b140d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.413+0000 7f0b1be00640 1 -- 172.21.15.123:0/1602065223 >> v1:172.21.15.153:6789/0 conn(0x7f0b14050ed0 legacy=0x7f0b14046d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.413+0000 7f0b1be00640 1 -- 172.21.15.123:0/1602065223 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b14035840 con 0x7f0b140334e0 2024-04-25T12:34:51.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.416+0000 7f0b1be00640 1 --2- 172.21.15.123:0/1602065223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b140334e0 0x7f0b14006a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009870 tx=0x7f0b1002a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e91a9f32012c5bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.416+0000 7f4f5f200640 1 Processor -- start 2024-04-25T12:34:51.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.417+0000 7f4f5f200640 1 -- start start 2024-04-25T12:34:51.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.417+0000 7f0b1aa00640 1 -- 172.21.15.123:0/1602065223 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10037070 con 0x7f0b140334e0 2024-04-25T12:34:51.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.417+0000 7f0b1aa00640 1 -- 172.21.15.123:0/1602065223 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1002ac80 con 0x7f0b140334e0 2024-04-25T12:34:51.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.417+0000 7f0b1aa00640 1 -- 172.21.15.123:0/1602065223 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08001670 con 0x7f0b140334e0 2024-04-25T12:34:51.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.419+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b140334e0 msgr2=0x7f0b14006a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.419+0000 7f0b22c00640 1 --2- 172.21.15.123:0/1602065223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b140334e0 0x7f0b14006a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009870 tx=0x7f0b1002a1f0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.419+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 shutdown_connections 2024-04-25T12:34:51.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.419+0000 7fe83a400640 1 Processor -- start 2024-04-25T12:34:51.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.419+0000 7f4f5f200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f580c2720 0x7f4f5815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f0b22c00640 1 --2- 172.21.15.123:0/1602065223 >> v2:172.21.15.153:3300/0 conn(0x7f0b14045e70 0x7f0b140d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f0b22c00640 1 --2- 172.21.15.123:0/1602065223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b140334e0 0x7f0b14006a90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 >> 172.21.15.123:0/1602065223 conn(0x7f0b140503b0 msgr2=0x7f0b140507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f4f5f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f5806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7fe83a400640 1 -- start start 2024-04-25T12:34:51.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f4f5f200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f58161450 con 0x7f4f5806ccf0 2024-04-25T12:34:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f4f5f200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4f581616f0 con 0x7f4f580c2720 2024-04-25T12:34:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f4f5f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4f580c4030 con 0x7f4f580748a0 2024-04-25T12:34:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 shutdown_connections 2024-04-25T12:34:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.420+0000 7f4f5de00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f580c2720 0x7f4f5815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.421+0000 7f4f5de00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f580c2720 0x7f4f5815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55290/0 (socket says 172.21.15.123:55290) 2024-04-25T12:34:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.421+0000 7f4f5de00640 1 -- 172.21.15.123:0/2385092063 learned_addr learned my addr 172.21.15.123:0/2385092063 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.421+0000 7f4f5d400640 1 --2- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f5806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.423+0000 7fe83a400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8340c4960 0x7fe83415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.422+0000 7f4f5de00640 1 -- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 msgr2=0x7f4f5806c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.422+0000 7f4f5de00640 1 --2- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f5806c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.422+0000 7f4f5de00640 1 -- 172.21.15.123:0/2385092063 >> v1:172.21.15.153:6789/0 conn(0x7f4f5806ccf0 legacy=0x7f4f580c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.422+0000 7f4f5de00640 1 -- 172.21.15.123:0/2385092063 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f580c4310 con 0x7f4f580c2720 2024-04-25T12:34:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.423+0000 7fe83a400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe834160b30 0x7fe834160f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.424+0000 7f4f5de00640 1 --2- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f5815de60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009870 tx=0x7f4f4c02a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e76ccfa997b69a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.424+0000 7fe833400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8340c4960 0x7fe83415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.424+0000 7fe83a400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8340c85a0 con 0x7fe8340c7e60 2024-04-25T12:34:51.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.424+0000 7fe83a400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe8340c70a0 con 0x7fe834160b30 2024-04-25T12:34:51.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.424+0000 7fe83a400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe8340c6560 con 0x7fe8340c4960 2024-04-25T12:34:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.425+0000 7fe833400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8340c4960 0x7fe83415a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55298/0 (socket says 172.21.15.123:55298) 2024-04-25T12:34:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.425+0000 7fe833400640 1 -- 172.21.15.123:0/2703121617 learned_addr learned my addr 172.21.15.123:0/2703121617 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.425+0000 7fe833e00640 1 --2- 172.21.15.123:0/2703121617 >> v2:172.21.15.123:3300/0 conn(0x7fe834160b30 0x7fe834160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.425+0000 7f4f5ca00640 1 -- 172.21.15.123:0/2385092063 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c037070 con 0x7f4f580c2720 2024-04-25T12:34:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.425+0000 7f4f5ca00640 1 -- 172.21.15.123:0/2385092063 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c032440 con 0x7f4f580c2720 2024-04-25T12:34:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.425+0000 7f4f5ca00640 1 -- 172.21.15.123:0/2385092063 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c032830 con 0x7f4f580c2720 2024-04-25T12:34:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.425+0000 7f36a5c00640 1 Processor -- start 2024-04-25T12:34:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.426+0000 7f36a5c00640 1 -- start start 2024-04-25T12:34:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.426+0000 7f4f5f200640 1 -- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 msgr2=0x7f4f5815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.426+0000 7f4f5f200640 1 --2- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f5815de60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009870 tx=0x7f4f4c02a290 comp rx=0 tx=0).stop 2024-04-25T12:34:51.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7f4f5f200640 1 -- 172.21.15.123:0/2385092063 shutdown_connections 2024-04-25T12:34:51.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7f4f5f200640 1 --2- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f5806c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7f4f5f200640 1 --2- 172.21.15.123:0/2385092063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f5815de60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.423+0000 7f0b22c00640 1 -- 172.21.15.123:0/1602065223 wait complete. 2024-04-25T12:34:51.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7f4f5f200640 1 -- 172.21.15.123:0/2385092063 >> 172.21.15.123:0/2385092063 conn(0x7f4f58070b90 msgr2=0x7f4f58070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7fe833400640 1 -- 172.21.15.123:0/2703121617 >> v2:172.21.15.123:3300/0 conn(0x7fe834160b30 msgr2=0x7fe834160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7fe833400640 1 --2- 172.21.15.123:0/2703121617 >> v2:172.21.15.123:3300/0 conn(0x7fe834160b30 0x7fe834160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.428+0000 7f4f5f200640 1 -- 172.21.15.123:0/2385092063 shutdown_connections 2024-04-25T12:34:51.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.428+0000 7f4f5f200640 1 -- 172.21.15.123:0/2385092063 wait complete. 2024-04-25T12:34:51.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7fe833400640 1 -- 172.21.15.123:0/2703121617 >> v1:172.21.15.153:6789/0 conn(0x7fe8340c7e60 legacy=0x7fe834073df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:34:51.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.427+0000 7fe833400640 1 -- 172.21.15.123:0/2703121617 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8340c6800 con 0x7fe8340c4960 2024-04-25T12:34:51.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.428+0000 7f0b22c00640 1 Processor -- start 2024-04-25T12:34:51.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.428+0000 7fe833e00640 1 --2- 172.21.15.123:0/2703121617 >> v2:172.21.15.123:3300/0 conn(0x7fe834160b30 0x7fe834160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:51.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f36a006c570 0x7f36a01526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a00b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a00c9600 con 0x7f36a00c4380 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f36a00c9bc0 con 0x7f36a006c570 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f36a00c98e0 con 0x7f36a00cc720 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a00b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a00b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35962/0 (socket says 172.21.15.123:35962) 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.430+0000 7f36a5200640 1 -- 172.21.15.123:0/882565657 learned_addr learned my addr 172.21.15.123:0/882565657 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7f3697400640 1 --2- 172.21.15.123:0/882565657 >> v2:172.21.15.153:3300/0 conn(0x7f36a006c570 0x7f36a01526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7fe833400640 1 --2- 172.21.15.123:0/2703121617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8340c4960 0x7fe83415a8c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8340cdd40 tx=0x7fe81c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6e96abcf150451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7f36a5200640 1 -- 172.21.15.123:0/882565657 >> v2:172.21.15.153:3300/0 conn(0x7f36a006c570 msgr2=0x7f36a01526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7f36a5200640 1 --2- 172.21.15.123:0/882565657 >> v2:172.21.15.153:3300/0 conn(0x7f36a006c570 0x7f36a01526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7f36a5200640 1 -- 172.21.15.123:0/882565657 >> v1:172.21.15.153:6789/0 conn(0x7f36a00c4380 legacy=0x7f36a00c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7fe832000640 1 -- 172.21.15.123:0/2703121617 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c037070 con 0x7fe8340c4960 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7fe832000640 1 -- 172.21.15.123:0/2703121617 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c032440 con 0x7fe8340c4960 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7fe832000640 1 -- 172.21.15.123:0/2703121617 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c032810 con 0x7fe8340c4960 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.433+0000 7fe83a400640 1 -- 172.21.15.123:0/2703121617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8340c4960 msgr2=0x7fe83415a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.433+0000 7fe83a400640 1 --2- 172.21.15.123:0/2703121617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8340c4960 0x7fe83415a8c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8340cdd40 tx=0x7fe81c02a2c0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.433+0000 7fe83a400640 1 -- 172.21.15.123:0/2703121617 shutdown_connections 2024-04-25T12:34:51.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.433+0000 7fe83a400640 1 --2- 172.21.15.123:0/2703121617 >> v2:172.21.15.123:3300/0 conn(0x7fe834160b30 0x7fe834160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.433+0000 7fe83a400640 1 --2- 172.21.15.123:0/2703121617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8340c4960 0x7fe83415a8c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.433+0000 7fe83a400640 1 -- 172.21.15.123:0/2703121617 >> 172.21.15.123:0/2703121617 conn(0x7fe8340710e0 msgr2=0x7fe8340cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f4f5f200640 1 Processor -- start 2024-04-25T12:34:51.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7fe83a400640 1 -- 172.21.15.123:0/2703121617 shutdown_connections 2024-04-25T12:34:51.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.432+0000 7f0b22c00640 1 -- start start 2024-04-25T12:34:51.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f0b22c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b140334e0 0x7f0b140cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f0b22c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b14045e70 0x7f0b140d7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f0b22c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14050ed0 0x7f0b140d8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f0b22c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14035560 con 0x7f0b14050ed0 2024-04-25T12:34:51.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f0b22c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14035280 con 0x7f0b140334e0 2024-04-25T12:34:51.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f0b22c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0b14006570 con 0x7f0b14045e70 2024-04-25T12:34:51.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7f4f5f200640 1 -- start start 2024-04-25T12:34:51.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.435+0000 7f0b20c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14050ed0 0x7f0b140d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.435+0000 7f0b1be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b140334e0 0x7f0b140cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.435+0000 7f0b1be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b140334e0 0x7f0b140cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55314/0 (socket says 172.21.15.123:55314) 2024-04-25T12:34:51.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.435+0000 7f0b1be00640 1 -- 172.21.15.123:0/206510561 learned_addr learned my addr 172.21.15.123:0/206510561 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.435+0000 7f0b1b400640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b14045e70 0x7f0b140d7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.434+0000 7fe83a400640 1 -- 172.21.15.123:0/2703121617 wait complete. 2024-04-25T12:34:51.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.431+0000 7f36a5200640 1 -- 172.21.15.123:0/882565657 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a01584f0 con 0x7f36a00cc720 2024-04-25T12:34:51.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.436+0000 7f5f39800640 1 Processor -- start 2024-04-25T12:34:51.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.436+0000 7f36a5200640 1 --2- 172.21.15.123:0/882565657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a00b2fa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3698009870 tx=0x7f369802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b47be8b193ed476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.436+0000 7f0b1b400640 1 -- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b140334e0 msgr2=0x7f0b140cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.436+0000 7f0b1b400640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b140334e0 0x7f0b140cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.436+0000 7f0b1b400640 1 -- 172.21.15.123:0/206510561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14050ed0 msgr2=0x7f0b140d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.436+0000 7f5f39800640 1 -- start start 2024-04-25T12:34:51.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f369f400640 1 -- 172.21.15.123:0/882565657 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698037070 con 0x7f36a00cc720 2024-04-25T12:34:51.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f369f400640 1 -- 172.21.15.123:0/882565657 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698032440 con 0x7f36a00cc720 2024-04-25T12:34:51.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f369f400640 1 -- 172.21.15.123:0/882565657 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698032810 con 0x7f36a00cc720 2024-04-25T12:34:51.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f0b1b400640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b14050ed0 0x7f0b140d8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f0b1b400640 1 -- 172.21.15.123:0/206510561 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b140df560 con 0x7f0b14045e70 2024-04-25T12:34:51.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f0b1be00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b140334e0 0x7f0b140cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:51.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f4f5f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f5806ccf0 0x7f4f580c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f0b1b400640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b14045e70 0x7f0b140d7dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000b60 tx=0x7f0b1c069db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7fe83a400640 1 Processor -- start 2024-04-25T12:34:51.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7fe83a400640 1 -- start start 2024-04-25T12:34:51.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.437+0000 7f4f5f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f580c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c06dbc0 con 0x7f0b14045e70 2024-04-25T12:34:51.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1c06dd80 con 0x7f0b14045e70 2024-04-25T12:34:51.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c096620 con 0x7f0b14045e70 2024-04-25T12:34:51.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f4f5f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f58157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f36a5c00640 1 -- 172.21.15.123:0/882565657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 msgr2=0x7f36a00b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f36a5c00640 1 --2- 172.21.15.123:0/882565657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a00b2fa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3698009870 tx=0x7f369802a250 comp rx=0 tx=0).stop 2024-04-25T12:34:51.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f36a5c00640 1 -- 172.21.15.123:0/882565657 shutdown_connections 2024-04-25T12:34:51.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f36a5c00640 1 --2- 172.21.15.123:0/882565657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a00b2fa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f36a5c00640 1 --2- 172.21.15.123:0/882565657 >> v2:172.21.15.153:3300/0 conn(0x7f36a006c570 0x7f36a01526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f36a5c00640 1 -- 172.21.15.123:0/882565657 >> 172.21.15.123:0/882565657 conn(0x7f36a0070b90 msgr2=0x7f36a0070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f4f5d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f580c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.438+0000 7f4f5d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f580c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35976/0 (socket says 172.21.15.123:35976) 2024-04-25T12:34:51.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f4f5e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f58157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f4f5e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f58157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55320/0 (socket says 172.21.15.123:55320) 2024-04-25T12:34:51.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f4f5d400640 1 -- 172.21.15.123:0/1674821973 learned_addr learned my addr 172.21.15.123:0/1674821973 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f36a5c00640 1 -- 172.21.15.123:0/882565657 shutdown_connections 2024-04-25T12:34:51.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f4f5f200640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58074470 con 0x7f4f5806ccf0 2024-04-25T12:34:51.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b141712c0 con 0x7f0b14045e70 2024-04-25T12:34:51.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f4f5de00640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f5806ccf0 0x7f4f580c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f4f5f200640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f580742f0 con 0x7f4f580c2720 2024-04-25T12:34:51.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f4f5f200640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4f580c88b0 con 0x7f4f580748a0 2024-04-25T12:34:51.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f4f5e800640 1 -- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 msgr2=0x7f4f580c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f4f5e800640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f580748a0 0x7f4f580c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f4f5e800640 1 -- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f5806ccf0 msgr2=0x7f4f580c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f4f5e800640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f5806ccf0 0x7f4f580c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f4f5e800640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f581e8930 con 0x7f4f580c2720 2024-04-25T12:34:51.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b140da310 con 0x7f0b14045e70 2024-04-25T12:34:51.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7fe83a400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8340c4960 0x7fe834150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7fe83a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8340c7e60 0x7fe83414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7fe83a400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe834160b30 0x7fe834155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7fe83a400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe834161480 con 0x7fe8340c7e60 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7fe83a400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340c6dc0 con 0x7fe834160b30 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.440+0000 7fe83a400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe8340c8270 con 0x7fe8340c4960 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.441+0000 7fe833400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8340c4960 0x7fe834150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.441+0000 7fe833400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8340c4960 0x7fe834150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35988/0 (socket says 172.21.15.123:35988) 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.441+0000 7fe833400640 1 -- 172.21.15.123:0/121133019 learned_addr learned my addr 172.21.15.123:0/121133019 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.441+0000 7fe832a00640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8340c7e60 0x7fe83414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f53b0400640 1 Processor -- start 2024-04-25T12:34:51.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f53b0400640 1 -- start start 2024-04-25T12:34:51.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b141710d0 con 0x7f0b14045e70 2024-04-25T12:34:51.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7fe833400640 1 -- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe834160b30 msgr2=0x7fe834155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7fe833400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe834160b30 0x7fe834155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7fe833400640 1 -- 172.21.15.123:0/121133019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8340c7e60 msgr2=0x7fe83414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7fe833400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8340c7e60 0x7fe83414efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7fe833400640 1 -- 172.21.15.123:0/121133019 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8341508b0 con 0x7fe8340c4960 2024-04-25T12:34:51.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7f4f5e800640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f58157310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c700 tx=0x7f4f5400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7f5f39800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f3415aca0 0x7f5f34161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.439+0000 7f36a5c00640 1 -- 172.21.15.123:0/882565657 wait complete. 2024-04-25T12:34:51.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7fe833400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8340c4960 0x7fe834150370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009960 tx=0x7fe81c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.442+0000 7f5f39800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f3415b0b0 con 0x7f5f340c3b50 2024-04-25T12:34:51.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7f5f39800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f34073480 con 0x7f5f340c3740 2024-04-25T12:34:51.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7f5f39800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5f34074170 con 0x7f5f3415aca0 2024-04-25T12:34:51.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7f5f32000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5f340c3740 0x7f5f340ca060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:41534/0 (socket says 172.21.15.123:41534) 2024-04-25T12:34:51.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7f5f32000640 1 -- 172.21.15.123:0/1292416540 learned_addr learned my addr 172.21.15.123:0/1292416540 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:51.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0b1c0967c0 con 0x7f0b14045e70 2024-04-25T12:34:51.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54016020 con 0x7f4f580c2720 2024-04-25T12:34:51.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3875844436 0 0) 0x7f5f34073480 con 0x7f5f340c3740 2024-04-25T12:34:51.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.443+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f18003520 con 0x7f5f340c3740 2024-04-25T12:34:51.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f5f32a00640 1 --2- 172.21.15.123:0/1292416540 >> v2:172.21.15.123:3300/0 conn(0x7f5f3415aca0 0x7f5f34161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f36a5c00640 1 Processor -- start 2024-04-25T12:34:51.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f54006d50 con 0x7f4f580c2720 2024-04-25T12:34:51.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f4f5f200640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f580c7160 con 0x7f4f580c2720 2024-04-25T12:34:51.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f4f5f200640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f58158270 con 0x7f4f580c2720 2024-04-25T12:34:51.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54003bb0 con 0x7f4f580c2720 2024-04-25T12:34:51.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f36a5c00640 1 -- start start 2024-04-25T12:34:51.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4067069351 0 0) 0x7f5f18003520 con 0x7f5f340c3740 2024-04-25T12:34:51.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 >> v2:172.21.15.123:3300/0 conn(0x7f5f3415aca0 msgr2=0x7f5f34161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f53b0400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53a80cd060 0x7f53a80b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f53b0400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a80cd470 con 0x7f53a815aec0 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f53b0400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a80cd6f0 con 0x7f53a80d1be0 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f53b0400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f53a80cd990 con 0x7f53a80cd060 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f53ad000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53a80cd060 0x7f53a80b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f53ad000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53a80cd060 0x7f53a80b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35398/0 (socket says 172.21.15.123:35398) 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7f53ad000640 1 -- 172.21.15.123:0/147178058 learned_addr learned my addr 172.21.15.123:0/147178058 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.445+0000 7f4f5f200640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f34001030 con 0x7f4f580c2720 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.445+0000 7f5f31600640 1 --2- 172.21.15.123:0/1292416540 >> v2:172.21.15.123:3300/0 conn(0x7f5f3415aca0 0x7f5f34161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.445+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 >> v1:172.21.15.123:6789/0 conn(0x7f5f340c3b50 legacy=0x7f5f3416b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.445+0000 7f53ad000640 1 -- 172.21.15.123:0/147178058 >> v1:172.21.15.153:6789/0 conn(0x7f53a80d1be0 legacy=0x7f53a816b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.444+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c037040 con 0x7fe8340c4960 2024-04-25T12:34:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.445+0000 7f53a7e00640 1 -- 172.21.15.123:0/147178058 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314780224 0 0) 0x7f53a80cd6f0 con 0x7f53a80d1be0 2024-04-25T12:34:51.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.445+0000 7f53ad000640 1 -- 172.21.15.123:0/147178058 >> v1:172.21.15.123:6789/0 conn(0x7f53a815aec0 legacy=0x7f53a80cedc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:34:51.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.445+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c042070 con 0x7fe8340c4960 2024-04-25T12:34:51.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.446+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c030960 con 0x7fe8340c4960 2024-04-25T12:34:51.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.446+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83414df60 con 0x7fe8340c4960 2024-04-25T12:34:51.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.446+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83414fb70 con 0x7fe8340c4960 2024-04-25T12:34:51.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe83414ff50 con 0x7fe8340c4960 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f34073760 con 0x7f5f340c3740 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f5f20001670 con 0x7f5f340c3740 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 >> v1:172.21.15.153:6789/0 conn(0x7f5f340c3740 legacy=0x7f5f340ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f180039b0 0x7f5f18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f18004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f18005270 0x7f5f1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1800c1c0 con 0x7f5f18005270 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1800c370 con 0x7f5f180039b0 2024-04-25T12:34:51.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1800c570 con 0x7f5f18004800 2024-04-25T12:34:51.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.446+0000 7f53ad000640 1 -- 172.21.15.123:0/147178058 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a8160b80 con 0x7f53a80cd060 2024-04-25T12:34:51.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f53ad000640 1 --2- 172.21.15.123:0/147178058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80b6010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f539c009870 tx=0x7f539c02a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd77de8b1b8b8cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.447+0000 7f5f32a00640 1 --2- 172.21.15.123:0/1292416540 >> v2:172.21.15.123:3300/0 conn(0x7f5f3415aca0 0x7f5f34161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:51.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f5f32000640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f18004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.449+0000 7f5f32a00640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f180039b0 0x7f5f18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.449+0000 7f53a7e00640 1 -- 172.21.15.123:0/147178058 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c037070 con 0x7f53a80cd060 2024-04-25T12:34:51.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.449+0000 7f53a7e00640 1 -- 172.21.15.123:0/147178058 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539c032640 con 0x7f53a80cd060 2024-04-25T12:34:51.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.449+0000 7f53a7e00640 1 -- 172.21.15.123:0/147178058 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c037070 con 0x7f53a80cd060 2024-04-25T12:34:51.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.449+0000 7f5f33400640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f18005270 0x7f5f1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f36a5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a006c570 0x7f36a013fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f36a5c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a00c4380 0x7f36a0151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f36a5c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a0154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f36a5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00b3870 con 0x7f36a006c570 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f36a5c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00b36f0 con 0x7f36a00c4380 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.448+0000 7f36a5c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f36a0158a10 con 0x7f36a00cc720 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f5f32000640 1 -- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f180039b0 msgr2=0x7f5f18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f5f32000640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f180039b0 0x7f5f18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f5f32000640 1 -- 172.21.15.123:0/1292416540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f18005270 msgr2=0x7f5f1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f36a5200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a0154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f3697400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a00c4380 0x7f36a0151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.449+0000 7f0b0fe00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0afc074530 0x7f0afc076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0b1c111bf0 con 0x7f0b14045e70 2024-04-25T12:34:51.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f0b0fe00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0afc07dcf0 0x7f0afc080180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0afc080850 con 0x7f0afc07dcf0 2024-04-25T12:34:51.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c114340 con 0x7f0b14045e70 2024-04-25T12:34:51.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f0b1be00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0afc074530 0x7f0afc076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f36a5200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a0154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36018/0 (socket says 172.21.15.123:36018) 2024-04-25T12:34:51.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe81c030b00 con 0x7fe8340c4960 2024-04-25T12:34:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f36a5200640 1 -- 172.21.15.123:0/128211137 learned_addr learned my addr 172.21.15.123:0/128211137 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f3697400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a00c4380 0x7f36a0151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55348/0 (socket says 172.21.15.123:55348) 2024-04-25T12:34:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.451+0000 7fe82b400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8100744b0 0x7fe8100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe81c0b63d0 con 0x7fe8340c4960 2024-04-25T12:34:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7f53b0400640 1 -- 172.21.15.123:0/147178058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 msgr2=0x7f53a80b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7f53b0400640 1 --2- 172.21.15.123:0/147178058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80b6010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f539c009870 tx=0x7f539c02a3c0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.451+0000 7f36a5200640 1 -- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a00c4380 msgr2=0x7f36a0151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.451+0000 7f36a5200640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a00c4380 0x7f36a0151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.451+0000 7f36a5200640 1 -- 172.21.15.123:0/128211137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a006c570 msgr2=0x7f36a013fe60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:34:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7f369fe00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a006c570 0x7f36a013fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f5f32000640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f18005270 0x7f5f1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f5f32000640 1 -- 172.21.15.123:0/1292416540 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f34073760 con 0x7f5f18004800 2024-04-25T12:34:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f5f33400640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f18005270 0x7f5f1800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.450+0000 7f5f33400640 1 -- 172.21.15.123:0/1292416540 reap_dead start 2024-04-25T12:34:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7fe832a00640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8100744b0 0x7fe8100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7fe82b400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe81007dc70 0x7fe810080100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8100807d0 con 0x7fe81007dc70 2024-04-25T12:34:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe81c0b67c0 con 0x7fe8340c4960 2024-04-25T12:34:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.451+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f540050d0 con 0x7f4f580c2720 2024-04-25T12:34:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7f4f52000640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4f3c0745b0 0x7f4f3c076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.452+0000 7f5f32000640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f18004c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f34072f70 tx=0x7f5f18005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7915e37db3bf00df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f0b1be00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0afc074530 0x7f0afc076a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0b1002a230 tx=0x7f0b1002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4f54095c70 con 0x7f4f580c2720 2024-04-25T12:34:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7fe833e00640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe81007dc70 0x7fe810080100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f53b0400640 1 -- 172.21.15.123:0/147178058 shutdown_connections 2024-04-25T12:34:51.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f53b0400640 1 --2- 172.21.15.123:0/147178058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80b6010 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f53b0400640 1 -- 172.21.15.123:0/147178058 >> 172.21.15.123:0/147178058 conn(0x7f53a808d940 msgr2=0x7f53a808dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f4f52000640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4f3c07dd70 0x7f4f3c080200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f4f5de00640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4f3c0745b0 0x7f4f3c076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f53b0400640 1 -- 172.21.15.123:0/147178058 shutdown_connections 2024-04-25T12:34:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f53b0400640 1 -- 172.21.15.123:0/147178058 wait complete. 2024-04-25T12:34:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f36a5200640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a006c570 0x7f36a013fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f36a5200640 1 -- 172.21.15.123:0/128211137 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a01403a0 con 0x7f36a00cc720 2024-04-25T12:34:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f369fe00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36a006c570 0x7f36a013fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f4f5d400640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4f3c07dd70 0x7f4f3c080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f54024e50 con 0x7f4f3c07dd70 2024-04-25T12:34:51.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f54096060 con 0x7f4f580c2720 2024-04-25T12:34:51.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.454+0000 7f4f5de00640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4f3c0745b0 0x7f4f3c076ac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c034000 tx=0x7f4f4c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f0b20c00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0afc07dcf0 0x7f0afc080180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.453+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2002e070 con 0x7f5f18004800 2024-04-25T12:34:51.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.454+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f20005ad0 con 0x7f5f18004800 2024-04-25T12:34:51.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.454+0000 7f5f31600640 1 -- 172.21.15.123:0/1292416540 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20007d30 con 0x7f5f18004800 2024-04-25T12:34:51.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.454+0000 7fe832a00640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8100744b0 0x7fe8100769c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8340cb780 tx=0x7fe82c0094f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.454+0000 7fe833e00640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe81007dc70 0x7fe810080100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.454+0000 7f4f5d400640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4f3c07dd70 0x7f4f3c080200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.455+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8100807d0 con 0x7fe81007dc70 2024-04-25T12:34:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.455+0000 7f5f39800640 1 -- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 msgr2=0x7f5f18004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.455+0000 7f5f39800640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f18004c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f34072f70 tx=0x7f5f18005830 comp rx=0 tx=0).stop 2024-04-25T12:34:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.454+0000 7f36a5200640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a0154d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3698009990 tx=0x7f36980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f5f39800640 1 -- 172.21.15.123:0/1292416540 shutdown_connections 2024-04-25T12:34:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f5f39800640 1 --2- 172.21.15.123:0/1292416540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f18004c10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f5f39800640 1 -- 172.21.15.123:0/1292416540 >> 172.21.15.123:0/1292416540 conn(0x7f5f3406f440 msgr2=0x7f5f3406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698041020 con 0x7f36a00cc720 2024-04-25T12:34:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698030700 con 0x7f36a00cc720 2024-04-25T12:34:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698037040 con 0x7f36a00cc720 2024-04-25T12:34:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a013ede0 con 0x7f36a00cc720 2024-04-25T12:34:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f5f39800640 1 -- 172.21.15.123:0/1292416540 shutdown_connections 2024-04-25T12:34:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a013f380 con 0x7f36a00cc720 2024-04-25T12:34:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.456+0000 7f5f39800640 1 -- 172.21.15.123:0/1292416540 wait complete. 2024-04-25T12:34:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.458+0000 7f53b0400640 1 Processor -- start 2024-04-25T12:34:51.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.459+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3678001030 con 0x7f36a00cc720 2024-04-25T12:34:51.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.458+0000 7f53b0400640 1 -- start start 2024-04-25T12:34:51.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.459+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f54024e50 con 0x7f4f3c07dd70 2024-04-25T12:34:51.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.458+0000 7f0b20c00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0afc07dcf0 0x7f0afc080180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.459+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0afc080850 con 0x7f0afc07dcf0 2024-04-25T12:34:51.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.460+0000 7f5f39800640 1 Processor -- start 2024-04-25T12:34:51.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.461+0000 7f5f39800640 1 -- start start 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.461+0000 7f53b0400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.461+0000 7f53b0400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a80d1be0 0x7f53a8164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53b0400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a815aec0 0x7f53a8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53b0400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8161420 con 0x7f53a80cd060 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53ada00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53ada00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35422/0 (socket says 172.21.15.123:35422) 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53ada00640 1 -- 172.21.15.123:0/2533008437 learned_addr learned my addr 172.21.15.123:0/2533008437 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8161140 con 0x7f53a815aec0 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f53a8161940 con 0x7f53a80d1be0 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.462+0000 7f53ae400640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a815aec0 0x7f53a8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.463+0000 7f53ad000640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a80d1be0 0x7f53a8164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.463+0000 7f53ada00640 1 -- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a80d1be0 msgr2=0x7f53a8164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.463+0000 7f53ada00640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53a80d1be0 0x7f53a8164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.463+0000 7f53ada00640 1 -- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a815aec0 msgr2=0x7f53a8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.463+0000 7f53ada00640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a815aec0 0x7f53a8155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.463+0000 7f53ada00640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a80c9f40 con 0x7f53a80cd060 2024-04-25T12:34:51.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.464+0000 7f5f39800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f3414ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.464+0000 7f53ada00640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80ca720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f539400c3c0 tx=0x7f539400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.464+0000 7f5f39800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f341614c0 0x7f5f341fa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.464+0000 7f5f39800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f34154db0 0x7f5f341551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.464+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940052d0 con 0x7f53a80cd060 2024-04-25T12:34:51.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394005470 con 0x7f53a80cd060 2024-04-25T12:34:51.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940052d0 con 0x7f53a80cd060 2024-04-25T12:34:51.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f5f39800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34071220 con 0x7f5f34154db0 2024-04-25T12:34:51.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f5f39800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34073230 con 0x7f5f341614c0 2024-04-25T12:34:51.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.466+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a8156dc0 con 0x7f53a80cd060 2024-04-25T12:34:51.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f5f39800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5f34070bd0 con 0x7f5f18004800 2024-04-25T12:34:51.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f3414ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.466+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a80ca200 con 0x7f53a80cd060 2024-04-25T12:34:51.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f3414ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36032/0 (socket says 172.21.15.123:36032) 2024-04-25T12:34:51.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f5f32a00640 1 -- 172.21.15.123:0/1681072139 learned_addr learned my addr 172.21.15.123:0/1681072139 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.466+0000 7f5f33400640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f34154db0 0x7f5f341551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.466+0000 7f5f32000640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f341614c0 0x7f5f341fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.467+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53a81572e0 con 0x7f53a80cd060 2024-04-25T12:34:51.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.468+0000 7f5f32a00640 1 -- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f341614c0 msgr2=0x7f5f341fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.468+0000 7f5f32a00640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f341614c0 0x7f5f341fa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.468+0000 7f5f32a00640 1 -- 172.21.15.123:0/1681072139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f34154db0 msgr2=0x7f5f341551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.468+0000 7f5f32a00640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f34154db0 0x7f5f341551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.468+0000 7f5f32a00640 1 -- 172.21.15.123:0/1681072139 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341fae80 con 0x7f5f18004800 2024-04-25T12:34:51.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.468+0000 7f5f32000640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f341614c0 0x7f5f341fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:51.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.465+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3698044020 con 0x7f36a00cc720 2024-04-25T12:34:51.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.468+0000 7f5f33400640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f34154db0 0x7f5f341551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:51.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.469+0000 7f5f32a00640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f3414ffc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f240116f0 tx=0x7f5f24011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.469+0000 7f369cc00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f367c0744b0 0x7f367c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.469+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24019070 con 0x7f5f18004800 2024-04-25T12:34:51.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.469+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3415e3a0 con 0x7f5f18004800 2024-04-25T12:34:51.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.470+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3415ec00 con 0x7f5f18004800 2024-04-25T12:34:51.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.470+0000 7f369fe00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f367c0744b0 0x7f367c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.470+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f36980b5c60 con 0x7f36a00cc720 2024-04-25T12:34:51.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.471+0000 7f369cc00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f367c07dc70 0x7f367c080100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.471+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3698007e80 con 0x7f367c07dc70 2024-04-25T12:34:51.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.471+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36980b82c0 con 0x7f36a00cc720 2024-04-25T12:34:51.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.470+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f24004220 con 0x7f5f18004800 2024-04-25T12:34:51.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.470+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400f040 con 0x7f5f18004800 2024-04-25T12:34:51.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.471+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f0c001030 con 0x7f5f18004800 2024-04-25T12:34:51.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.472+0000 7f3697400640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f367c07dc70 0x7f367c080100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.473+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5394007020 con 0x7f53a80cd060 2024-04-25T12:34:51.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.474+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f2400f7c0 con 0x7f5f18004800 2024-04-25T12:34:51.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.475+0000 7f53a5600640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53880746b0 0x7f5388076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f5f2aa00640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f14074580 0x7f5f14076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5f240958d0 con 0x7f5f18004800 2024-04-25T12:34:51.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f369fe00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f367c0744b0 0x7f367c0769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3690000f60 tx=0x7f3690009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f5f32000640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f14074580 0x7f5f14076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f5f2aa00640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f5f1407dd40 0x7f5f140801d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f2400ee90 con 0x7f5f1407dd40 2024-04-25T12:34:51.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.477+0000 7f5f33400640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f5f1407dd40 0x7f5f140801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.477+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f2409a2a0 con 0x7f5f18004800 2024-04-25T12:34:51.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5394092660 con 0x7f53a80cd060 2024-04-25T12:34:51.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f53a5600640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f538807ddf0 0x7f5388080280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.477+0000 7f5f33400640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f5f1407dd40 0x7f5f140801d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5388080950 con 0x7f538807ddf0 2024-04-25T12:34:51.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5394099060 con 0x7f53a80cd060 2024-04-25T12:34:51.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f53ad000640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53880746b0 0x7f5388076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.476+0000 7f53ae400640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f538807ddf0 0x7f5388080280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.478+0000 7f5f32000640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f14074580 0x7f5f14076a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f20033000 tx=0x7f5f2002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.478+0000 7f53ae400640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f538807ddf0 0x7f5388080280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.478+0000 7f53ad000640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53880746b0 0x7f5388076bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f539c034000 tx=0x7f539c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.478+0000 7f3697400640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f367c07dc70 0x7f367c080100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.478+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f2400ee90 con 0x7f5f1407dd40 2024-04-25T12:34:51.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.479+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5388080950 con 0x7f538807ddf0 2024-04-25T12:34:51.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.479+0000 7f2374c00640 1 Processor -- start 2024-04-25T12:34:51.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.480+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3698007e80 con 0x7f367c07dc70 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.480+0000 7f2374c00640 1 -- start start 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f2374c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f237006d440 0x7f2370156670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f2374c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23700c72c0 0x7f23700c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f2374c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f237006d850 con 0x7f23701606e0 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f2374c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2370160af0 con 0x7f23700c72c0 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f2374c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f237015fc40 con 0x7f237006d440 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f236d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f237006d440 0x7f2370156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f236d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f237006d440 0x7f2370156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36046/0 (socket says 172.21.15.123:36046) 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.483+0000 7f236d400640 1 -- 172.21.15.123:0/3935018690 learned_addr learned my addr 172.21.15.123:0/3935018690 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.484+0000 7f236d400640 1 -- 172.21.15.123:0/3935018690 >> v2:172.21.15.123:3300/0 conn(0x7f23700c72c0 msgr2=0x7f23700c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.484+0000 7f236d400640 1 --2- 172.21.15.123:0/3935018690 >> v2:172.21.15.123:3300/0 conn(0x7f23700c72c0 0x7f23700c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.484+0000 7f236d400640 1 -- 172.21.15.123:0/3935018690 >> v1:172.21.15.123:6789/0 conn(0x7f23701606e0 legacy=0x7f237015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.484+0000 7f236d400640 1 -- 172.21.15.123:0/3935018690 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23700cc920 con 0x7f237006d440 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.486+0000 7f236d400640 1 --2- 172.21.15.123:0/3935018690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f237006d440 0x7f2370156670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23700cc5b0 tx=0x7f235c02a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd781d3f5bb3561b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.486+0000 7f236ca00640 1 -- 172.21.15.123:0/3935018690 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c037070 con 0x7f237006d440 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.486+0000 7f236ca00640 1 -- 172.21.15.123:0/3935018690 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c032440 con 0x7f237006d440 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.486+0000 7f236ca00640 1 -- 172.21.15.123:0/3935018690 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c032810 con 0x7f237006d440 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.487+0000 7f2374c00640 1 -- 172.21.15.123:0/3935018690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f237006d440 msgr2=0x7f2370156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.487+0000 7f2374c00640 1 --2- 172.21.15.123:0/3935018690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f237006d440 0x7f2370156670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23700cc5b0 tx=0x7f235c02a2a0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.488+0000 7f2374c00640 1 -- 172.21.15.123:0/3935018690 shutdown_connections 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.488+0000 7f2374c00640 1 --2- 172.21.15.123:0/3935018690 >> v2:172.21.15.123:3300/0 conn(0x7f23700c72c0 0x7f23700c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.488+0000 7f2374c00640 1 --2- 172.21.15.123:0/3935018690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f237006d440 0x7f2370156670 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.488+0000 7f2374c00640 1 -- 172.21.15.123:0/3935018690 >> 172.21.15.123:0/3935018690 conn(0x7f2370068a50 msgr2=0x7f23700d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.488+0000 7f2374c00640 1 -- 172.21.15.123:0/3935018690 shutdown_connections 2024-04-25T12:34:51.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.488+0000 7f2374c00640 1 -- 172.21.15.123:0/3935018690 wait complete. 2024-04-25T12:34:51.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.489+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b14005570 con 0x7f0afc07dcf0 2024-04-25T12:34:51.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.490+0000 7f0b0fe00640 1 -- 172.21.15.123:0/206510561 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b14005570 con 0x7f0afc07dcf0 2024-04-25T12:34:51.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.492+0000 7f2374c00640 1 Processor -- start 2024-04-25T12:34:51.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.492+0000 7f2374c00640 1 -- start start 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0afc07dcf0 msgr2=0x7f0afc080180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0afc07dcf0 0x7f0afc080180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0afc074530 msgr2=0x7f0afc076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0afc074530 0x7f0afc076a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0b1002a230 tx=0x7f0b1002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b14045e70 msgr2=0x7f0b140d7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 --2- 172.21.15.123:0/206510561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b14045e70 0x7f0b140d7dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000b60 tx=0x7f0b1c069db0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b20c00640 1 -- 172.21.15.123:0/206510561 reap_dead start 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 shutdown_connections 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.494+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 >> 172.21.15.123:0/206510561 conn(0x7f0b140503b0 msgr2=0x7f0b140507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.495+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 shutdown_connections 2024-04-25T12:34:51.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.495+0000 7f0b22c00640 1 -- 172.21.15.123:0/206510561 wait complete. 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.495+0000 7f2374c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237006d440 0x7f23701f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.495+0000 7f2374c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23700c72c0 0x7f23701e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.496+0000 7f236de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237006d440 0x7f23701f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.496+0000 7f236de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237006d440 0x7f23701f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55380/0 (socket says 172.21.15.123:55380) 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.496+0000 7f236de00640 1 -- 172.21.15.123:0/1666691075 learned_addr learned my addr 172.21.15.123:0/1666691075 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.497+0000 7f2374c00640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23701606e0 0x7f23700cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.497+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23701601c0 con 0x7f23700c72c0 2024-04-25T12:34:51.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.497+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f237015fee0 con 0x7f237006d440 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.497+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f23700cc620 con 0x7f23701606e0 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.497+0000 7f236e800640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23701606e0 0x7f23700cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.498+0000 7f236d400640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23700c72c0 0x7f23701e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.499+0000 7f236e800640 1 -- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237006d440 msgr2=0x7f23701f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.499+0000 7f236e800640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237006d440 0x7f23701f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.499+0000 7f236e800640 1 -- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23700c72c0 msgr2=0x7f23701e6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.499+0000 7f236e800640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23700c72c0 0x7f23701e6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.499+0000 7f236e800640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23701f6fe0 con 0x7f23701606e0 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.499+0000 7f236d400640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23700c72c0 0x7f23701e6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:51.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.499+0000 7f236e800640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23701606e0 0x7f23700cde90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f236800bdf0 tx=0x7f236800bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.498+0000 7f4f5f200640 1 -- 172.21.15.123:0/1674821973 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f34002dd0 con 0x7f4f3c07dd70 2024-04-25T12:34:51.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.500+0000 7f4f52000640 1 -- 172.21.15.123:0/1674821973 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f34002dd0 con 0x7f4f3c07dd70 2024-04-25T12:34:51.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.500+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368019070 con 0x7f23701606e0 2024-04-25T12:34:51.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.500+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2368014440 con 0x7f23701606e0 2024-04-25T12:34:51.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.500+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368014830 con 0x7f23701606e0 2024-04-25T12:34:51.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.500+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23701e7300 con 0x7f23701606e0 2024-04-25T12:34:51.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f47400640 1 -- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4f3c07dd70 msgr2=0x7f4f3c080200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f47400640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4f3c07dd70 0x7f4f3c080200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f47400640 1 -- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4f3c0745b0 msgr2=0x7f4f3c076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f47400640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4f3c0745b0 0x7f4f3c076ac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c034000 tx=0x7f4f4c02e000 comp rx=0 tx=0).stop 2024-04-25T12:34:51.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f47400640 1 -- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 msgr2=0x7f4f58157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f47400640 1 --2- 172.21.15.123:0/1674821973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f580c2720 0x7f4f58157310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c700 tx=0x7f4f5400cbc0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.500+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23700c95c0 con 0x7f23701606e0 2024-04-25T12:34:51.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f5e800640 1 -- 172.21.15.123:0/1674821973 reap_dead start 2024-04-25T12:34:51.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f4f47400640 1 -- 172.21.15.123:0/1674821973 shutdown_connections 2024-04-25T12:34:51.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.501+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2348001030 con 0x7f23701606e0 2024-04-25T12:34:51.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.502+0000 7f4f47400640 1 -- 172.21.15.123:0/1674821973 >> 172.21.15.123:0/1674821973 conn(0x7f4f58070b90 msgr2=0x7f4f581606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.503+0000 7f4f47400640 1 -- 172.21.15.123:0/1674821973 shutdown_connections 2024-04-25T12:34:51.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.503+0000 7f4f47400640 1 -- 172.21.15.123:0/1674821973 wait complete. 2024-04-25T12:34:51.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.504+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe834076da0 con 0x7fe81007dc70 2024-04-25T12:34:51.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.504+0000 7fe82b400640 1 -- 172.21.15.123:0/121133019 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe834076da0 con 0x7fe81007dc70 2024-04-25T12:34:51.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.507+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f236801c070 con 0x7f23701606e0 2024-04-25T12:34:51.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.507+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe81007dc70 msgr2=0x7fe810080100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.507+0000 7fe83a400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe81007dc70 0x7fe810080100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.508+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8100744b0 msgr2=0x7fe8100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.508+0000 7fe83a400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8100744b0 0x7fe8100769c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8340cb780 tx=0x7fe82c0094f0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.508+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8340c4960 msgr2=0x7fe834150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.508+0000 7fe83a400640 1 --2- 172.21.15.123:0/121133019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8340c4960 0x7fe834150370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009960 tx=0x7fe81c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.508+0000 7fe833e00640 1 -- 172.21.15.123:0/121133019 reap_dead start 2024-04-25T12:34:51.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.508+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 shutdown_connections 2024-04-25T12:34:51.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.508+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 >> 172.21.15.123:0/121133019 conn(0x7fe8340710e0 msgr2=0x7fe83414d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.509+0000 7f2366000640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2350074580 0x7f2350076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.509+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2368095b50 con 0x7f23701606e0 2024-04-25T12:34:51.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.510+0000 7f2366000640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f235007dd40 0x7f23500801d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.510+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23500808a0 con 0x7f235007dd40 2024-04-25T12:34:51.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.510+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2368095f40 con 0x7f23701606e0 2024-04-25T12:34:51.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.511+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 shutdown_connections 2024-04-25T12:34:51.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.511+0000 7fe83a400640 1 -- 172.21.15.123:0/121133019 wait complete. 2024-04-25T12:34:51.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.511+0000 7f236de00640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2350074580 0x7f2350076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.511+0000 7f236d400640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f235007dd40 0x7f23500801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.513+0000 7f236d400640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f235007dd40 0x7f23500801d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.515+0000 7f236de00640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2350074580 0x7f2350076a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23600099a0 tx=0x7f23600090e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.515+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23500808a0 con 0x7f235007dd40 2024-04-25T12:34:51.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.515+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53a8076700 con 0x7f538807ddf0 2024-04-25T12:34:51.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.517+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5f0c002dd0 con 0x7f5f1407dd40 2024-04-25T12:34:51.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.517+0000 7f53a5600640 1 -- 172.21.15.123:0/2533008437 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53a8076700 con 0x7f538807ddf0 2024-04-25T12:34:51.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.517+0000 7f5f2aa00640 1 -- 172.21.15.123:0/1681072139 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5f0c002dd0 con 0x7f5f1407dd40 2024-04-25T12:34:51.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.518+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f538807ddf0 msgr2=0x7f5388080280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.518+0000 7f53b0400640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f538807ddf0 0x7f5388080280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.518+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53880746b0 msgr2=0x7f5388076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.518+0000 7f53b0400640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53880746b0 0x7f5388076bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f539c034000 tx=0x7f539c02e000 comp rx=0 tx=0).stop 2024-04-25T12:34:51.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.518+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 msgr2=0x7f53a80ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.518+0000 7f53b0400640 1 --2- 172.21.15.123:0/2533008437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53a80cd060 0x7f53a80ca720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f539400c3c0 tx=0x7f539400c880 comp rx=0 tx=0).stop 2024-04-25T12:34:51.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.519+0000 7f53ae400640 1 -- 172.21.15.123:0/2533008437 reap_dead start 2024-04-25T12:34:51.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.519+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 shutdown_connections 2024-04-25T12:34:51.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.519+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 >> 172.21.15.123:0/2533008437 conn(0x7f53a808d940 msgr2=0x7f53a808dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f5f1407dd40 msgr2=0x7f5f140801d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f5f1407dd40 0x7f5f140801d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f14074580 msgr2=0x7f5f14076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f14074580 0x7f5f14076a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f20033000 tx=0x7f5f2002f000 comp rx=0 tx=0).stop 2024-04-25T12:34:51.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 msgr2=0x7f5f3414ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 --2- 172.21.15.123:0/1681072139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f18004800 0x7f5f3414ffc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f240116f0 tx=0x7f5f24011bb0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f33400640 1 -- 172.21.15.123:0/1681072139 reap_dead start 2024-04-25T12:34:51.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 shutdown_connections 2024-04-25T12:34:51.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.520+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 >> 172.21.15.123:0/1681072139 conn(0x7f5f3406f440 msgr2=0x7f5f340c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.521+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3678002dd0 con 0x7f367c07dc70 2024-04-25T12:34:51.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.521+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 shutdown_connections 2024-04-25T12:34:51.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.521+0000 7f5f39800640 1 -- 172.21.15.123:0/1681072139 wait complete. 2024-04-25T12:34:51.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.521+0000 7f369cc00640 1 -- 172.21.15.123:0/128211137 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3678002dd0 con 0x7f367c07dc70 2024-04-25T12:34:51.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.519+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 shutdown_connections 2024-04-25T12:34:51.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.522+0000 7f53b0400640 1 -- 172.21.15.123:0/2533008437 wait complete. 2024-04-25T12:34:51.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f367c07dc70 msgr2=0x7f367c080100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f367c07dc70 0x7f367c080100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f367c0744b0 msgr2=0x7f367c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f367c0744b0 0x7f367c0769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3690000f60 tx=0x7f3690009040 comp rx=0 tx=0).stop 2024-04-25T12:34:51.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 msgr2=0x7f36a0154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 --2- 172.21.15.123:0/128211137 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36a00cc720 0x7f36a0154d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3698009990 tx=0x7f36980046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5200640 1 -- 172.21.15.123:0/128211137 reap_dead start 2024-04-25T12:34:51.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 shutdown_connections 2024-04-25T12:34:51.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.523+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 >> 172.21.15.123:0/128211137 conn(0x7f36a0070b90 msgr2=0x7f36a0072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.527+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 shutdown_connections 2024-04-25T12:34:51.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.527+0000 7f36a5c00640 1 -- 172.21.15.123:0/128211137 wait complete. 2024-04-25T12:34:51.534 INFO:teuthology.orchestra.run.smithi123.stdout:60129542154 2024-04-25T12:34:51.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T12:34:51.549 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T12:34:51.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T12:34:51.558 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T12:34:51.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.558+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2348002dd0 con 0x7f235007dd40 2024-04-25T12:34:51.559 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T12:34:51.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.558+0000 7f2366000640 1 -- 172.21.15.123:0/1666691075 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2348002dd0 con 0x7f235007dd40 2024-04-25T12:34:51.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.559+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f235007dd40 msgr2=0x7f23500801d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.559+0000 7f2374c00640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f235007dd40 0x7f23500801d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.559+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2350074580 msgr2=0x7f2350076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.559+0000 7f2374c00640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2350074580 0x7f2350076a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23600099a0 tx=0x7f23600090e0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.559+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23701606e0 msgr2=0x7f23700cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.559+0000 7f2374c00640 1 --2- 172.21.15.123:0/1666691075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23701606e0 0x7f23700cde90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f236800bdf0 tx=0x7f236800bee0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.560+0000 7f236e800640 1 -- 172.21.15.123:0/1666691075 reap_dead start 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.560+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 shutdown_connections 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.560+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 >> 172.21.15.123:0/1666691075 conn(0x7f2370068a50 msgr2=0x7f23700b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.560+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 shutdown_connections 2024-04-25T12:34:51.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.560+0000 7f2374c00640 1 -- 172.21.15.123:0/1666691075 wait complete. 2024-04-25T12:34:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T12:34:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T12:34:51.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T12:34:51.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T12:34:51.568 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T12:34:51.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T12:34:51.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.569+0000 7f99af200640 1 Processor -- start 2024-04-25T12:34:51.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.570+0000 7f99af200640 1 -- start start 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99af200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80c90e0 0x7f99a80c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99af200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99a815c550 0x7f99a80ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99af200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a80c5370 con 0x7f99a80719a0 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99af200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f99a80c5650 con 0x7f99a815c550 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99af200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f99a80c5930 con 0x7f99a80c90e0 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99a7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80c90e0 0x7f99a80c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99a7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80c90e0 0x7f99a80c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36060/0 (socket says 172.21.15.123:36060) 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99a7e00640 1 -- 172.21.15.123:0/1548755506 learned_addr learned my addr 172.21.15.123:0/1548755506 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.572+0000 7f99a7400640 1 --2- 172.21.15.123:0/1548755506 >> v2:172.21.15.153:3300/0 conn(0x7f99a815c550 0x7f99a80ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.573+0000 7f99a7e00640 1 -- 172.21.15.123:0/1548755506 >> v2:172.21.15.153:3300/0 conn(0x7f99a815c550 msgr2=0x7f99a80ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.573+0000 7f99a7e00640 1 --2- 172.21.15.123:0/1548755506 >> v2:172.21.15.153:3300/0 conn(0x7f99a815c550 0x7f99a80ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.573+0000 7f99a7e00640 1 -- 172.21.15.123:0/1548755506 >> v1:172.21.15.123:6789/0 conn(0x7f99a80719a0 legacy=0x7f99a815fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.573+0000 7f99a7e00640 1 -- 172.21.15.123:0/1548755506 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a80c88e0 con 0x7f99a80c90e0 2024-04-25T12:34:51.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.575+0000 7f99a7e00640 1 --2- 172.21.15.123:0/1548755506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80c90e0 0x7f99a80c9af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9998009870 tx=0x7f999802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ca3757dac97546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.575+0000 7f99a6a00640 1 -- 172.21.15.123:0/1548755506 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998037070 con 0x7f99a80c90e0 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.575+0000 7f99a6a00640 1 -- 172.21.15.123:0/1548755506 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999802ae50 con 0x7f99a80c90e0 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.575+0000 7f99a6a00640 1 -- 172.21.15.123:0/1548755506 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998032860 con 0x7f99a80c90e0 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 -- 172.21.15.123:0/1548755506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80c90e0 msgr2=0x7f99a80c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 --2- 172.21.15.123:0/1548755506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80c90e0 0x7f99a80c9af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9998009870 tx=0x7f999802a300 comp rx=0 tx=0).stop 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 -- 172.21.15.123:0/1548755506 shutdown_connections 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 --2- 172.21.15.123:0/1548755506 >> v2:172.21.15.153:3300/0 conn(0x7f99a815c550 0x7f99a80ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 --2- 172.21.15.123:0/1548755506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80c90e0 0x7f99a80c9af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 -- 172.21.15.123:0/1548755506 >> 172.21.15.123:0/1548755506 conn(0x7f99a8070e80 msgr2=0x7f99a8071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 -- 172.21.15.123:0/1548755506 shutdown_connections 2024-04-25T12:34:51.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.577+0000 7f99af200640 1 -- 172.21.15.123:0/1548755506 wait complete. 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.581+0000 7f99af200640 1 Processor -- start 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.581+0000 7f99af200640 1 -- start start 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.583+0000 7f99af200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80719a0 0x7f99a80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.583+0000 7f99af200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99a80c90e0 0x7f99a81e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.583+0000 7f99af200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a815c550 0x7f99a81e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.583+0000 7f99af200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8156c20 con 0x7f99a80c90e0 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.583+0000 7f99af200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99a80ca650 con 0x7f99a815c550 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.583+0000 7f99af200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f99a815c960 con 0x7f99a80719a0 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.584+0000 7f99a7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80719a0 0x7f99a80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.584+0000 7f99a7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80719a0 0x7f99a80cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36062/0 (socket says 172.21.15.123:36062) 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.584+0000 7f99a7e00640 1 -- 172.21.15.123:0/3915344717 learned_addr learned my addr 172.21.15.123:0/3915344717 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.584+0000 7f99ad200640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a815c550 0x7f99a81e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.584+0000 7f99a7400640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99a80c90e0 0x7f99a81e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.586+0000 7f99a7e00640 1 -- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a815c550 msgr2=0x7f99a81e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.586+0000 7f99a7e00640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a815c550 0x7f99a81e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.586+0000 7f99a7e00640 1 -- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99a80c90e0 msgr2=0x7f99a81e79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.586+0000 7f99a7e00640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99a80c90e0 0x7f99a81e79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.586+0000 7f99a7e00640 1 -- 172.21.15.123:0/3915344717 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a80cdb50 con 0x7f99a80719a0 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.586+0000 7f99a7400640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99a80c90e0 0x7f99a81e79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.588+0000 7f99a7e00640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80719a0 0x7f99a80cd3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f99980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.589+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998041020 con 0x7f99a80719a0 2024-04-25T12:34:51.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.589+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999803c870 con 0x7f99a80719a0 2024-04-25T12:34:51.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.589+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998037040 con 0x7f99a80719a0 2024-04-25T12:34:51.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.589+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a8148210 con 0x7f99a80719a0 2024-04-25T12:34:51.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.590+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a814e0b0 con 0x7f99a80719a0 2024-04-25T12:34:51.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.591+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99a8076700 con 0x7f99a80719a0 2024-04-25T12:34:51.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.594+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9998044020 con 0x7f99a80719a0 2024-04-25T12:34:51.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.596+0000 7f999fe00640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9988074530 0x7f9988076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.596+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f999807bfc0 con 0x7f99a80719a0 2024-04-25T12:34:51.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.596+0000 7f99a7400640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9988074530 0x7f9988076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.596+0000 7f999fe00640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f998807ea10 0x7f9988080f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.596+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99880815d0 con 0x7f998807ea10 2024-04-25T12:34:51.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.597+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f999807c1c0 con 0x7f99a80719a0 2024-04-25T12:34:51.598 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T12:34:51.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T12:34:51.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.597+0000 7f99ad200640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f998807ea10 0x7f9988080f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.599+0000 7f99ad200640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f998807ea10 0x7f9988080f00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.599+0000 7f99a7400640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9988074530 0x7f9988076a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99a8071940 tx=0x7f9994009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.603+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99880815d0 con 0x7f998807ea10 2024-04-25T12:34:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.639+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99a8074b80 con 0x7f998807ea10 2024-04-25T12:34:51.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.643+0000 7f999fe00640 1 -- 172.21.15.123:0/3915344717 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f99a8074b80 con 0x7f998807ea10 2024-04-25T12:34:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.648+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f998807ea10 msgr2=0x7f9988080f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.648+0000 7f99af200640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f998807ea10 0x7f9988080f00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.648+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9988074530 msgr2=0x7f9988076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.648+0000 7f99af200640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9988074530 0x7f9988076a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99a8071940 tx=0x7f9994009190 comp rx=0 tx=0).stop 2024-04-25T12:34:51.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.648+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80719a0 msgr2=0x7f99a80cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.648+0000 7f99af200640 1 --2- 172.21.15.123:0/3915344717 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99a80719a0 0x7f99a80cd3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f99980046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.650+0000 7f99ad200640 1 -- 172.21.15.123:0/3915344717 reap_dead start 2024-04-25T12:34:51.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.650+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 shutdown_connections 2024-04-25T12:34:51.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.650+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 >> 172.21.15.123:0/3915344717 conn(0x7f99a8070e80 msgr2=0x7f99a8152820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.652+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 shutdown_connections 2024-04-25T12:34:51.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.652+0000 7f99af200640 1 -- 172.21.15.123:0/3915344717 wait complete. 2024-04-25T12:34:51.691 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T12:34:51.692 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T12:34:51.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.968+0000 7f4b92600640 1 Processor -- start 2024-04-25T12:34:51.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.968+0000 7f4b92600640 1 -- start start 2024-04-25T12:34:51.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.971+0000 7f4b92600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4b8c0c2720 0x7f4b8c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.971+0000 7f4b92600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0ccaa0 0x7f4b8c16cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b92600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b8c0cceb0 0x7f4b8c16d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b91200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4b8c0c2720 0x7f4b8c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0ccaa0 0x7f4b8c16cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b91200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4b8c0c2720 0x7f4b8c0cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55402/0 (socket says 172.21.15.123:55402) 2024-04-25T12:34:51.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0ccaa0 0x7f4b8c16cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36078/0 (socket says 172.21.15.123:36078) 2024-04-25T12:34:51.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b91200640 1 -- 172.21.15.123:0/1391705678 learned_addr learned my addr 172.21.15.123:0/1391705678 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c0c4030 con 0x7f4b8c0cceb0 2024-04-25T12:34:51.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c0c42d0 con 0x7f4b8c0c2720 2024-04-25T12:34:51.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.972+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c0c4590 con 0x7f4b8c0ccaa0 2024-04-25T12:34:51.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.973+0000 7f4b91200640 1 -- 172.21.15.123:0/1391705678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0ccaa0 msgr2=0x7f4b8c16cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.973+0000 7f4b91200640 1 --2- 172.21.15.123:0/1391705678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0ccaa0 0x7f4b8c16cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.973+0000 7f4b91200640 1 -- 172.21.15.123:0/1391705678 >> v2:172.21.15.123:3300/0 conn(0x7f4b8c0cceb0 msgr2=0x7f4b8c16d320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:34:51.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.974+0000 7f4b91c00640 1 --2- 172.21.15.123:0/1391705678 >> v2:172.21.15.123:3300/0 conn(0x7f4b8c0cceb0 0x7f4b8c16d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.975+0000 7f4b91200640 1 --2- 172.21.15.123:0/1391705678 >> v2:172.21.15.123:3300/0 conn(0x7f4b8c0cceb0 0x7f4b8c16d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.975+0000 7f4b91200640 1 -- 172.21.15.123:0/1391705678 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c0c6fc0 con 0x7f4b8c0c2720 2024-04-25T12:34:51.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.977+0000 7f4b91c00640 1 --2- 172.21.15.123:0/1391705678 >> v2:172.21.15.123:3300/0 conn(0x7f4b8c0cceb0 0x7f4b8c16d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:51.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.977+0000 7f4b91200640 1 --2- 172.21.15.123:0/1391705678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c0cc090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c06d450 tx=0x7f4b7802a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=964714634ffd34f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.981+0000 7f4b8b400640 1 -- 172.21.15.123:0/1391705678 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78037070 con 0x7f4b8c0c2720 2024-04-25T12:34:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.981+0000 7f4b8b400640 1 -- 172.21.15.123:0/1391705678 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b78032440 con 0x7f4b8c0c2720 2024-04-25T12:34:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.981+0000 7f4b8b400640 1 -- 172.21.15.123:0/1391705678 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78037070 con 0x7f4b8c0c2720 2024-04-25T12:34:51.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.982+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0c2720 msgr2=0x7f4b8c0cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.982+0000 7f4b92600640 1 --2- 172.21.15.123:0/1391705678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c0cc090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c06d450 tx=0x7f4b7802a160 comp rx=0 tx=0).stop 2024-04-25T12:34:51.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.985+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 shutdown_connections 2024-04-25T12:34:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.985+0000 7f4b92600640 1 --2- 172.21.15.123:0/1391705678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0ccaa0 0x7f4b8c16cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.985+0000 7f4b92600640 1 --2- 172.21.15.123:0/1391705678 >> v2:172.21.15.123:3300/0 conn(0x7f4b8c0cceb0 0x7f4b8c16d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.985+0000 7f4b92600640 1 --2- 172.21.15.123:0/1391705678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c0cc090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.985+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 >> 172.21.15.123:0/1391705678 conn(0x7f4b8c070b90 msgr2=0x7f4b8c070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.986+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 shutdown_connections 2024-04-25T12:34:51.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.986+0000 7f4b92600640 1 -- 172.21.15.123:0/1391705678 wait complete. 2024-04-25T12:34:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.988+0000 7f902be00640 1 Processor -- start 2024-04-25T12:34:51.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.988+0000 7f902be00640 1 -- start start 2024-04-25T12:34:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.989+0000 7f094ae00640 1 Processor -- start 2024-04-25T12:34:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.989+0000 7f094ae00640 1 -- start start 2024-04-25T12:34:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.991+0000 7f4b92600640 1 Processor -- start 2024-04-25T12:34:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.991+0000 7f4b92600640 1 -- start start 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.992+0000 7f094ae00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f093c045350 0x7f093c0df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.992+0000 7f094ae00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f093c045b50 0x7f093c046aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.992+0000 7f094ae00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093c0dcb60 con 0x7f093c046fe0 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.992+0000 7f094ae00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f093c0066f0 con 0x7f093c045350 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.992+0000 7f094ae00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f093c0334e0 con 0x7f093c045b50 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f0949000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f093c045b50 0x7f093c046aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f0949a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f093c045350 0x7f093c0df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f094a400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f093c046fe0 0x7f093c0356c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56024/0 (socket says 172.21.15.123:56024) 2024-04-25T12:34:51.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f094a400640 1 -- 172.21.15.123:0/443084120 learned_addr learned my addr 172.21.15.123:0/443084120 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.994+0000 7f0949000640 1 -- 172.21.15.123:0/443084120 >> v2:172.21.15.123:3300/0 conn(0x7f093c045350 msgr2=0x7f093c0df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f902be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f90240d9830 0x7f90240dff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f902be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9024007970 0x7f9024037480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f902be00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90240d9c40 con 0x7f9024037070 2024-04-25T12:34:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f902be00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f902404fef0 con 0x7f9024007970 2024-04-25T12:34:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.993+0000 7f902be00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f90240e0460 con 0x7f90240d9830 2024-04-25T12:34:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.995+0000 7f902aa00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f90240d9830 0x7f90240dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.995+0000 7f902b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9024007970 0x7f9024037480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.995+0000 7f902b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9024007970 0x7f9024037480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35466/0 (socket says 172.21.15.123:35466) 2024-04-25T12:34:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.995+0000 7f902b400640 1 -- 172.21.15.123:0/3935808733 learned_addr learned my addr 172.21.15.123:0/3935808733 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:51.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.994+0000 7f0949000640 1 --2- 172.21.15.123:0/443084120 >> v2:172.21.15.123:3300/0 conn(0x7f093c045350 0x7f093c0df630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.994+0000 7f0949000640 1 -- 172.21.15.123:0/443084120 >> v1:172.21.15.123:6789/0 conn(0x7f093c046fe0 legacy=0x7f093c0356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.994+0000 7f0949000640 1 -- 172.21.15.123:0/443084120 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c0dce00 con 0x7f093c045b50 2024-04-25T12:34:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f0949000640 1 --2- 172.21.15.123:0/443084120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 0x7f093c046aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f093c006870 tx=0x7f09440a9a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77326ee3c3547c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f4b92600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c203290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f0943e00640 1 -- 172.21.15.123:0/443084120 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440ad6c0 con 0x7f093c045b50 2024-04-25T12:34:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f0943e00640 1 -- 172.21.15.123:0/443084120 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09440ad860 con 0x7f093c045b50 2024-04-25T12:34:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f0943e00640 1 -- 172.21.15.123:0/443084120 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440adcd0 con 0x7f093c045b50 2024-04-25T12:34:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f4b92600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0ccaa0 0x7f4b8c1ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.997+0000 7f87d7e00640 1 Processor -- start 2024-04-25T12:34:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f902b400640 1 -- 172.21.15.123:0/3935808733 >> v2:172.21.15.153:3300/0 conn(0x7f90240d9830 msgr2=0x7f90240dff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f902b400640 1 --2- 172.21.15.123:0/3935808733 >> v2:172.21.15.153:3300/0 conn(0x7f90240d9830 0x7f90240dff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f902b400640 1 -- 172.21.15.123:0/3935808733 >> v1:172.21.15.123:6789/0 conn(0x7f9024037070 legacy=0x7f9024007260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.996+0000 7f902b400640 1 -- 172.21.15.123:0/3935808733 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90240064e0 con 0x7f9024007970 2024-04-25T12:34:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.998+0000 7f094ae00640 1 -- 172.21.15.123:0/443084120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 msgr2=0x7f093c046aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.998+0000 7f094ae00640 1 --2- 172.21.15.123:0/443084120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 0x7f093c046aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f093c006870 tx=0x7f09440a9a50 comp rx=0 tx=0).stop 2024-04-25T12:34:52.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.997+0000 7f87d7e00640 1 -- start start 2024-04-25T12:34:52.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.998+0000 7f902b400640 1 --2- 172.21.15.123:0/3935808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f9024037480 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f902c065b40 tx=0x7f902c0a7930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1676e4e6558076d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.998+0000 7f4b92600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0cceb0 0x7f4b8c0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f9029600640 1 -- 172.21.15.123:0/3935808733 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0ad4a0 con 0x7f9024007970 2024-04-25T12:34:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0ccaa0 0x7f4b8c1ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f094ae00640 1 -- 172.21.15.123:0/443084120 shutdown_connections 2024-04-25T12:34:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f9029600640 1 -- 172.21.15.123:0/3935808733 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c0aa070 con 0x7f9024007970 2024-04-25T12:34:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f9029600640 1 -- 172.21.15.123:0/3935808733 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0ad8e0 con 0x7f9024007970 2024-04-25T12:34:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f094ae00640 1 --2- 172.21.15.123:0/443084120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 0x7f093c046aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f094ae00640 1 --2- 172.21.15.123:0/443084120 >> v2:172.21.15.123:3300/0 conn(0x7f093c045350 0x7f093c0df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f094ae00640 1 -- 172.21.15.123:0/443084120 >> 172.21.15.123:0/443084120 conn(0x7f093c0503b0 msgr2=0x7f093c0507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f094ae00640 1 -- 172.21.15.123:0/443084120 shutdown_connections 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b92600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c0cc5d0 con 0x7f4b8c0c2720 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b91c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0cceb0 0x7f4b8c0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b91c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0cceb0 0x7f4b8c0c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36090/0 (socket says 172.21.15.123:36090) 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b91c00640 1 -- 172.21.15.123:0/865388622 learned_addr learned my addr 172.21.15.123:0/865388622 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c0c7580 con 0x7f4b8c0ccaa0 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c0b36c0 con 0x7f4b8c0cceb0 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f4b91200640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c203290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:51.999+0000 7f094ae00640 1 -- 172.21.15.123:0/443084120 wait complete. 2024-04-25T12:34:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f87d7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d80c4380 0x7f87d80c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f902be00640 1 -- 172.21.15.123:0/3935808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 msgr2=0x7f9024037480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f902be00640 1 --2- 172.21.15.123:0/3935808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f9024037480 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f902c065b40 tx=0x7f902c0a7930 comp rx=0 tx=0).stop 2024-04-25T12:34:52.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f4b91200640 1 -- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0cceb0 msgr2=0x7f4b8c0c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f87d7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87d80cc720 0x7f87d80b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f87d7e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d80ccc30 con 0x7f87d806c570 2024-04-25T12:34:52.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f87d7e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f87d80c9bc0 con 0x7f87d80cc720 2024-04-25T12:34:52.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f87d7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f87d80c9600 con 0x7f87d80c4380 2024-04-25T12:34:52.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f4b91200640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0cceb0 0x7f4b8c0c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f902be00640 1 -- 172.21.15.123:0/3935808733 shutdown_connections 2024-04-25T12:34:52.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f902be00640 1 --2- 172.21.15.123:0/3935808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f9024037480 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f902be00640 1 --2- 172.21.15.123:0/3935808733 >> v2:172.21.15.153:3300/0 conn(0x7f90240d9830 0x7f90240dff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f902be00640 1 -- 172.21.15.123:0/3935808733 >> 172.21.15.123:0/3935808733 conn(0x7f9024054840 msgr2=0x7f9024054c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f87d6000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87d80cc720 0x7f87d80b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f87d6000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87d80cc720 0x7f87d80b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35494/0 (socket says 172.21.15.123:35494) 2024-04-25T12:34:52.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f87d6000640 1 -- 172.21.15.123:0/517140681 learned_addr learned my addr 172.21.15.123:0/517140681 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f902be00640 1 -- 172.21.15.123:0/3935808733 shutdown_connections 2024-04-25T12:34:52.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f902be00640 1 -- 172.21.15.123:0/3935808733 wait complete. 2024-04-25T12:34:52.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f4318a00640 1 Processor -- start 2024-04-25T12:34:52.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.001+0000 7f4318a00640 1 -- start start 2024-04-25T12:34:52.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.002+0000 7f87d6000640 1 -- 172.21.15.123:0/517140681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d80c4380 msgr2=0x7f87d80c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:52.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f4b91200640 1 -- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0ccaa0 msgr2=0x7f4b8c1ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f4b91200640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0ccaa0 0x7f4b8c1ffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.000+0000 7f4b91200640 1 -- 172.21.15.123:0/865388622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c165c70 con 0x7f4b8c0c2720 2024-04-25T12:34:52.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.004+0000 7f094ae00640 1 Processor -- start 2024-04-25T12:34:52.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.004+0000 7f094ae00640 1 -- start start 2024-04-25T12:34:52.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.004+0000 7f4318a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.004+0000 7f4318a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f430c0375e0 0x7f430c0e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.004+0000 7f4318a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f430c0e3810 con 0x7f430c00f200 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.004+0000 7f4318a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f430c04aa30 con 0x7f430c0375e0 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.004+0000 7f4318a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f430c04acf0 con 0x7f430c04a4f0 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.005+0000 7f4b91200640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c203290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0c4450 tx=0x7f4b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.005+0000 7f4312a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.005+0000 7f4312a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36116/0 (socket says 172.21.15.123:36116) 2024-04-25T12:34:52.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.005+0000 7f4312a00640 1 -- 172.21.15.123:0/884462634 learned_addr learned my addr 172.21.15.123:0/884462634 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.005+0000 7f4b88c00640 1 -- 172.21.15.123:0/865388622 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78041070 con 0x7f4b8c0c2720 2024-04-25T12:34:52.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f902be00640 1 Processor -- start 2024-04-25T12:34:52.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4312a00640 1 -- 172.21.15.123:0/884462634 >> v2:172.21.15.153:3300/0 conn(0x7f430c0375e0 msgr2=0x7f430c0e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:52.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4312a00640 1 --2- 172.21.15.123:0/884462634 >> v2:172.21.15.153:3300/0 conn(0x7f430c0375e0 0x7f430c0e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4312a00640 1 -- 172.21.15.123:0/884462634 >> v1:172.21.15.123:6789/0 conn(0x7f430c00f200 legacy=0x7f430c0075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f902be00640 1 -- start start 2024-04-25T12:34:52.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.002+0000 7f87d6000640 1 --2- 172.21.15.123:0/517140681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d80c4380 0x7f87d80c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.002+0000 7f87d6000640 1 -- 172.21.15.123:0/517140681 >> v1:172.21.15.153:6789/0 conn(0x7f87d806c570 legacy=0x7f87d815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:52.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.002+0000 7f87d6000640 1 -- 172.21.15.123:0/517140681 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d80c98e0 con 0x7f87d80cc720 2024-04-25T12:34:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c2027f0 con 0x7f4b8c0c2720 2024-04-25T12:34:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1664f0 con 0x7f4b8c0c2720 2024-04-25T12:34:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4b88c00640 1 -- 172.21.15.123:0/865388622 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b780307c0 con 0x7f4b8c0c2720 2024-04-25T12:34:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4b88c00640 1 -- 172.21.15.123:0/865388622 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78037070 con 0x7f4b8c0c2720 2024-04-25T12:34:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.006+0000 7f4312a00640 1 -- 172.21.15.123:0/884462634 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c04afd0 con 0x7f430c04a4f0 2024-04-25T12:34:52.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f4312a00640 1 --2- 172.21.15.123:0/884462634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0ddb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4300009870 tx=0x7f430002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f234316ab8bfe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f094ae00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093c045350 0x7f093c17ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f094ae00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 0x7f093c04a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f094ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c046fe0 0x7f093c0d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f094ae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f093c033a10 con 0x7f093c046fe0 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f094ae00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f093c033730 con 0x7f093c045b50 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f094ae00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f093c0063b0 con 0x7f093c045350 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f0949a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093c045350 0x7f093c17ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f0949a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093c045350 0x7f093c17ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36122/0 (socket says 172.21.15.123:36122) 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.007+0000 7f0949a00640 1 -- 172.21.15.123:0/1542904379 learned_addr learned my addr 172.21.15.123:0/1542904379 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f094a400640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c046fe0 0x7f093c0d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f0949000640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 0x7f093c04a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f87d6000640 1 --2- 172.21.15.123:0/517140681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80cc720 0x7f87d80b2fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009870 tx=0x7f87d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c6e52a66c3ba4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f094a400640 1 -- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093c045350 msgr2=0x7f093c17ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f094a400640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093c045350 0x7f093c17ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f094a400640 1 -- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 msgr2=0x7f093c04a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.008+0000 7f094a400640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 0x7f093c04a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.009+0000 7f094a400640 1 -- 172.21.15.123:0/1542904379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c0d4aa0 con 0x7f093c046fe0 2024-04-25T12:34:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.009+0000 7f094a400640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c046fe0 0x7f093c0d2a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f093000b410 tx=0x7f093000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.009+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b64005290 con 0x7f4b8c0c2720 2024-04-25T12:34:52.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.010+0000 7f4312000640 1 -- 172.21.15.123:0/884462634 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300037070 con 0x7f430c04a4f0 2024-04-25T12:34:52.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.010+0000 7f4312000640 1 -- 172.21.15.123:0/884462634 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4300032440 con 0x7f430c04a4f0 2024-04-25T12:34:52.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.010+0000 7f4312000640 1 -- 172.21.15.123:0/884462634 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300032810 con 0x7f430c04a4f0 2024-04-25T12:34:52.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.011+0000 7f87d5600640 1 -- 172.21.15.123:0/517140681 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0037070 con 0x7f87d80cc720 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.011+0000 7f87d5600640 1 -- 172.21.15.123:0/517140681 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d0032440 con 0x7f87d80cc720 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.011+0000 7f87d5600640 1 -- 172.21.15.123:0/517140681 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0037070 con 0x7f87d80cc720 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f4318a00640 1 -- 172.21.15.123:0/884462634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 msgr2=0x7f430c0ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f4318a00640 1 --2- 172.21.15.123:0/884462634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0ddb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4300009870 tx=0x7f430002a270 comp rx=0 tx=0).stop 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f4318a00640 1 -- 172.21.15.123:0/884462634 shutdown_connections 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f4318a00640 1 --2- 172.21.15.123:0/884462634 >> v2:172.21.15.153:3300/0 conn(0x7f430c0375e0 0x7f430c0e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f4318a00640 1 --2- 172.21.15.123:0/884462634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0ddb80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f4318a00640 1 -- 172.21.15.123:0/884462634 >> 172.21.15.123:0/884462634 conn(0x7f430c00e550 msgr2=0x7f430c00e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f0941600640 1 -- 172.21.15.123:0/1542904379 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930015020 con 0x7f093c046fe0 2024-04-25T12:34:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c179830 con 0x7f093c046fe0 2024-04-25T12:34:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f0941600640 1 -- 172.21.15.123:0/1542904379 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0930011830 con 0x7f093c046fe0 2024-04-25T12:34:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.013+0000 7f87d7e00640 1 -- 172.21.15.123:0/517140681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80cc720 msgr2=0x7f87d80b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f0941600640 1 -- 172.21.15.123:0/1542904379 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930015020 con 0x7f093c046fe0 2024-04-25T12:34:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.013+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c0d2f60 con 0x7f093c046fe0 2024-04-25T12:34:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.013+0000 7f87d7e00640 1 --2- 172.21.15.123:0/517140681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80cc720 0x7f87d80b2fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009870 tx=0x7f87d002a210 comp rx=0 tx=0).stop 2024-04-25T12:34:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f87d7e00640 1 -- 172.21.15.123:0/517140681 shutdown_connections 2024-04-25T12:34:52.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f4318a00640 1 -- 172.21.15.123:0/884462634 shutdown_connections 2024-04-25T12:34:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f87d7e00640 1 --2- 172.21.15.123:0/517140681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80cc720 0x7f87d80b2fa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f87d7e00640 1 --2- 172.21.15.123:0/517140681 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d80c4380 0x7f87d80c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f87d7e00640 1 -- 172.21.15.123:0/517140681 >> 172.21.15.123:0/517140681 conn(0x7f87d8070b90 msgr2=0x7f87d8070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f4318a00640 1 -- 172.21.15.123:0/884462634 wait complete. 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.011+0000 7f902be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f902404c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.011+0000 7f902be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9024037070 0x7f90240cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f902be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90240d9830 0x7f9024179f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f902be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9024006a60 con 0x7f9024007970 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f902be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9024006780 con 0x7f90240d9830 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f902be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9024006d40 con 0x7f9024037070 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f902b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90240d9830 0x7f9024179f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f902b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90240d9830 0x7f9024179f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55458/0 (socket says 172.21.15.123:55458) 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.012+0000 7f902b400640 1 -- 172.21.15.123:0/1089318073 learned_addr learned my addr 172.21.15.123:0/1089318073 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f902aa00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f902404c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.015+0000 7f4b88c00640 1 -- 172.21.15.123:0/865388622 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b78044070 con 0x7f4b8c0c2720 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.015+0000 7f902a000640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9024037070 0x7f90240cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.016+0000 7f902aa00640 1 -- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9024037070 msgr2=0x7f90240cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.016+0000 7f902aa00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9024037070 0x7f90240cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.016+0000 7f902aa00640 1 -- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90240d9830 msgr2=0x7f9024179f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.016+0000 7f902aa00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90240d9830 0x7f9024179f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.016+0000 7f902aa00640 1 -- 172.21.15.123:0/1089318073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902404ca40 con 0x7f9024007970 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.017+0000 7f902aa00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f902404c500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f901c00d550 tx=0x7f901c00da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.017+0000 7f4b88c00640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b6c0745b0 0x7f4b6c076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.017+0000 7f4b88c00640 1 -- 172.21.15.123:0/865388622 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4b780b5f90 con 0x7f4b8c0c2720 2024-04-25T12:34:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.018+0000 7ffb8be00640 1 Processor -- start 2024-04-25T12:34:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.018+0000 7f4b8be00640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b6c0745b0 0x7f4b6c076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.018+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f093c179620 con 0x7f093c046fe0 2024-04-25T12:34:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.014+0000 7f87d7e00640 1 -- 172.21.15.123:0/517140681 shutdown_connections 2024-04-25T12:34:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.016+0000 7f87d7e00640 1 -- 172.21.15.123:0/517140681 wait complete. 2024-04-25T12:34:52.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.019+0000 7f4b8be00640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b6c0745b0 0x7f4b6c076ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006fd0 tx=0x7f4b7c013040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.019+0000 7f4318a00640 1 Processor -- start 2024-04-25T12:34:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.019+0000 7f4318a00640 1 -- start start 2024-04-25T12:34:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.018+0000 7ffb8be00640 1 -- start start 2024-04-25T12:34:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.021+0000 7ffb8be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb8c1575f0 0x7ffb8c0c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.021+0000 7ffb8be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb8c06c570 0x7ffb8c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.021+0000 7f87d7e00640 1 Processor -- start 2024-04-25T12:34:52.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.021+0000 7f87d7e00640 1 -- start start 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.021+0000 7ffb8a000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb8c06c570 0x7ffb8c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.022+0000 7ffb8aa00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb8c1575f0 0x7ffb8c0c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.021+0000 7ffb8be00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c06d750 con 0x7ffb8c1607f0 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.022+0000 7ffb8be00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8c0c81d0 con 0x7ffb8c06c570 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.022+0000 7ffb8be00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8c0c65b0 con 0x7ffb8c1575f0 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.022+0000 7ffb8b400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ffb8c1607f0 0x7ffb8c160c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:41554/0 (socket says 172.21.15.123:41554) 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.022+0000 7f4318a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f430c00f200 0x7f430c0e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.022+0000 7ffb8b400640 1 -- 172.21.15.123:0/2226593273 learned_addr learned my addr 172.21.15.123:0/2226593273 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:52.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4318a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f430c0375e0 0x7f430c0d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4318a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7ffb89600640 1 -- 172.21.15.123:0/2226593273 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876369991 0 0) 0x7ffb8c06d750 con 0x7ffb8c1607f0 2024-04-25T12:34:52.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7ffb89600640 1 -- 172.21.15.123:0/2226593273 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb70003520 con 0x7ffb8c1607f0 2024-04-25T12:34:52.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7ffb8aa00640 1 -- 172.21.15.123:0/2226593273 >> v2:172.21.15.123:3300/0 conn(0x7ffb8c06c570 msgr2=0x7ffb8c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:52.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7ffb8aa00640 1 --2- 172.21.15.123:0/2226593273 >> v2:172.21.15.123:3300/0 conn(0x7ffb8c06c570 0x7ffb8c06c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7ffb8aa00640 1 -- 172.21.15.123:0/2226593273 >> v1:172.21.15.153:6789/0 conn(0x7ffb8c1607f0 legacy=0x7ffb8c160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7ffb8aa00640 1 -- 172.21.15.123:0/2226593273 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0c6890 con 0x7ffb8c1575f0 2024-04-25T12:34:52.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4318a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f430c050c10 con 0x7f430c00f200 2024-04-25T12:34:52.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4318a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f430c04b590 con 0x7f430c0375e0 2024-04-25T12:34:52.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4318a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f430c007cb0 con 0x7f430c04a4f0 2024-04-25T12:34:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4312a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f430c0375e0 0x7f430c0d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4312a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f430c0375e0 0x7f430c0d8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55470/0 (socket says 172.21.15.123:55470) 2024-04-25T12:34:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.023+0000 7f4312a00640 1 -- 172.21.15.123:0/3768857735 learned_addr learned my addr 172.21.15.123:0/3768857735 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f4313400640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f430c00f200 0x7f430c0e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d806c570 0x7f87d81f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80c4380 0x7f87d81f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87d80cc720 0x7f87d81f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f4312a00640 1 -- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 msgr2=0x7f430c0e67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f431a000640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f4312a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0e67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d6a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d806c570 0x7f87d81f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d6a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d806c570 0x7f87d81f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36144/0 (socket says 172.21.15.123:36144) 2024-04-25T12:34:52.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81528b0 con 0x7f87d80c4380 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d7e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87d80c2c60 con 0x7f87d80cc720 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f87d7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f87d8152a30 con 0x7f87d806c570 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f87d7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87d80cc720 0x7f87d81f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f87d7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87d80cc720 0x7f87d81f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55480/0 (socket says 172.21.15.123:55480) 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f87d7400640 1 -- 172.21.15.123:0/1097528536 learned_addr learned my addr 172.21.15.123:0/1097528536 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.024+0000 7f4312a00640 1 -- 172.21.15.123:0/3768857735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f430c00f200 msgr2=0x7f430c0e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f4312a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f430c00f200 0x7f430c0e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f4312a00640 1 -- 172.21.15.123:0/3768857735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c0e5b20 con 0x7f430c0375e0 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f431a000640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:52.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f4313400640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f430c00f200 0x7f430c0e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7ffb8aa00640 1 --2- 172.21.15.123:0/2226593273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c1575f0 0x7ffb8c0c4960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0c4830 tx=0x7ffb7c02a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8be106a495a5b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f87d6000640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80c4380 0x7f87d81f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7ffb89600640 1 -- 172.21.15.123:0/2226593273 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c037070 con 0x7ffb8c1575f0 2024-04-25T12:34:52.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7ffb89600640 1 -- 172.21.15.123:0/2226593273 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c032440 con 0x7ffb8c1575f0 2024-04-25T12:34:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f4312a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f430c0375e0 0x7f430c0d8d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4300009870 tx=0x7f43000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.026+0000 7f87d6a00640 1 -- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87d80cc720 msgr2=0x7f87d81f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.026+0000 7f4307400640 1 -- 172.21.15.123:0/3768857735 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300041070 con 0x7f430c0375e0 2024-04-25T12:34:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.026+0000 7f4307400640 1 -- 172.21.15.123:0/3768857735 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430003c930 con 0x7f430c0375e0 2024-04-25T12:34:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.026+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c0e1c30 con 0x7f430c0375e0 2024-04-25T12:34:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f87d6a00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87d80cc720 0x7f87d81f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f87d6a00640 1 -- 172.21.15.123:0/1097528536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80c4380 msgr2=0x7f87d81f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.026+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c0e4b00 con 0x7f430c0375e0 2024-04-25T12:34:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f9022a00640 1 -- 172.21.15.123:0/1089318073 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c0125f0 con 0x7f9024007970 2024-04-25T12:34:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f9022a00640 1 -- 172.21.15.123:0/1089318073 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f901c019070 con 0x7f9024007970 2024-04-25T12:34:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.025+0000 7f9022a00640 1 -- 172.21.15.123:0/1089318073 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c0129b0 con 0x7f9024007970 2024-04-25T12:34:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f87d6a00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80c4380 0x7f87d81f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f87d6a00640 1 -- 172.21.15.123:0/1097528536 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d81f5020 con 0x7f87d806c570 2024-04-25T12:34:52.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f87d7400640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87d80cc720 0x7f87d81f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f87d6a00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d806c570 0x7f87d81f4ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b440 tx=0x7f87c800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f87d6000640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80c4380 0x7f87d81f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.026+0000 7ffb8be00640 1 -- 172.21.15.123:0/2226593273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c1575f0 msgr2=0x7ffb8c0c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.026+0000 7ffb8be00640 1 --2- 172.21.15.123:0/2226593273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c1575f0 0x7ffb8c0c4960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0c4830 tx=0x7ffb7c02a160 comp rx=0 tx=0).stop 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7ffb8be00640 1 -- 172.21.15.123:0/2226593273 shutdown_connections 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7ffb8be00640 1 --2- 172.21.15.123:0/2226593273 >> v2:172.21.15.123:3300/0 conn(0x7ffb8c06c570 0x7ffb8c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7ffb8be00640 1 --2- 172.21.15.123:0/2226593273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c1575f0 0x7ffb8c0c4960 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0c4830 tx=0x7ffb7c02a160 comp rx=0 tx=0).stop 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7ffb8be00640 1 -- 172.21.15.123:0/2226593273 >> 172.21.15.123:0/2226593273 conn(0x7ffb8c0c9f60 msgr2=0x7ffb8c0ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7ffb8be00640 1 -- 172.21.15.123:0/2226593273 shutdown_connections 2024-04-25T12:34:52.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f4307400640 1 -- 172.21.15.123:0/3768857735 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300041070 con 0x7f430c0375e0 2024-04-25T12:34:52.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.028+0000 7ffb8be00640 1 -- 172.21.15.123:0/2226593273 wait complete. 2024-04-25T12:34:52.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.028+0000 7f87cea00640 1 -- 172.21.15.123:0/1097528536 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8011830 con 0x7f87d806c570 2024-04-25T12:34:52.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.028+0000 7f87cea00640 1 -- 172.21.15.123:0/1097528536 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c8016070 con 0x7f87d806c570 2024-04-25T12:34:52.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.028+0000 7f87cea00640 1 -- 172.21.15.123:0/1097528536 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8011c10 con 0x7f87d806c570 2024-04-25T12:34:52.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.029+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d80ab4e0 con 0x7f87d806c570 2024-04-25T12:34:52.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.029+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d8155830 con 0x7f87d806c570 2024-04-25T12:34:52.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.029+0000 7f4b88c00640 1 -- 172.21.15.123:0/865388622 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b7804b030 con 0x7f4b8c0c2720 2024-04-25T12:34:52.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.031+0000 7f6a0d000640 1 Processor -- start 2024-04-25T12:34:52.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.032+0000 7f6a0d000640 1 -- start start 2024-04-25T12:34:52.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f902be00640 1 -- 172.21.15.123:0/1089318073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902404e140 con 0x7f9024007970 2024-04-25T12:34:52.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.027+0000 7f902be00640 1 -- 172.21.15.123:0/1089318073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90240e0ec0 con 0x7f9024007970 2024-04-25T12:34:52.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.030+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90240e10c0 con 0x7f9024007970 2024-04-25T12:34:52.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.029+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f430c005740 con 0x7f430c0375e0 2024-04-25T12:34:52.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.032+0000 7f4307400640 1 -- 172.21.15.123:0/3768857735 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4300044070 con 0x7f430c0375e0 2024-04-25T12:34:52.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.031+0000 7f0941600640 1 -- 172.21.15.123:0/1542904379 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0930018020 con 0x7f093c046fe0 2024-04-25T12:34:52.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.033+0000 7f0941600640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09240746b0 0x7f0924076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.033+0000 7f0941600640 1 -- 172.21.15.123:0/1542904379 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0930095490 con 0x7f093c046fe0 2024-04-25T12:34:52.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.033+0000 7f4307400640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f42f40748b0 0x7f42f4076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.034+0000 7ffb8be00640 1 Processor -- start 2024-04-25T12:34:52.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.034+0000 7f0949a00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09240746b0 0x7f0924076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.034+0000 7f4307400640 1 -- 172.21.15.123:0/3768857735 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f430003c730 con 0x7f430c0375e0 2024-04-25T12:34:52.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.034+0000 7f4313400640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f42f40748b0 0x7f42f4076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.035+0000 7f6a0d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a080c2870 0x7f6a080ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.035+0000 7f6a0d000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080ca820 0x7f6a08166ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.035+0000 7f6a0d000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a080c4a60 con 0x7f6a08156940 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.035+0000 7f6a0d000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6a080c45d0 con 0x7f6a080c2870 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.035+0000 7f6a0d000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6a080c8dd0 con 0x7f6a080ca820 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.035+0000 7f6a05e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f6a08156940 0x7f6a0815c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:41562/0 (socket says 172.21.15.123:41562) 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f6a05e00640 1 -- 172.21.15.123:0/4262862235 learned_addr learned my addr 172.21.15.123:0/4262862235 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f6a05400640 1 --2- 172.21.15.123:0/4262862235 >> v2:172.21.15.123:3300/0 conn(0x7f6a080c2870 0x7f6a080ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f0949a00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09240746b0 0x7f0924076bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f093800ba70 tx=0x7f0938008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f0941600640 1 -- 172.21.15.123:0/1542904379 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f093001f020 con 0x7f093c046fe0 2024-04-25T12:34:52.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.034+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b0005290 con 0x7f87d806c570 2024-04-25T12:34:52.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.037+0000 7f87cea00640 1 -- 172.21.15.123:0/1097528536 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f87c801b460 con 0x7f87d806c570 2024-04-25T12:34:52.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f6a05400640 1 -- 172.21.15.123:0/4262862235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080ca820 msgr2=0x7f6a08166ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:34:52.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f6a05400640 1 --2- 172.21.15.123:0/4262862235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080ca820 0x7f6a08166ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f6a05400640 1 -- 172.21.15.123:0/4262862235 >> v1:172.21.15.153:6789/0 conn(0x7f6a08156940 legacy=0x7f6a0815c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.036+0000 7f6a05400640 1 -- 172.21.15.123:0/4262862235 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a080c9070 con 0x7f6a080c2870 2024-04-25T12:34:52.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.037+0000 7f4313400640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f42f40748b0 0x7f42f4076dc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43080015e0 tx=0x7f4308009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.035+0000 7ffb8be00640 1 -- start start 2024-04-25T12:34:52.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb8c06c570 0x7ffb8c1faa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c1607f0 0x7ffb8c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c15ef90 0x7ffb8c14a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c0c8490 con 0x7ffb8c1607f0 2024-04-25T12:34:52.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c0c6e50 con 0x7ffb8c15ef90 2024-04-25T12:34:52.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c0c7e60 con 0x7ffb8c06c570 2024-04-25T12:34:52.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c15ef90 0x7ffb8c14a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c15ef90 0x7ffb8c14a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55486/0 (socket says 172.21.15.123:55486) 2024-04-25T12:34:52.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.039+0000 7f4307400640 1 -- 172.21.15.123:0/3768857735 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f430007fe10 con 0x7f430c0375e0 2024-04-25T12:34:52.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8b400640 1 -- 172.21.15.123:0/232741584 learned_addr learned my addr 172.21.15.123:0/232741584 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.038+0000 7ffb8aa00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb8c06c570 0x7ffb8c1faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.039+0000 7ffb8a000640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c1607f0 0x7ffb8c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.039+0000 7ffb8b400640 1 -- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb8c06c570 msgr2=0x7ffb8c1faa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.039+0000 7ffb8b400640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb8c06c570 0x7ffb8c1faa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.039+0000 7ffb8b400640 1 -- 172.21.15.123:0/232741584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c1607f0 msgr2=0x7ffb8c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.040+0000 7f87cea00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f87b40745b0 0x7f87b4076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.040+0000 7f9022a00640 1 -- 172.21.15.123:0/1089318073 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f901c012b50 con 0x7f9024007970 2024-04-25T12:34:52.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.040+0000 7ffb8b400640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c1607f0 0x7ffb8c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.040+0000 7ffb8b400640 1 -- 172.21.15.123:0/232741584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c1fafd0 con 0x7ffb8c15ef90 2024-04-25T12:34:52.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.041+0000 7f6a05400640 1 --2- 172.21.15.123:0/4262862235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080c2870 0x7f6a080ca2e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a81b7606a59ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.041+0000 7ffb8b400640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c15ef90 0x7ffb8c14a5b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c760 tx=0x7ffb8400cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.041+0000 7ffb8a000640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c1607f0 0x7ffb8c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:52.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.041+0000 7ffb82a00640 1 -- 172.21.15.123:0/232741584 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84018070 con 0x7ffb8c15ef90 2024-04-25T12:34:52.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7f9022a00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f900c074530 0x7f900c076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7ffb82a00640 1 -- 172.21.15.123:0/232741584 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8400a640 con 0x7ffb8c15ef90 2024-04-25T12:34:52.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7ffb82a00640 1 -- 172.21.15.123:0/232741584 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400a9f0 con 0x7ffb8c15ef90 2024-04-25T12:34:52.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.041+0000 7f87cea00640 1 -- 172.21.15.123:0/1097528536 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f87c8094fc0 con 0x7f87d806c570 2024-04-25T12:34:52.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c14c240 con 0x7ffb8c15ef90 2024-04-25T12:34:52.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7f9022a00640 1 -- 172.21.15.123:0/1089318073 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f901c0961b0 con 0x7f9024007970 2024-04-25T12:34:52.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c1601f0 con 0x7ffb8c15ef90 2024-04-25T12:34:52.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7f87d6000640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f87b40745b0 0x7f87b4076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.042+0000 7f902a000640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f900c074530 0x7f900c076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.045+0000 7f902a000640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f900c074530 0x7f900c076a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9018006fd0 tx=0x7f9018008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.046+0000 7f6a04a00640 1 -- 172.21.15.123:0/4262862235 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4037070 con 0x7f6a080c2870 2024-04-25T12:34:52.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.045+0000 7f87cea00640 1 -- 172.21.15.123:0/1097528536 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87c805eda0 con 0x7f87d806c570 2024-04-25T12:34:52.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.045+0000 7f87d6000640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f87b40745b0 0x7f87b4076ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87d0034000 tx=0x7f87d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.046+0000 7f9022a00640 1 -- 172.21.15.123:0/1089318073 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90240e10c0 con 0x7f9024007970 2024-04-25T12:34:52.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.046+0000 7f6a04a00640 1 -- 172.21.15.123:0/4262862235 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f4032440 con 0x7f6a080c2870 2024-04-25T12:34:52.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.046+0000 7f6a04a00640 1 -- 172.21.15.123:0/4262862235 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4037070 con 0x7f6a080c2870 2024-04-25T12:34:52.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7f6a0d000640 1 -- 172.21.15.123:0/4262862235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080c2870 msgr2=0x7f6a080ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7f6a0d000640 1 --2- 172.21.15.123:0/4262862235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080c2870 0x7f6a080ca2e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402a250 comp rx=0 tx=0).stop 2024-04-25T12:34:52.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb64005290 con 0x7ffb8c15ef90 2024-04-25T12:34:52.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7ffb82a00640 1 -- 172.21.15.123:0/232741584 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffb8400ab90 con 0x7ffb8c15ef90 2024-04-25T12:34:52.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7f6a0d000640 1 -- 172.21.15.123:0/4262862235 shutdown_connections 2024-04-25T12:34:52.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7f6a0d000640 1 --2- 172.21.15.123:0/4262862235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080ca820 0x7f6a08166ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7f6a0d000640 1 --2- 172.21.15.123:0/4262862235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080c2870 0x7f6a080ca2e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.048+0000 7f6a0d000640 1 -- 172.21.15.123:0/4262862235 >> 172.21.15.123:0/4262862235 conn(0x7f6a0806c750 msgr2=0x7f6a0806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.049+0000 7f6a0d000640 1 -- 172.21.15.123:0/4262862235 shutdown_connections 2024-04-25T12:34:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.049+0000 7f6a0d000640 1 -- 172.21.15.123:0/4262862235 wait complete. 2024-04-25T12:34:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.049+0000 7ffb82a00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb6c0744e0 0x7ffb6c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.050+0000 7ffb82a00640 1 -- 172.21.15.123:0/232741584 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffb84094be0 con 0x7ffb8c15ef90 2024-04-25T12:34:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.050+0000 7ffb8aa00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb6c0744e0 0x7ffb6c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.053+0000 7f6a0d000640 1 Processor -- start 2024-04-25T12:34:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.053+0000 7f6a0d000640 1 -- start start 2024-04-25T12:34:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.054+0000 7ffb8aa00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb6c0744e0 0x7ffb6c0769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c034000 tx=0x7ffb7c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.056+0000 7f6a0d000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080c2870 0x7f6a081e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.056+0000 7f6a0d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a080ca820 0x7f6a081e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.056+0000 7f6a0d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a08156940 0x7f6a081e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.056+0000 7f6a0d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a080c4240 con 0x7f6a08156940 2024-04-25T12:34:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.056+0000 7f6a0d000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a080c86d0 con 0x7f6a080ca820 2024-04-25T12:34:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.056+0000 7f6a0d000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0815cc50 con 0x7f6a080c2870 2024-04-25T12:34:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.054+0000 7ffb82a00640 1 -- 172.21.15.123:0/232741584 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb8405ef90 con 0x7ffb8c15ef90 2024-04-25T12:34:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.057+0000 7f6a05400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a080ca820 0x7f6a081e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.057+0000 7f6a05400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a080ca820 0x7f6a081e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:55490/0 (socket says 172.21.15.123:55490) 2024-04-25T12:34:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.057+0000 7f6a05e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080c2870 0x7f6a081e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.057+0000 7f6a05e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080c2870 0x7f6a081e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36156/0 (socket says 172.21.15.123:36156) 2024-04-25T12:34:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.057+0000 7f6a05e00640 1 -- 172.21.15.123:0/3045361034 learned_addr learned my addr 172.21.15.123:0/3045361034 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.058+0000 7f6a06800640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a08156940 0x7f6a081e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.059+0000 7f6a06800640 1 -- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080c2870 msgr2=0x7f6a081e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.059+0000 7f6a06800640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080c2870 0x7f6a081e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.059+0000 7f6a06800640 1 -- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a080ca820 msgr2=0x7f6a081e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.059+0000 7f6a06800640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a080ca820 0x7f6a081e7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.059+0000 7f6a05400640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a080ca820 0x7f6a081e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.059+0000 7f6a06800640 1 -- 172.21.15.123:0/3045361034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a081e2620 con 0x7f6a08156940 2024-04-25T12:34:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.060+0000 7f6a05e00640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080c2870 0x7f6a081e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:52.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.060+0000 7f6a06800640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a08156940 0x7f6a081e5a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000bf70 tx=0x7f6a0000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.065+0000 7f69fe000640 1 -- 172.21.15.123:0/3045361034 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00019070 con 0x7f6a08156940 2024-04-25T12:34:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.065+0000 7f69fe000640 1 -- 172.21.15.123:0/3045361034 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a00014440 con 0x7f6a08156940 2024-04-25T12:34:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.065+0000 7f69fe000640 1 -- 172.21.15.123:0/3045361034 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00014830 con 0x7f6a08156940 2024-04-25T12:34:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.069+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a081e7690 con 0x7f6a08156940 2024-04-25T12:34:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.069+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a081e6670 con 0x7f6a08156940 2024-04-25T12:34:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.071+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a081e27f0 con 0x7f6a08156940 2024-04-25T12:34:52.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.077+0000 7f69fe000640 1 -- 172.21.15.123:0/3045361034 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a0001c070 con 0x7f6a08156940 2024-04-25T12:34:52.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.078+0000 7f69fe000640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f69e80744e0 0x7f69e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.079+0000 7f6a05e00640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f69e80744e0 0x7f69e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.080+0000 7f6a05e00640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f69e80744e0 0x7f69e80769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005880 tx=0x7f69f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.084+0000 7f69fe000640 1 -- 172.21.15.123:0/3045361034 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6a0005b5b0 con 0x7f6a08156940 2024-04-25T12:34:52.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.092+0000 7f69fe000640 1 -- 172.21.15.123:0/3045361034 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a0005efc0 con 0x7f6a08156940 2024-04-25T12:34:52.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.326+0000 7f92cfe00640 1 Processor -- start 2024-04-25T12:34:52.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.329+0000 7f92cfe00640 1 -- start start 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.332+0000 7f92cfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f92d00cdca0 0x7f92d00ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.332+0000 7f92cfe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f92d00c6800 0x7f92d015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.332+0000 7f92cfe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d00cedf0 con 0x7f92d01568c0 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.332+0000 7f92cfe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f92d00cebd0 con 0x7f92d00cdca0 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.332+0000 7f92cfe00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f92d00cf0d0 con 0x7f92d00c6800 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.332+0000 7f92c6000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f92d00cdca0 0x7f92d00ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.334+0000 7f92cf400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f92d00c6800 0x7f92d015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.334+0000 7f92cea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f92d01568c0 0x7f92d00ccf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56058/0 (socket says 172.21.15.123:56058) 2024-04-25T12:34:52.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.334+0000 7f92cea00640 1 -- 172.21.15.123:0/1995659961 learned_addr learned my addr 172.21.15.123:0/1995659961 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:52.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.337+0000 7f92c6000640 1 -- 172.21.15.123:0/1995659961 >> v2:172.21.15.153:3300/0 conn(0x7f92d00c6800 msgr2=0x7f92d015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:52.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.337+0000 7f92c6000640 1 --2- 172.21.15.123:0/1995659961 >> v2:172.21.15.153:3300/0 conn(0x7f92d00c6800 0x7f92d015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.337+0000 7f92c6000640 1 -- 172.21.15.123:0/1995659961 >> v1:172.21.15.123:6789/0 conn(0x7f92d01568c0 legacy=0x7f92d00ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.337+0000 7f92c6000640 1 -- 172.21.15.123:0/1995659961 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d00c7df0 con 0x7f92d00cdca0 2024-04-25T12:34:52.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.347+0000 7f92c6000640 1 --2- 172.21.15.123:0/1995659961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d00cdca0 0x7f92d00ce690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92d0159fc0 tx=0x7f92c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a5f277f70cde2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.350+0000 7f92ce000640 1 -- 172.21.15.123:0/1995659961 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0037070 con 0x7f92d00cdca0 2024-04-25T12:34:52.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.350+0000 7f92ce000640 1 -- 172.21.15.123:0/1995659961 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c0032440 con 0x7f92d00cdca0 2024-04-25T12:34:52.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.351+0000 7f92cfe00640 1 -- 172.21.15.123:0/1995659961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d00cdca0 msgr2=0x7f92d00ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.351+0000 7f92cfe00640 1 --2- 172.21.15.123:0/1995659961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d00cdca0 0x7f92d00ce690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92d0159fc0 tx=0x7f92c002a1a0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.355+0000 7f92cfe00640 1 -- 172.21.15.123:0/1995659961 shutdown_connections 2024-04-25T12:34:52.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.355+0000 7f92cfe00640 1 --2- 172.21.15.123:0/1995659961 >> v2:172.21.15.153:3300/0 conn(0x7f92d00c6800 0x7f92d015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.355+0000 7f92cfe00640 1 --2- 172.21.15.123:0/1995659961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d00cdca0 0x7f92d00ce690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.355+0000 7f92cfe00640 1 -- 172.21.15.123:0/1995659961 >> 172.21.15.123:0/1995659961 conn(0x7f92d006dcd0 msgr2=0x7f92d006e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.361+0000 7f92cfe00640 1 -- 172.21.15.123:0/1995659961 shutdown_connections 2024-04-25T12:34:52.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.361+0000 7f92cfe00640 1 -- 172.21.15.123:0/1995659961 wait complete. 2024-04-25T12:34:52.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.382+0000 7f92cfe00640 1 Processor -- start 2024-04-25T12:34:52.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.382+0000 7f92cfe00640 1 -- start start 2024-04-25T12:34:52.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.400+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92d00c6800 0x7f92d01e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.400+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92d01e4e50 0x7f92d00d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.400+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d0154f90 0x7f92d01553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.400+0000 7f92cea00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92d00c6800 0x7f92d01e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.400+0000 7f92cea00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92d00c6800 0x7f92d01e5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:36172/0 (socket says 172.21.15.123:36172) 2024-04-25T12:34:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.400+0000 7f92cea00640 1 -- 172.21.15.123:0/525398914 learned_addr learned my addr 172.21.15.123:0/525398914 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.400+0000 7f92c6000640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92d01e4e50 0x7f92d00d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.401+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92d00c83b0 con 0x7f92d0154f90 2024-04-25T12:34:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.401+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92d00c8720 con 0x7f92d01e4e50 2024-04-25T12:34:52.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.401+0000 7f92cf400640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d0154f90 0x7f92d01553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.401+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f92d00cd630 con 0x7f92d00c6800 2024-04-25T12:34:52.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.402+0000 7f92c6000640 1 -- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92d00c6800 msgr2=0x7f92d01e5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.402+0000 7f92c6000640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92d00c6800 0x7f92d01e5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.402+0000 7f92c6000640 1 -- 172.21.15.123:0/525398914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d0154f90 msgr2=0x7f92d01553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.402+0000 7f92c6000640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d0154f90 0x7f92d01553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.402+0000 7f92c6000640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d01e6430 con 0x7f92d01e4e50 2024-04-25T12:34:52.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.405+0000 7f92cf400640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d0154f90 0x7f92d01553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:52.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.408+0000 7f92c6000640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92d01e4e50 0x7f92d00d0960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f92d00cf3a0 tx=0x7f92c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.408+0000 7f92c7e00640 1 -- 172.21.15.123:0/525398914 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0040030 con 0x7f92d01e4e50 2024-04-25T12:34:52.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.408+0000 7f92c7e00640 1 -- 172.21.15.123:0/525398914 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c0038040 con 0x7f92d01e4e50 2024-04-25T12:34:52.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.408+0000 7f92c7e00640 1 -- 172.21.15.123:0/525398914 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0037070 con 0x7f92d01e4e50 2024-04-25T12:34:52.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.409+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d01e5730 con 0x7f92d01e4e50 2024-04-25T12:34:52.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.409+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d015d110 con 0x7f92d01e4e50 2024-04-25T12:34:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.429+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4b64005850 con 0x7f4b8c0c2720 2024-04-25T12:34:52.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.429+0000 7f4b88c00640 1 -- 172.21.15.123:0/865388622 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b7807fe00 con 0x7f4b8c0c2720 2024-04-25T12:34:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.426+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d01fa610 con 0x7f92d01e4e50 2024-04-25T12:34:52.430 INFO:teuthology.orchestra.run.smithi123.stdout:60129542153 2024-04-25T12:34:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.428+0000 7f92c7e00640 1 -- 172.21.15.123:0/525398914 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f92c0043070 con 0x7f92d01e4e50 2024-04-25T12:34:52.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.429+0000 7f92c7e00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f92ac0745b0 0x7f92ac076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:52.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.429+0000 7f92c7e00640 1 -- 172.21.15.123:0/525398914 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f92c00b54d0 con 0x7f92d01e4e50 2024-04-25T12:34:52.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.430+0000 7f92cea00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f92ac0745b0 0x7f92ac076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.433+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b6c0745b0 msgr2=0x7f4b6c076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.433+0000 7f4b92600640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b6c0745b0 0x7f4b6c076ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006fd0 tx=0x7f4b7c013040 comp rx=0 tx=0).stop 2024-04-25T12:34:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.433+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c0c2720 msgr2=0x7f4b8c203290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.433+0000 7f4b92600640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c203290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0c4450 tx=0x7f4b780046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.434+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 shutdown_connections 2024-04-25T12:34:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.434+0000 7f4b92600640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b6c0745b0 0x7f4b6c076ac0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.434+0000 7f4b92600640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b8c0cceb0 0x7f4b8c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.434+0000 7f4b92600640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b8c0ccaa0 0x7f4b8c1ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.434+0000 7f4b92600640 1 --2- 172.21.15.123:0/865388622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c0c2720 0x7f4b8c203290 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.435+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 >> 172.21.15.123:0/865388622 conn(0x7f4b8c070b90 msgr2=0x7f4b8c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.435+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 shutdown_connections 2024-04-25T12:34:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.435+0000 7f4b92600640 1 -- 172.21.15.123:0/865388622 wait complete. 2024-04-25T12:34:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.437+0000 7f92cea00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f92ac0745b0 0x7f92ac076ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000f60 tx=0x7f92b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:52.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.447+0000 7f92c7e00640 1 -- 172.21.15.123:0/525398914 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92c007f990 con 0x7f92d01e4e50 2024-04-25T12:34:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.462+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f093c005570 con 0x7f093c046fe0 2024-04-25T12:34:52.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.462+0000 7f0941600640 1 -- 172.21.15.123:0/1542904379 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f093005f950 con 0x7f093c046fe0 2024-04-25T12:34:52.463 INFO:teuthology.orchestra.run.smithi123.stdout:60129542154 2024-04-25T12:34:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.466+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09240746b0 msgr2=0x7f0924076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.466+0000 7f094ae00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09240746b0 0x7f0924076bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f093800ba70 tx=0x7f0938008040 comp rx=0 tx=0).stop 2024-04-25T12:34:52.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.466+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c046fe0 msgr2=0x7f093c0d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.466+0000 7f094ae00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c046fe0 0x7f093c0d2a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f093000b410 tx=0x7f093000b8d0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.467+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 shutdown_connections 2024-04-25T12:34:52.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.467+0000 7f094ae00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09240746b0 0x7f0924076bc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.467+0000 7f094ae00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c046fe0 0x7f093c0d2a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.467+0000 7f094ae00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f093c045b50 0x7f093c04a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.467+0000 7f094ae00640 1 --2- 172.21.15.123:0/1542904379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093c045350 0x7f093c17ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.467+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 >> 172.21.15.123:0/1542904379 conn(0x7f093c0503b0 msgr2=0x7f093c006920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.468+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 shutdown_connections 2024-04-25T12:34:52.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.468+0000 7f094ae00640 1 -- 172.21.15.123:0/1542904379 wait complete. 2024-04-25T12:34:52.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.1 2024-04-25T12:34:52.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.494+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f430c0e1460 con 0x7f430c0375e0 2024-04-25T12:34:52.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.495+0000 7f4307400640 1 -- 172.21.15.123:0/3768857735 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f430007fe10 con 0x7f430c0375e0 2024-04-25T12:34:52.498 INFO:teuthology.orchestra.run.smithi123.stdout:60129542154 2024-04-25T12:34:52.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f42f40748b0 msgr2=0x7f42f4076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f42f40748b0 0x7f42f4076dc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43080015e0 tx=0x7f4308009040 comp rx=0 tx=0).stop 2024-04-25T12:34:52.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f430c0375e0 msgr2=0x7f430c0d8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f430c0375e0 0x7f430c0d8d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4300009870 tx=0x7f43000046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 shutdown_connections 2024-04-25T12:34:52.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f42f40748b0 0x7f42f4076dc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f430c04a4f0 0x7f430c0e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f430c0375e0 0x7f430c0d8d60 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4300009870 tx=0x7f43000046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 --2- 172.21.15.123:0/3768857735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f430c00f200 0x7f430c0e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.500+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f87b0005580 con 0x7f87d806c570 2024-04-25T12:34:52.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 >> 172.21.15.123:0/3768857735 conn(0x7f430c00e550 msgr2=0x7f430c04ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.499+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 shutdown_connections 2024-04-25T12:34:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.500+0000 7f4318a00640 1 -- 172.21.15.123:0/3768857735 wait complete. 2024-04-25T12:34:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.501+0000 7f87cea00640 1 -- 172.21.15.123:0/1097528536 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f87c80630a0 con 0x7f87d806c570 2024-04-25T12:34:52.502 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T12:34:52.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.7 2024-04-25T12:34:52.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f87b40745b0 msgr2=0x7f87b4076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f87b40745b0 0x7f87b4076ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87d0034000 tx=0x7f87d002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d806c570 msgr2=0x7f87d81f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d806c570 0x7f87d81f4ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b440 tx=0x7f87c800b900 comp rx=0 tx=0).stop 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 shutdown_connections 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f87b40745b0 0x7f87b4076ac0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87d80cc720 0x7f87d81f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d80c4380 0x7f87d81f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 --2- 172.21.15.123:0/1097528536 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87d806c570 0x7f87d81f4ae0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b440 tx=0x7f87c800b900 comp rx=0 tx=0).stop 2024-04-25T12:34:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.506+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 >> 172.21.15.123:0/1097528536 conn(0x7f87d8070b90 msgr2=0x7f87d80c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.507+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 shutdown_connections 2024-04-25T12:34:52.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.507+0000 7f87d7e00640 1 -- 172.21.15.123:0/1097528536 wait complete. 2024-04-25T12:34:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.510+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9024005570 con 0x7f9024007970 2024-04-25T12:34:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.511+0000 7f9022a00640 1 -- 172.21.15.123:0/1089318073 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f901c060700 con 0x7f9024007970 2024-04-25T12:34:52.511 INFO:teuthology.orchestra.run.smithi123.stdout:60129542154 2024-04-25T12:34:52.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.515+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f900c074530 msgr2=0x7f900c076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.515+0000 7f9017e00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f900c074530 0x7f900c076a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9018006fd0 tx=0x7f9018008040 comp rx=0 tx=0).stop 2024-04-25T12:34:52.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.515+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 msgr2=0x7f902404c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.515+0000 7f9017e00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f902404c500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f901c00d550 tx=0x7f901c00da10 comp rx=0 tx=0).stop 2024-04-25T12:34:52.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.516+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 shutdown_connections 2024-04-25T12:34:52.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.516+0000 7f9017e00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f900c074530 0x7f900c076a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.516+0000 7f9017e00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90240d9830 0x7f9024179f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.516+0000 7f9017e00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9024037070 0x7f90240cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.517+0000 7f9017e00640 1 --2- 172.21.15.123:0/1089318073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9024007970 0x7f902404c500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.517+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 >> 172.21.15.123:0/1089318073 conn(0x7f9024054840 msgr2=0x7f90240dd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.517+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 shutdown_connections 2024-04-25T12:34:52.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.517+0000 7f9017e00640 1 -- 172.21.15.123:0/1089318073 wait complete. 2024-04-25T12:34:52.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.0 2024-04-25T12:34:52.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.541+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ffb64005580 con 0x7ffb8c15ef90 2024-04-25T12:34:52.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.542+0000 7ffb82a00640 1 -- 172.21.15.123:0/232741584 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb8405e890 con 0x7ffb8c15ef90 2024-04-25T12:34:52.542 INFO:teuthology.orchestra.run.smithi123.stdout:60129542154 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb6c0744e0 msgr2=0x7ffb6c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb6c0744e0 0x7ffb6c0769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c034000 tx=0x7ffb7c02e000 comp rx=0 tx=0).stop 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c15ef90 msgr2=0x7ffb8c14a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c15ef90 0x7ffb8c14a5b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c760 tx=0x7ffb8400cc20 comp rx=0 tx=0).stop 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 shutdown_connections 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ffb6c0744e0 0x7ffb6c0769f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb8c15ef90 0x7ffb8c14a5b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c1607f0 0x7ffb8c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 --2- 172.21.15.123:0/232741584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffb8c06c570 0x7ffb8c1faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.544+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 >> 172.21.15.123:0/232741584 conn(0x7ffb8c0c9f60 msgr2=0x7ffb8c0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.545+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 shutdown_connections 2024-04-25T12:34:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.545+0000 7ffb8be00640 1 -- 172.21.15.123:0/232741584 wait complete. 2024-04-25T12:34:52.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.5 2024-04-25T12:34:52.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.4 2024-04-25T12:34:52.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.6 2024-04-25T12:34:52.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.622+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6a08159f60 con 0x7f6a08156940 2024-04-25T12:34:52.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.622+0000 7f69fe000640 1 -- 172.21.15.123:0/3045361034 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a000632c0 con 0x7f6a08156940 2024-04-25T12:34:52.623 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T12:34:52.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.624+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f69e80744e0 msgr2=0x7f69e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.624+0000 7f6a0d000640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f69e80744e0 0x7f69e80769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005880 tx=0x7f69f8009040 comp rx=0 tx=0).stop 2024-04-25T12:34:52.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.624+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a08156940 msgr2=0x7f6a081e5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.624+0000 7f6a0d000640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a08156940 0x7f6a081e5a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000bf70 tx=0x7f6a0000c4d0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 shutdown_connections 2024-04-25T12:34:52.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f69e80744e0 0x7f69e80769f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a08156940 0x7f6a081e5a40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a080ca820 0x7f6a081e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 --2- 172.21.15.123:0/3045361034 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a080c2870 0x7f6a081e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 >> 172.21.15.123:0/3045361034 conn(0x7f6a0806c750 msgr2=0x7f6a08152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 shutdown_connections 2024-04-25T12:34:52.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.625+0000 7f6a0d000640 1 -- 172.21.15.123:0/3045361034 wait complete. 2024-04-25T12:34:52.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.3 2024-04-25T12:34:52.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.740+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f92d0076da0 con 0x7f92d01e4e50 2024-04-25T12:34:52.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.740+0000 7f92c7e00640 1 -- 172.21.15.123:0/525398914 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92c007f290 con 0x7f92d01e4e50 2024-04-25T12:34:52.741 INFO:teuthology.orchestra.run.smithi123.stdout:60129542154 2024-04-25T12:34:52.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f92ac0745b0 msgr2=0x7f92ac076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f92ac0745b0 0x7f92ac076ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000f60 tx=0x7f92b8009040 comp rx=0 tx=0).stop 2024-04-25T12:34:52.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92d01e4e50 msgr2=0x7f92d00d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92d01e4e50 0x7f92d00d0960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f92d00cf3a0 tx=0x7f92c00046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 shutdown_connections 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f92ac0745b0 0x7f92ac076ac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d0154f90 0x7f92d01553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92d01e4e50 0x7f92d00d0960 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 --2- 172.21.15.123:0/525398914 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f92d00c6800 0x7f92d01e5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 >> 172.21.15.123:0/525398914 conn(0x7f92d006dcd0 msgr2=0x7f92d01e7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 shutdown_connections 2024-04-25T12:34:52.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:52.743+0000 7f92cfe00640 1 -- 172.21.15.123:0/525398914 wait complete. 2024-04-25T12:34:52.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.2 2024-04-25T12:34:53.487 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T12:34:53.507 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T12:34:53.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T12:34:53.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T12:34:53.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T12:34:53.551 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-25T12:34:53.551+0000 7f1a9b000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T12:34:53.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T12:34:53.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T12:34:53.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T12:34:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.867+0000 7febdca00640 1 Processor -- start 2024-04-25T12:34:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.868+0000 7febdca00640 1 -- start start 2024-04-25T12:34:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.873+0000 7febdca00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7febd813e0b0 0x7febd806c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.874+0000 7febdca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd80ccf20 0x7febd80c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.875+0000 7febdca00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd8158380 con 0x7febd806cab0 2024-04-25T12:34:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.875+0000 7febdca00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7febd80c6810 con 0x7febd813e0b0 2024-04-25T12:34:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.875+0000 7febdca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7febd8158620 con 0x7febd80ccf20 2024-04-25T12:34:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.875+0000 7febd7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd80ccf20 0x7febd80c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.875+0000 7febd7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd80ccf20 0x7febd80c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38152/0 (socket says 172.21.15.123:38152) 2024-04-25T12:34:53.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.875+0000 7febd7e00640 1 -- 172.21.15.123:0/3972510208 learned_addr learned my addr 172.21.15.123:0/3972510208 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.875+0000 7febd7400640 1 --2- 172.21.15.123:0/3972510208 >> v2:172.21.15.123:3300/0 conn(0x7febd813e0b0 0x7febd806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.876+0000 7febd7e00640 1 -- 172.21.15.123:0/3972510208 >> v2:172.21.15.123:3300/0 conn(0x7febd813e0b0 msgr2=0x7febd806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:53.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.876+0000 7febd7e00640 1 --2- 172.21.15.123:0/3972510208 >> v2:172.21.15.123:3300/0 conn(0x7febd813e0b0 0x7febd806c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.876+0000 7febd7e00640 1 -- 172.21.15.123:0/3972510208 >> v1:172.21.15.153:6789/0 conn(0x7febd806cab0 legacy=0x7febd80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.876+0000 7febd7e00640 1 -- 172.21.15.123:0/3972510208 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd8158900 con 0x7febd80ccf20 2024-04-25T12:34:53.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.876+0000 7febd7400640 1 --2- 172.21.15.123:0/3972510208 >> v2:172.21.15.123:3300/0 conn(0x7febd813e0b0 0x7febd806c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:53.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.881+0000 7febd7e00640 1 --2- 172.21.15.123:0/3972510208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd80ccf20 0x7febd80c6cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7febc40078a0 tx=0x7febc402ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcaf07de1ba5c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.886+0000 7febd6000640 1 -- 172.21.15.123:0/3972510208 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4004230 con 0x7febd80ccf20 2024-04-25T12:34:53.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.886+0000 7febd6000640 1 -- 172.21.15.123:0/3972510208 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc40043d0 con 0x7febd80ccf20 2024-04-25T12:34:53.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.887+0000 7febdca00640 1 -- 172.21.15.123:0/3972510208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd80ccf20 msgr2=0x7febd80c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.887+0000 7febdca00640 1 --2- 172.21.15.123:0/3972510208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd80ccf20 0x7febd80c6cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7febc40078a0 tx=0x7febc402ce30 comp rx=0 tx=0).stop 2024-04-25T12:34:53.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.891+0000 7febdca00640 1 -- 172.21.15.123:0/3972510208 shutdown_connections 2024-04-25T12:34:53.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.891+0000 7febdca00640 1 --2- 172.21.15.123:0/3972510208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd80ccf20 0x7febd80c6cc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.891+0000 7febdca00640 1 --2- 172.21.15.123:0/3972510208 >> v2:172.21.15.123:3300/0 conn(0x7febd813e0b0 0x7febd806c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.891+0000 7febdca00640 1 -- 172.21.15.123:0/3972510208 >> 172.21.15.123:0/3972510208 conn(0x7febd80c9f60 msgr2=0x7febd80ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:53.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.892+0000 7febdca00640 1 -- 172.21.15.123:0/3972510208 shutdown_connections 2024-04-25T12:34:53.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.893+0000 7febdca00640 1 -- 172.21.15.123:0/3972510208 wait complete. 2024-04-25T12:34:53.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.900+0000 7febdca00640 1 Processor -- start 2024-04-25T12:34:53.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.901+0000 7febdca00640 1 -- start start 2024-04-25T12:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.904+0000 7febdca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd806cab0 0x7febd81df360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.904+0000 7febdca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febd80ccf20 0x7febd81e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.905+0000 7febd7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd806cab0 0x7febd81df360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.905+0000 7febd7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd806cab0 0x7febd81df360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39458/0 (socket says 172.21.15.123:39458) 2024-04-25T12:34:53.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.906+0000 7febd7400640 1 -- 172.21.15.123:0/702742385 learned_addr learned my addr 172.21.15.123:0/702742385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.906+0000 7febdca00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd813e0b0 0x7febd8151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.906+0000 7febd6a00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febd80ccf20 0x7febd81e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.906+0000 7febdca00640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febd81591a0 con 0x7febd806cab0 2024-04-25T12:34:53.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.906+0000 7febdca00640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febd8158ec0 con 0x7febd80ccf20 2024-04-25T12:34:53.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.907+0000 7febd7e00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd813e0b0 0x7febd8151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.907+0000 7febdca00640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7febd813e4c0 con 0x7febd813e0b0 2024-04-25T12:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.908+0000 7febd7400640 1 -- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd813e0b0 msgr2=0x7febd8151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.908+0000 7febd7400640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd813e0b0 0x7febd8151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.908+0000 7febd7400640 1 -- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febd80ccf20 msgr2=0x7febd81e6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.908+0000 7febd7400640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febd80ccf20 0x7febd81e6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.908+0000 7febd7400640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd8156580 con 0x7febd806cab0 2024-04-25T12:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.909+0000 7febd7400640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd806cab0 0x7febd81df360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7febd000ce10 tx=0x7febd000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.910+0000 7febd6a00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febd80ccf20 0x7febd81e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.910+0000 7febcf400640 1 -- 172.21.15.123:0/702742385 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000f890 con 0x7febd806cab0 2024-04-25T12:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.910+0000 7febcf400640 1 -- 172.21.15.123:0/702742385 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd000fa30 con 0x7febd806cab0 2024-04-25T12:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.910+0000 7febcf400640 1 -- 172.21.15.123:0/702742385 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000fe20 con 0x7febd806cab0 2024-04-25T12:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.910+0000 7febdca00640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd81df8a0 con 0x7febd806cab0 2024-04-25T12:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.910+0000 7febdca00640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd8155d30 con 0x7febd806cab0 2024-04-25T12:34:53.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.913+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb0005290 con 0x7febd806cab0 2024-04-25T12:34:53.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.917+0000 7febcf400640 1 -- 172.21.15.123:0/702742385 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febd0018400 con 0x7febd806cab0 2024-04-25T12:34:53.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.917+0000 7febd7e00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd813e0b0 0x7febd8151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.926+0000 7febcf400640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febb80745b0 0x7febb8076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.926+0000 7febd6a00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febb80745b0 0x7febb8076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.927+0000 7febcf400640 1 -- 172.21.15.123:0/702742385 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7febd0096200 con 0x7febd806cab0 2024-04-25T12:34:53.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.929+0000 7febd6a00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febb80745b0 0x7febb8076ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febc8000f60 tx=0x7febc8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.933+0000 7febcf400640 1 -- 172.21.15.123:0/702742385 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febd0014030 con 0x7febd806cab0 2024-04-25T12:34:53.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.934+0000 7f39d5c00640 1 Processor -- start 2024-04-25T12:34:53.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.935+0000 7f1686000640 1 Processor -- start 2024-04-25T12:34:53.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.935+0000 7f39d5c00640 1 -- start start 2024-04-25T12:34:53.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.936+0000 7f1686000640 1 -- start start 2024-04-25T12:34:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.938+0000 7f39d5c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f39d0163ec0 0x7f39d015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.938+0000 7f1686000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16780495e0 0x7f16780073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.939+0000 7f1686000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1678050af0 con 0x7f1678050050 2024-04-25T12:34:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.939+0000 7f1686000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1678050db0 con 0x7f1678007910 2024-04-25T12:34:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.939+0000 7f1686000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1678050700 con 0x7f16780495e0 2024-04-25T12:34:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.939+0000 7f39d5c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8930 0x7f39d006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f39d5c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d0164390 con 0x7f39d00c8520 2024-04-25T12:34:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f39d5c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f39d0164e70 con 0x7f39d0163ec0 2024-04-25T12:34:53.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.939+0000 7f1684c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16780495e0 0x7f16780073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.939+0000 7f1684c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16780495e0 0x7f16780073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39464/0 (socket says 172.21.15.123:39464) 2024-04-25T12:34:53.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.939+0000 7f1684c00640 1 -- 172.21.15.123:0/48387395 learned_addr learned my addr 172.21.15.123:0/48387395 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f1684c00640 1 -- 172.21.15.123:0/48387395 >> v1:172.21.15.153:6789/0 conn(0x7f1678007910 legacy=0x7f167804f940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T12:34:53.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f1684c00640 1 -- 172.21.15.123:0/48387395 >> v1:172.21.15.123:6789/0 conn(0x7f1678050050 legacy=0x7f16780e9da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:34:53.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f39d5c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f39d0160b70 con 0x7f39d00c8930 2024-04-25T12:34:53.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f39cfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f39d0163ec0 0x7f39d015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f39cfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f39d0163ec0 0x7f39d015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39468/0 (socket says 172.21.15.123:39468) 2024-04-25T12:34:53.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.940+0000 7f39cfe00640 1 -- 172.21.15.123:0/1130474371 learned_addr learned my addr 172.21.15.123:0/1130474371 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.941+0000 7f39cfe00640 1 -- 172.21.15.123:0/1130474371 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8930 msgr2=0x7f39d006c570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T12:34:53.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.941+0000 7f39cfe00640 1 --2- 172.21.15.123:0/1130474371 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8930 0x7f39d006c570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.941+0000 7f39cfe00640 1 -- 172.21.15.123:0/1130474371 >> v1:172.21.15.153:6789/0 conn(0x7f39d00c8520 legacy=0x7f39d0161000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T12:34:53.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.941+0000 7f39cfe00640 1 -- 172.21.15.123:0/1130474371 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d0164630 con 0x7f39d0163ec0 2024-04-25T12:34:53.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.941+0000 7f1684c00640 1 -- 172.21.15.123:0/48387395 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16780e35c0 con 0x7f16780495e0 2024-04-25T12:34:53.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.942+0000 7fb01d200640 1 Processor -- start 2024-04-25T12:34:53.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.942+0000 7f39cfe00640 1 --2- 172.21.15.123:0/1130474371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d0163ec0 0x7f39d015ac10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f39d00ce520 tx=0x7f39bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c57cc80ff1c75d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.943+0000 7f1684c00640 1 --2- 172.21.15.123:0/48387395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f16780073d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f166c009870 tx=0x7f166c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53885adaec229e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.943+0000 7f39cea00640 1 -- 172.21.15.123:0/1130474371 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc037070 con 0x7f39d0163ec0 2024-04-25T12:34:53.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.943+0000 7f39cea00640 1 -- 172.21.15.123:0/1130474371 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39bc032440 con 0x7f39d0163ec0 2024-04-25T12:34:53.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.943+0000 7fb01d200640 1 -- start start 2024-04-25T12:34:53.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.943+0000 7f39cea00640 1 -- 172.21.15.123:0/1130474371 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc032810 con 0x7f39d0163ec0 2024-04-25T12:34:53.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.945+0000 7f167f400640 1 -- 172.21.15.123:0/48387395 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c037070 con 0x7f16780495e0 2024-04-25T12:34:53.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.945+0000 7f167f400640 1 -- 172.21.15.123:0/48387395 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c032640 con 0x7f16780495e0 2024-04-25T12:34:53.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.946+0000 7f167f400640 1 -- 172.21.15.123:0/48387395 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c032a10 con 0x7f16780495e0 2024-04-25T12:34:53.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.946+0000 7f39d5c00640 1 -- 172.21.15.123:0/1130474371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d0163ec0 msgr2=0x7f39d015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.946+0000 7f39d5c00640 1 --2- 172.21.15.123:0/1130474371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d0163ec0 0x7f39d015ac10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f39d00ce520 tx=0x7f39bc02a210 comp rx=0 tx=0).stop 2024-04-25T12:34:53.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.946+0000 7f39d5c00640 1 -- 172.21.15.123:0/1130474371 shutdown_connections 2024-04-25T12:34:53.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.946+0000 7f39d5c00640 1 --2- 172.21.15.123:0/1130474371 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8930 0x7f39d006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.946+0000 7f39d5c00640 1 --2- 172.21.15.123:0/1130474371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d0163ec0 0x7f39d015ac10 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f39d00ce520 tx=0x7f39bc02a210 comp rx=0 tx=0).stop 2024-04-25T12:34:53.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.946+0000 7f39d5c00640 1 -- 172.21.15.123:0/1130474371 >> 172.21.15.123:0/1130474371 conn(0x7f39d00cd600 msgr2=0x7f39d00cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:53.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.947+0000 7fb01d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb01004c910 0x7fb010006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.948+0000 7fb01d200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb0100decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.948+0000 7fb01d200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0100df6b0 con 0x7fb010047180 2024-04-25T12:34:53.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.948+0000 7fb01d200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb010046a10 con 0x7fb01004c910 2024-04-25T12:34:53.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.948+0000 7f39d5c00640 1 -- 172.21.15.123:0/1130474371 shutdown_connections 2024-04-25T12:34:53.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.948+0000 7fb01d200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb010033540 con 0x7fb0100de8e0 2024-04-25T12:34:53.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.948+0000 7fb016a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb010047180 0x7fb0100d52c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45788/0 (socket says 172.21.15.123:45788) 2024-04-25T12:34:53.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.949+0000 7fb017400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb01004c910 0x7fb010006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.949+0000 7fb017400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb01004c910 0x7fb010006060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39472/0 (socket says 172.21.15.123:39472) 2024-04-25T12:34:53.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.949+0000 7fb017400640 1 -- 172.21.15.123:0/1497933964 learned_addr learned my addr 172.21.15.123:0/1497933964 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.950+0000 7fb016000640 1 -- 172.21.15.123:0/1497933964 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4103537665 0 0) 0x7fb0100df6b0 con 0x7fb010047180 2024-04-25T12:34:53.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.948+0000 7f39d5c00640 1 -- 172.21.15.123:0/1130474371 wait complete. 2024-04-25T12:34:53.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.950+0000 7fb017400640 1 -- 172.21.15.123:0/1497933964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 msgr2=0x7fb0100decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:53.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.950+0000 7fb017e00640 1 --2- 172.21.15.123:0/1497933964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb0100decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.951+0000 7fb017400640 1 --2- 172.21.15.123:0/1497933964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb0100decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.951+0000 7fb017400640 1 -- 172.21.15.123:0/1497933964 >> v1:172.21.15.123:6789/0 conn(0x7fb010047180 legacy=0x7fb0100d52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.951+0000 7fb017400640 1 -- 172.21.15.123:0/1497933964 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0100df410 con 0x7fb01004c910 2024-04-25T12:34:53.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.952+0000 7fb017e00640 1 --2- 172.21.15.123:0/1497933964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb0100decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.953+0000 7fb017400640 1 --2- 172.21.15.123:0/1497933964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb010006060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb01806c060 tx=0x7fb0180a9d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654c504e5528d0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.953+0000 7f1686000640 1 -- 172.21.15.123:0/48387395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 msgr2=0x7f16780073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.953+0000 7f1686000640 1 --2- 172.21.15.123:0/48387395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f16780073d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f166c009870 tx=0x7f166c02a300 comp rx=0 tx=0).stop 2024-04-25T12:34:53.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.954+0000 7fb016000640 1 -- 172.21.15.123:0/1497933964 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0180b0070 con 0x7fb01004c910 2024-04-25T12:34:53.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.954+0000 7fb016000640 1 -- 172.21.15.123:0/1497933964 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0180aa910 con 0x7fb01004c910 2024-04-25T12:34:53.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.954+0000 7fb016000640 1 -- 172.21.15.123:0/1497933964 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0180aada0 con 0x7fb01004c910 2024-04-25T12:34:53.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.954+0000 7f39d5c00640 1 Processor -- start 2024-04-25T12:34:53.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.955+0000 7f1686000640 1 -- 172.21.15.123:0/48387395 shutdown_connections 2024-04-25T12:34:53.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.955+0000 7f1686000640 1 --2- 172.21.15.123:0/48387395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f16780073d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.955+0000 7f1686000640 1 -- 172.21.15.123:0/48387395 >> 172.21.15.123:0/48387395 conn(0x7f167800e380 msgr2=0x7f167800e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:53.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.955+0000 7f39d5c00640 1 -- start start 2024-04-25T12:34:53.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 -- 172.21.15.123:0/1497933964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 msgr2=0x7fb010006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 --2- 172.21.15.123:0/1497933964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb010006060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb01806c060 tx=0x7fb0180a9d10 comp rx=0 tx=0).stop 2024-04-25T12:34:53.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 -- 172.21.15.123:0/1497933964 shutdown_connections 2024-04-25T12:34:53.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 --2- 172.21.15.123:0/1497933964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb0100decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 --2- 172.21.15.123:0/1497933964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb010006060 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 -- 172.21.15.123:0/1497933964 >> 172.21.15.123:0/1497933964 conn(0x7fb010050380 msgr2=0x7fb010050790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:53.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 -- 172.21.15.123:0/1497933964 shutdown_connections 2024-04-25T12:34:53.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.956+0000 7fb01d200640 1 -- 172.21.15.123:0/1497933964 wait complete. 2024-04-25T12:34:53.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.955+0000 7f1686000640 1 -- 172.21.15.123:0/48387395 shutdown_connections 2024-04-25T12:34:53.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.955+0000 7f1686000640 1 -- 172.21.15.123:0/48387395 wait complete. 2024-04-25T12:34:53.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.959+0000 7f1686000640 1 Processor -- start 2024-04-25T12:34:53.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.961+0000 7f39d5c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8520 0x7f39d015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.961+0000 7fb01d200640 1 Processor -- start 2024-04-25T12:34:53.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.960+0000 7f1686000640 1 -- start start 2024-04-25T12:34:53.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.961+0000 7fb01d200640 1 -- start start 2024-04-25T12:34:53.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.961+0000 7f39d5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d00c8930 0x7f39d0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.962+0000 7f39cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d00c8930 0x7f39d0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.962+0000 7f39cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d00c8930 0x7f39d0158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39486/0 (socket says 172.21.15.123:39486) 2024-04-25T12:34:53.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.962+0000 7f39d5c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39d0158750 0x7f39d0158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1686000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1678007910 0x7f167817a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1686000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f167816e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1686000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1678050050 0x7f16780536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1686000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16780e38a0 con 0x7f16780495e0 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1686000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f167804f420 con 0x7f1678007910 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1685600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1678050050 0x7f16780536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1685600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1678050050 0x7f16780536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38190/0 (socket says 172.21.15.123:38190) 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1685600640 1 -- 172.21.15.123:0/1790458753 learned_addr learned my addr 172.21.15.123:0/1790458753 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.963+0000 7f1686000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f167800f1c0 con 0x7f1678050050 2024-04-25T12:34:53.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.962+0000 7f39cfe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8520 0x7f39d015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.964+0000 7f39d5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39d00c9230 con 0x7f39d00c8930 2024-04-25T12:34:53.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.964+0000 7f39d5c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0164b90 con 0x7f39d0158750 2024-04-25T12:34:53.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.964+0000 7f39d5c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f39d00c93b0 con 0x7f39d00c8520 2024-04-25T12:34:53.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.964+0000 7fb01d200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010047180 0x7fb010048a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb01d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb01004e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.964+0000 7f39cf400640 1 -- 172.21.15.123:0/872578973 learned_addr learned my addr 172.21.15.123:0/872578973 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb01d200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb01004f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb017400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010047180 0x7fb010048a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb017400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010047180 0x7fb010048a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33988/0 (socket says 172.21.15.123:33988) 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb016a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb01004e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb016a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb01004e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39492/0 (socket says 172.21.15.123:39492) 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb017400640 1 -- 172.21.15.123:0/345795283 learned_addr learned my addr 172.21.15.123:0/345795283 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb01004bf50 con 0x7fb01004c910 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb01004bc70 con 0x7fb010047180 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7f1684c00640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1678007910 0x7f167817a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7f167fe00640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f167816e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.965+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb010046c60 con 0x7fb0100de8e0 2024-04-25T12:34:53.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.966+0000 7fb017e00640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb01004f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.966+0000 7f39d5200640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39d0158750 0x7f39d0158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.966+0000 7f1685600640 1 -- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1678007910 msgr2=0x7f167817a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.966+0000 7f1685600640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1678007910 0x7f167817a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f1685600640 1 -- 172.21.15.123:0/1790458753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 msgr2=0x7f167816e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f1685600640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f167816e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7fb016a00640 1 -- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 msgr2=0x7fb01004f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7fb016a00640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb01004f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7fb016a00640 1 -- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010047180 msgr2=0x7fb010048a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7fb016a00640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010047180 0x7fb010048a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7fb016a00640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01004ed30 con 0x7fb01004c910 2024-04-25T12:34:53.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f1685600640 1 -- 172.21.15.123:0/1790458753 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167816e610 con 0x7f1678050050 2024-04-25T12:34:53.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.968+0000 7f1685600640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1678050050 0x7f16780536b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f168006b4c0 tx=0x7f1680069cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.968+0000 7f167fe00640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f167816e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:53.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.968+0000 7fb016a00640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb01004e7f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb010045770 tx=0x7fb00400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f39cfe00640 1 -- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39d0158750 msgr2=0x7f39d0158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f39cfe00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39d0158750 0x7f39d0158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f39cfe00640 1 -- 172.21.15.123:0/872578973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d00c8930 msgr2=0x7f39d0158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f39cfe00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d00c8930 0x7f39d0158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.967+0000 7f39cfe00640 1 -- 172.21.15.123:0/872578973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d015ed90 con 0x7f39d00c8520 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.968+0000 7fb00b400640 1 -- 172.21.15.123:0/345795283 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040074e0 con 0x7fb01004c910 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.968+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb010175a10 con 0x7fb01004c910 2024-04-25T12:34:53.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.970+0000 7f39cfe00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8520 0x7f39d015e850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39bc031000 tx=0x7f39bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.969+0000 7f167cc00640 1 -- 172.21.15.123:0/1790458753 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168008da40 con 0x7f1678050050 2024-04-25T12:34:53.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.969+0000 7f167cc00640 1 -- 172.21.15.123:0/1790458753 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1680091070 con 0x7f1678050050 2024-04-25T12:34:53.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.969+0000 7f167cc00640 1 -- 172.21.15.123:0/1790458753 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168008de40 con 0x7f1678050050 2024-04-25T12:34:53.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.971+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167816bce0 con 0x7f1678050050 2024-04-25T12:34:53.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.969+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb010175fd0 con 0x7fb01004c910 2024-04-25T12:34:53.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.969+0000 7fb00b400640 1 -- 172.21.15.123:0/345795283 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0040047e0 con 0x7fb01004c910 2024-04-25T12:34:53.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.970+0000 7fb00b400640 1 -- 172.21.15.123:0/345795283 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040074e0 con 0x7fb01004c910 2024-04-25T12:34:53.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.971+0000 7f39c7e00640 1 -- 172.21.15.123:0/872578973 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc041070 con 0x7f39d00c8520 2024-04-25T12:34:53.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.971+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d01ebc80 con 0x7f39d00c8520 2024-04-25T12:34:53.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.971+0000 7f39c7e00640 1 -- 172.21.15.123:0/872578973 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39bc03c870 con 0x7f39d00c8520 2024-04-25T12:34:53.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.971+0000 7f39c7e00640 1 -- 172.21.15.123:0/872578973 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc041070 con 0x7f39d00c8520 2024-04-25T12:34:53.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.973+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d01573d0 con 0x7f39d00c8520 2024-04-25T12:34:53.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.971+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f167816c2a0 con 0x7f1678050050 2024-04-25T12:34:53.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.972+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1678005570 con 0x7f1678050050 2024-04-25T12:34:53.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.974+0000 7fb00b400640 1 -- 172.21.15.123:0/345795283 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb004011070 con 0x7fb01004c910 2024-04-25T12:34:53.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.975+0000 7fb00b400640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faff8074670 0x7faff8076b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.976+0000 7fb00b400640 1 -- 172.21.15.123:0/345795283 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb004094b00 con 0x7fb01004c910 2024-04-25T12:34:53.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.976+0000 7f3643200640 1 Processor -- start 2024-04-25T12:34:53.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.976+0000 7fb017400640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faff8074670 0x7faff8076b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.977+0000 7f3643200640 1 -- start start 2024-04-25T12:34:53.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.978+0000 7fb017400640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faff8074670 0x7faff8076b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0180520f0 tx=0x7fb018070040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.978+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff4005290 con 0x7fb01004c910 2024-04-25T12:34:53.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.978+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d0157640 con 0x7f39d00c8520 2024-04-25T12:34:53.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.979+0000 7f39c7e00640 1 -- 172.21.15.123:0/872578973 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f39bc044070 con 0x7f39d00c8520 2024-04-25T12:34:53.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3643200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f36440caa80 0x7f364413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3643200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f364406c570 0x7f364406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3643200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3644158620 con 0x7f36440cc810 2024-04-25T12:34:53.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3643200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f36440c6810 con 0x7f36440caa80 2024-04-25T12:34:53.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3643200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f3644158900 con 0x7f364406c570 2024-04-25T12:34:53.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3641e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f36440caa80 0x7f364413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3641e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f36440caa80 0x7f364413ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39504/0 (socket says 172.21.15.123:39504) 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3641e00640 1 -- 172.21.15.123:0/2273519671 learned_addr learned my addr 172.21.15.123:0/2273519671 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.981+0000 7f3641400640 1 --2- 172.21.15.123:0/2273519671 >> v2:172.21.15.153:3300/0 conn(0x7f364406c570 0x7f364406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.982+0000 7f3641400640 1 -- 172.21.15.123:0/2273519671 >> v2:172.21.15.123:3300/0 conn(0x7f36440caa80 msgr2=0x7f364413ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.982+0000 7f3641400640 1 --2- 172.21.15.123:0/2273519671 >> v2:172.21.15.123:3300/0 conn(0x7f36440caa80 0x7f364413ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.982+0000 7f3641400640 1 -- 172.21.15.123:0/2273519671 >> v1:172.21.15.153:6789/0 conn(0x7f36440cc810 legacy=0x7f36440ccc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.982+0000 7f3641400640 1 -- 172.21.15.123:0/2273519671 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3644158be0 con 0x7f364406c570 2024-04-25T12:34:53.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.982+0000 7f167cc00640 1 -- 172.21.15.123:0/1790458753 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1680096470 con 0x7f1678050050 2024-04-25T12:34:53.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.982+0000 7f39c7e00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39b00746e0 0x7f39b0076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.983+0000 7f39c7e00640 1 -- 172.21.15.123:0/872578973 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f39bc0b5490 con 0x7f39d00c8520 2024-04-25T12:34:53.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.983+0000 7f39cf400640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39b00746e0 0x7f39b0076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.984+0000 7f167cc00640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1660074580 0x7f1660076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:53.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.984+0000 7f167cc00640 1 -- 172.21.15.123:0/1790458753 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1680112650 con 0x7f1678050050 2024-04-25T12:34:53.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.984+0000 7f167cc00640 1 -- 172.21.15.123:0/1790458753 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16800a5080 con 0x7f1678050050 2024-04-25T12:34:53.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.985+0000 7f1684c00640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1660074580 0x7f1660076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:53.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.985+0000 7f3641400640 1 --2- 172.21.15.123:0/2273519671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f364406c570 0x7f364406c980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f36440c5920 tx=0x7f363402a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a760a78ad7fbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.985+0000 7f39cf400640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39b00746e0 0x7f39b0076bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f39d015a4c0 tx=0x7f39c00090e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.985+0000 7f39c7e00640 1 -- 172.21.15.123:0/872578973 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39d0157640 con 0x7f39d00c8520 2024-04-25T12:34:53.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.985+0000 7f3640a00640 1 -- 172.21.15.123:0/2273519671 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634037070 con 0x7f364406c570 2024-04-25T12:34:53.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.985+0000 7f3640a00640 1 -- 172.21.15.123:0/2273519671 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3634032440 con 0x7f364406c570 2024-04-25T12:34:53.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.985+0000 7f3640a00640 1 -- 172.21.15.123:0/2273519671 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634032810 con 0x7f364406c570 2024-04-25T12:34:53.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.987+0000 7f3643200640 1 -- 172.21.15.123:0/2273519671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f364406c570 msgr2=0x7f364406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:53.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.987+0000 7f1684c00640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1660074580 0x7f1660076a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f166c034000 tx=0x7f166c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:53.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.987+0000 7f3643200640 1 --2- 172.21.15.123:0/2273519671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f364406c570 0x7f364406c980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f36440c5920 tx=0x7f363402a1f0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.990+0000 7f3643200640 1 -- 172.21.15.123:0/2273519671 shutdown_connections 2024-04-25T12:34:53.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.990+0000 7f3643200640 1 --2- 172.21.15.123:0/2273519671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f364406c570 0x7f364406c980 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.990+0000 7f3643200640 1 --2- 172.21.15.123:0/2273519671 >> v2:172.21.15.123:3300/0 conn(0x7f36440caa80 0x7f364413ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:53.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.990+0000 7f3643200640 1 -- 172.21.15.123:0/2273519671 >> 172.21.15.123:0/2273519671 conn(0x7f36440c9f60 msgr2=0x7f36440ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:53.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.990+0000 7f3643200640 1 -- 172.21.15.123:0/2273519671 shutdown_connections 2024-04-25T12:34:53.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.990+0000 7f3643200640 1 -- 172.21.15.123:0/2273519671 wait complete. 2024-04-25T12:34:53.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.993+0000 7f3643200640 1 Processor -- start 2024-04-25T12:34:53.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.995+0000 7fb00b400640 1 -- 172.21.15.123:0/345795283 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb00405e980 con 0x7fb01004c910 2024-04-25T12:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.995+0000 7f3643200640 1 -- start start 2024-04-25T12:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.997+0000 7f3643200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364406c570 0x7f364415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.997+0000 7f3643200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36440caa80 0x7f3644155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.997+0000 7f3643200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36440cc810 0x7f3644155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.997+0000 7f3643200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36440c8010 con 0x7f364406c570 2024-04-25T12:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.997+0000 7f3643200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36441591a0 con 0x7f36440caa80 2024-04-25T12:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:53.997+0000 7f3643200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3644155c80 con 0x7f36440cc810 2024-04-25T12:34:54.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.002+0000 7f3641400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36440caa80 0x7f3644155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.002+0000 7f3642800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36440cc810 0x7f3644155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.003+0000 7f3642800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36440cc810 0x7f3644155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38204/0 (socket says 172.21.15.123:38204) 2024-04-25T12:34:54.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.003+0000 7f3642800640 1 -- 172.21.15.123:0/988817973 learned_addr learned my addr 172.21.15.123:0/988817973 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:54.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.002+0000 7f3641e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364406c570 0x7f364415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.004+0000 7f3642800640 1 -- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36440caa80 msgr2=0x7f3644155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.004+0000 7f3642800640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36440caa80 0x7f3644155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.004+0000 7f3642800640 1 -- 172.21.15.123:0/988817973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364406c570 msgr2=0x7f364415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.004+0000 7f3642800640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364406c570 0x7f364415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.004+0000 7f3642800640 1 -- 172.21.15.123:0/988817973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3644159540 con 0x7f36440cc810 2024-04-25T12:34:54.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.005+0000 7f3641e00640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364406c570 0x7f364415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:34:54.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.008+0000 7f3641400640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36440caa80 0x7f3644155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:54.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.008+0000 7f3642800640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36440cc810 0x7f3644155740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f363c00ba10 tx=0x7f363c00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.009+0000 7f363a000640 1 -- 172.21.15.123:0/988817973 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c019070 con 0x7f36440cc810 2024-04-25T12:34:54.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.009+0000 7f363a000640 1 -- 172.21.15.123:0/988817973 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363c00cba0 con 0x7f36440cc810 2024-04-25T12:34:54.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.009+0000 7f363a000640 1 -- 172.21.15.123:0/988817973 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c014440 con 0x7f36440cc810 2024-04-25T12:34:54.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.011+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3644152d80 con 0x7f36440cc810 2024-04-25T12:34:54.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.011+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f364415b440 con 0x7f36440cc810 2024-04-25T12:34:54.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.017+0000 7f363a000640 1 -- 172.21.15.123:0/988817973 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f363c01c070 con 0x7f36440cc810 2024-04-25T12:34:54.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.019+0000 7f363a000640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3620074580 0x7f3620076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.020+0000 7f363a000640 1 -- 172.21.15.123:0/988817973 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f363c095310 con 0x7f36440cc810 2024-04-25T12:34:54.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.020+0000 7f3641e00640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3620074580 0x7f3620076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.021+0000 7f3641e00640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3620074580 0x7f3620076a90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f362c003940 tx=0x7f362c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.022+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644076da0 con 0x7f36440cc810 2024-04-25T12:34:54.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.024+0000 7f55e8c00640 1 Processor -- start 2024-04-25T12:34:54.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.025+0000 7f55e8c00640 1 -- start start 2024-04-25T12:34:54.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f363a000640 1 -- 172.21.15.123:0/988817973 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f363c05f740 con 0x7f36440cc810 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e8c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f55dc045380 0x7f55dc0df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e8c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc045b80 0x7f55dc046ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e8c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f55dc047010 0x7f55dc0356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e8c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f55dc006720 con 0x7f55dc047010 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e8c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f55dc033510 con 0x7f55dc045380 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e8c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f55dc0dcb90 con 0x7f55dc045b80 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e1e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f55dc045380 0x7f55dc0df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e1e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f55dc045380 0x7f55dc0df660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34016/0 (socket says 172.21.15.123:34016) 2024-04-25T12:34:54.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e1e00640 1 -- 172.21.15.123:0/4153449533 learned_addr learned my addr 172.21.15.123:0/4153449533 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:54.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.029+0000 7f55e1e00640 1 -- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc045b80 msgr2=0x7f55dc046ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:54.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e1400640 1 --2- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc045b80 0x7f55dc046ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.028+0000 7f55e2800640 1 --2- 172.21.15.123:0/4153449533 >> v2:172.21.15.123:3300/0 conn(0x7f55dc047010 0x7f55dc0356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.032+0000 7f55e1e00640 1 --2- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc045b80 0x7f55dc046ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.032+0000 7f55e1e00640 1 -- 172.21.15.123:0/4153449533 >> v2:172.21.15.123:3300/0 conn(0x7f55dc047010 msgr2=0x7f55dc0356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:54.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.032+0000 7f55e1e00640 1 --2- 172.21.15.123:0/4153449533 >> v2:172.21.15.123:3300/0 conn(0x7f55dc047010 0x7f55dc0356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.032+0000 7f55e1e00640 1 -- 172.21.15.123:0/4153449533 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc0e29d0 con 0x7f55dc045380 2024-04-25T12:34:54.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.032+0000 7f55e1400640 1 --2- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc045b80 0x7f55dc046ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:54.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.032+0000 7f55e2800640 1 --2- 172.21.15.123:0/4153449533 >> v2:172.21.15.123:3300/0 conn(0x7f55dc047010 0x7f55dc0356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:34:54.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.037+0000 7f55e1e00640 1 --2- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0df660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55e4065b40 tx=0x7f55e40a7910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87b937a30336d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.040+0000 7f55e0a00640 1 -- 172.21.15.123:0/4153449533 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e40b1020 con 0x7f55dc045380 2024-04-25T12:34:54.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.040+0000 7f55e0a00640 1 -- 172.21.15.123:0/4153449533 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e40ad6c0 con 0x7f55dc045380 2024-04-25T12:34:54.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.040+0000 7f55e0a00640 1 -- 172.21.15.123:0/4153449533 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e40adb30 con 0x7f55dc045380 2024-04-25T12:34:54.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.041+0000 7f55e8c00640 1 -- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 msgr2=0x7f55dc0df660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.041+0000 7f55e8c00640 1 --2- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0df660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55e4065b40 tx=0x7f55e40a7910 comp rx=0 tx=0).stop 2024-04-25T12:34:54.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.045+0000 7f55e8c00640 1 -- 172.21.15.123:0/4153449533 shutdown_connections 2024-04-25T12:34:54.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.045+0000 7f55e8c00640 1 --2- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc045b80 0x7f55dc046ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.045+0000 7f55e8c00640 1 --2- 172.21.15.123:0/4153449533 >> v2:172.21.15.123:3300/0 conn(0x7f55dc047010 0x7f55dc0356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.045+0000 7f55e8c00640 1 --2- 172.21.15.123:0/4153449533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0df660 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.045+0000 7f55e8c00640 1 -- 172.21.15.123:0/4153449533 >> 172.21.15.123:0/4153449533 conn(0x7f55dc0503e0 msgr2=0x7f55dc0507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.045+0000 7f55e8c00640 1 -- 172.21.15.123:0/4153449533 shutdown_connections 2024-04-25T12:34:54.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.045+0000 7f55e8c00640 1 -- 172.21.15.123:0/4153449533 wait complete. 2024-04-25T12:34:54.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.052+0000 7f55e8c00640 1 Processor -- start 2024-04-25T12:34:54.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.052+0000 7f55e8c00640 1 -- start start 2024-04-25T12:34:54.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.055+0000 7f55e8c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0de580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.056+0000 7f55e8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55dc045b80 0x7f55dc0daa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.056+0000 7f55e1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.056+0000 7f55e1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34028/0 (socket says 172.21.15.123:34028) 2024-04-25T12:34:54.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.056+0000 7f55e1e00640 1 -- 172.21.15.123:0/722540902 learned_addr learned my addr 172.21.15.123:0/722540902 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:54.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.057+0000 7f55e8c00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc047010 0x7f55dc0dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.057+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc033760 con 0x7f55dc045b80 2024-04-25T12:34:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.057+0000 7f55e1400640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55dc045b80 0x7f55dc0daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.057+0000 7f55e2800640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc047010 0x7f55dc0dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.057+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc0d6730 con 0x7f55dc045380 2024-04-25T12:34:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.058+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f55dc033a40 con 0x7f55dc047010 2024-04-25T12:34:54.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.059+0000 7f55e2800640 1 -- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 msgr2=0x7f55dc0de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.059+0000 7f55e2800640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.059+0000 7f55e2800640 1 -- 172.21.15.123:0/722540902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55dc045b80 msgr2=0x7f55dc0daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.059+0000 7f55e2800640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55dc045b80 0x7f55dc0daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.060+0000 7f55e2800640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc0d4d00 con 0x7f55dc047010 2024-04-25T12:34:54.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.060+0000 7f55e1400640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55dc045b80 0x7f55dc0daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:54.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.061+0000 7f55e2800640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc047010 0x7f55dc0dafc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55dc04e2b0 tx=0x7f55d400c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.061+0000 7f55da000640 1 -- 172.21.15.123:0/722540902 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4015070 con 0x7f55dc047010 2024-04-25T12:34:54.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.061+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc0deac0 con 0x7f55dc047010 2024-04-25T12:34:54.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.062+0000 7f55da000640 1 -- 172.21.15.123:0/722540902 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d4004220 con 0x7f55dc047010 2024-04-25T12:34:54.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.062+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc0db990 con 0x7f55dc047010 2024-04-25T12:34:54.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.062+0000 7f55da000640 1 -- 172.21.15.123:0/722540902 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4015070 con 0x7f55dc047010 2024-04-25T12:34:54.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.069+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc005290 con 0x7f55dc047010 2024-04-25T12:34:54.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.072+0000 7f55da000640 1 -- 172.21.15.123:0/722540902 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f55d4003610 con 0x7f55dc047010 2024-04-25T12:34:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.074+0000 7f55da000640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55c40746b0 0x7f55c4076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.074+0000 7f55da000640 1 -- 172.21.15.123:0/722540902 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f55d405ab60 con 0x7f55dc047010 2024-04-25T12:34:54.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.074+0000 7f55e1e00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55c40746b0 0x7f55c4076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.075+0000 7f55e1e00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55c40746b0 0x7f55c4076bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55e4065b40 tx=0x7f55e40af040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.079+0000 7f55da000640 1 -- 172.21.15.123:0/722540902 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55d405e570 con 0x7f55dc047010 2024-04-25T12:34:54.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.110+0000 7fb0df000640 1 Processor -- start 2024-04-25T12:34:54.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.111+0000 7fb0df000640 1 -- start start 2024-04-25T12:34:54.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.114+0000 7fb0df000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb0d00495d0 0x7fb0d00073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.114+0000 7fb0df000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d004f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.114+0000 7fb0df000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d0050ae0 con 0x7fb0d004fe70 2024-04-25T12:34:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.115+0000 7fb0df000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d00506f0 con 0x7fb0d00495d0 2024-04-25T12:34:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.115+0000 7fb0df000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d0050da0 con 0x7fb0d0007900 2024-04-25T12:34:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.115+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d004f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.115+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d004f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38246/0 (socket says 172.21.15.123:38246) 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.115+0000 7fb0d7400640 1 -- 172.21.15.123:0/711413601 learned_addr learned my addr 172.21.15.123:0/711413601 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.118+0000 7fb0d7e00640 1 --2- 172.21.15.123:0/711413601 >> v2:172.21.15.153:3300/0 conn(0x7fb0d00495d0 0x7fb0d00073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.118+0000 7fb0d7400640 1 -- 172.21.15.123:0/711413601 >> v2:172.21.15.153:3300/0 conn(0x7fb0d00495d0 msgr2=0x7fb0d00073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.118+0000 7fb0d7400640 1 --2- 172.21.15.123:0/711413601 >> v2:172.21.15.153:3300/0 conn(0x7fb0d00495d0 0x7fb0d00073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.118+0000 7fb0d7400640 1 -- 172.21.15.123:0/711413601 >> v1:172.21.15.153:6789/0 conn(0x7fb0d004fe70 legacy=0x7fb0d00df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.118+0000 7fb0d7400640 1 -- 172.21.15.123:0/711413601 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d00e35b0 con 0x7fb0d0007900 2024-04-25T12:34:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.119+0000 7fb0d7400640 1 --2- 172.21.15.123:0/711413601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d004f930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80520f0 tx=0x7fb0d80a78d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48cdc931d971ce15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.120+0000 7fb0d6a00640 1 -- 172.21.15.123:0/711413601 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80b4070 con 0x7fb0d0007900 2024-04-25T12:34:54.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.120+0000 7fb0d6a00640 1 -- 172.21.15.123:0/711413601 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d80af620 con 0x7fb0d0007900 2024-04-25T12:34:54.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.120+0000 7fb0d6a00640 1 -- 172.21.15.123:0/711413601 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80afa50 con 0x7fb0d0007900 2024-04-25T12:34:54.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.121+0000 7fb0df000640 1 -- 172.21.15.123:0/711413601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 msgr2=0x7fb0d004f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.121+0000 7fb0df000640 1 --2- 172.21.15.123:0/711413601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d004f930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80520f0 tx=0x7fb0d80a78d0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.121+0000 7fb0df000640 1 -- 172.21.15.123:0/711413601 shutdown_connections 2024-04-25T12:34:54.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.121+0000 7fb0df000640 1 --2- 172.21.15.123:0/711413601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d004f930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.121+0000 7fb0df000640 1 --2- 172.21.15.123:0/711413601 >> v2:172.21.15.153:3300/0 conn(0x7fb0d00495d0 0x7fb0d00073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.121+0000 7fb0df000640 1 -- 172.21.15.123:0/711413601 >> 172.21.15.123:0/711413601 conn(0x7fb0d000e370 msgr2=0x7fb0d000e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.123+0000 7fb0df000640 1 -- 172.21.15.123:0/711413601 shutdown_connections 2024-04-25T12:34:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.123+0000 7fb0df000640 1 -- 172.21.15.123:0/711413601 wait complete. 2024-04-25T12:34:54.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.134+0000 7fb0df000640 1 Processor -- start 2024-04-25T12:34:54.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.134+0000 7fb0df000640 1 -- start start 2024-04-25T12:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.137+0000 7fb0df000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d0006500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.137+0000 7fb0df000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00495d0 0x7fb0d00ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.137+0000 7fb0df000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d004fe70 0x7fb0d00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.137+0000 7fb0df000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0050280 con 0x7fb0d00495d0 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.137+0000 7fb0df000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d004f410 con 0x7fb0d004fe70 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.137+0000 7fb0df000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d00e3890 con 0x7fb0d0007900 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00495d0 0x7fb0d00ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00495d0 0x7fb0d00ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39548/0 (socket says 172.21.15.123:39548) 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 -- 172.21.15.123:0/2613982721 learned_addr learned my addr 172.21.15.123:0/2613982721 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 -- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 msgr2=0x7fb0d0006500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d0006500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 -- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d004fe70 msgr2=0x7fb0d00cf040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d004fe70 0x7fb0d00cf040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.138+0000 7fb0d7400640 1 -- 172.21.15.123:0/2613982721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d00d8d10 con 0x7fb0d00495d0 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.139+0000 7fb0d7400640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00495d0 0x7fb0d00ceb00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80771e0 tx=0x7fb0d806aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.141+0000 7fb0cbe00640 1 -- 172.21.15.123:0/2613982721 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80cf070 con 0x7fb0d00495d0 2024-04-25T12:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.141+0000 7fb0cbe00640 1 -- 172.21.15.123:0/2613982721 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d80aea60 con 0x7fb0d00495d0 2024-04-25T12:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.141+0000 7fb0cbe00640 1 -- 172.21.15.123:0/2613982721 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80cf070 con 0x7fb0d00495d0 2024-04-25T12:34:54.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.148+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d00d8fd0 con 0x7fb0d00495d0 2024-04-25T12:34:54.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.148+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d0178ce0 con 0x7fb0d00495d0 2024-04-25T12:34:54.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.152+0000 7fb0cbe00640 1 -- 172.21.15.123:0/2613982721 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb0d80d2070 con 0x7fb0d00495d0 2024-04-25T12:34:54.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.153+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d0005560 con 0x7fb0d00495d0 2024-04-25T12:34:54.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.154+0000 7fb0cbe00640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0b80746e0 0x7fb0b8076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.154+0000 7fb0cbe00640 1 -- 172.21.15.123:0/2613982721 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb0d8143a80 con 0x7fb0d00495d0 2024-04-25T12:34:54.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.155+0000 7fb0d7e00640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0b80746e0 0x7fb0b8076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.158+0000 7fb0d7e00640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0b80746e0 0x7fb0b8076bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00a790 tx=0x7fb0cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.170+0000 7fb0cbe00640 1 -- 172.21.15.123:0/2613982721 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0d810def0 con 0x7fb0d00495d0 2024-04-25T12:34:54.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.254+0000 7f47d1c00640 1 Processor -- start 2024-04-25T12:34:54.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.254+0000 7f47d1c00640 1 -- start start 2024-04-25T12:34:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.258+0000 7f47d1c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f47c40495e0 0x7f47c40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.259+0000 7f47d1c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c4050af0 con 0x7f47c4050050 2024-04-25T12:34:54.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.259+0000 7f47d1c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c4050db0 con 0x7f47c4007910 2024-04-25T12:34:54.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.259+0000 7f47d1c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f47c4050700 con 0x7f47c40495e0 2024-04-25T12:34:54.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.259+0000 7f47cb400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f47c4007910 0x7f47c404f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:40896/0 (socket says 172.21.15.123:40896) 2024-04-25T12:34:54.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.259+0000 7f47cb400640 1 -- 172.21.15.123:0/699590974 learned_addr learned my addr 172.21.15.123:0/699590974 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:54.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.259+0000 7f47d1200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f47c4050050 0x7f47c40e9da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45804/0 (socket says 172.21.15.123:45804) 2024-04-25T12:34:54.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.259+0000 7f47cbe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f47c40495e0 0x7f47c40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.261+0000 7f47caa00640 1 -- 172.21.15.123:0/699590974 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 24066836 0 0) 0x7f47c4050db0 con 0x7f47c4007910 2024-04-25T12:34:54.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.261+0000 7f47caa00640 1 -- 172.21.15.123:0/699590974 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47b4003520 con 0x7f47c4007910 2024-04-25T12:34:54.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.261+0000 7f47caa00640 1 -- 172.21.15.123:0/699590974 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927067041 0 0) 0x7f47c4050af0 con 0x7f47c4050050 2024-04-25T12:34:54.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.261+0000 7f47caa00640 1 -- 172.21.15.123:0/699590974 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47c4050db0 con 0x7f47c4050050 2024-04-25T12:34:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.261+0000 7f47cbe00640 1 -- 172.21.15.123:0/699590974 >> v1:172.21.15.153:6789/0 conn(0x7f47c4007910 legacy=0x7f47c404f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.261+0000 7f47cbe00640 1 -- 172.21.15.123:0/699590974 >> v1:172.21.15.123:6789/0 conn(0x7f47c4050050 legacy=0x7f47c40e9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.261+0000 7f47cbe00640 1 -- 172.21.15.123:0/699590974 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c40e35c0 con 0x7f47c40495e0 2024-04-25T12:34:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.267+0000 7f47cbe00640 1 --2- 172.21.15.123:0/699590974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c40495e0 0x7f47c40073d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47b80097a0 tx=0x7f47b802a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b47e65c4de241b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.271+0000 7f47caa00640 1 -- 172.21.15.123:0/699590974 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8037070 con 0x7f47c40495e0 2024-04-25T12:34:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.271+0000 7f47caa00640 1 -- 172.21.15.123:0/699590974 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b8032440 con 0x7f47c40495e0 2024-04-25T12:34:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.271+0000 7f47caa00640 1 -- 172.21.15.123:0/699590974 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8032830 con 0x7f47c40495e0 2024-04-25T12:34:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.272+0000 7f47d1c00640 1 -- 172.21.15.123:0/699590974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c40495e0 msgr2=0x7f47c40073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.272+0000 7f47d1c00640 1 --2- 172.21.15.123:0/699590974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c40495e0 0x7f47c40073d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47b80097a0 tx=0x7f47b802a290 comp rx=0 tx=0).stop 2024-04-25T12:34:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.273+0000 7f47d1c00640 1 -- 172.21.15.123:0/699590974 shutdown_connections 2024-04-25T12:34:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.273+0000 7f47d1c00640 1 --2- 172.21.15.123:0/699590974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c40495e0 0x7f47c40073d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.273+0000 7f47d1c00640 1 -- 172.21.15.123:0/699590974 >> 172.21.15.123:0/699590974 conn(0x7f47c400e380 msgr2=0x7f47c400e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.273+0000 7f47d1c00640 1 -- 172.21.15.123:0/699590974 shutdown_connections 2024-04-25T12:34:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.273+0000 7f47d1c00640 1 -- 172.21.15.123:0/699590974 wait complete. 2024-04-25T12:34:54.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.283+0000 7f47d1c00640 1 Processor -- start 2024-04-25T12:34:54.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.283+0000 7f47d1c00640 1 -- start start 2024-04-25T12:34:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.295+0000 7f47d1c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4007910 0x7f47c40db710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.295+0000 7f47d1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c40495e0 0x7f47c416ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.296+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4007910 0x7f47c40db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.296+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4007910 0x7f47c40db710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34070/0 (socket says 172.21.15.123:34070) 2024-04-25T12:34:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.296+0000 7f47cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c40495e0 0x7f47c416ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.297+0000 7f47cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c40495e0 0x7f47c416ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39550/0 (socket says 172.21.15.123:39550) 2024-04-25T12:34:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.297+0000 7f47cbe00640 1 -- 172.21.15.123:0/3739979293 learned_addr learned my addr 172.21.15.123:0/3739979293 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.297+0000 7f47d1c00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c4050050 0x7f47c40d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.301+0000 7f47d1200640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c4050050 0x7f47c40d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.301+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47c40e38a0 con 0x7f47c40495e0 2024-04-25T12:34:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.301+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47c404f420 con 0x7f47c4007910 2024-04-25T12:34:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.301+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f47c400f1c0 con 0x7f47c4050050 2024-04-25T12:34:54.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.302+0000 7f47cbe00640 1 -- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c4050050 msgr2=0x7f47c40d85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.302+0000 7f47cbe00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c4050050 0x7f47c40d85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.302+0000 7f47cbe00640 1 -- 172.21.15.123:0/3739979293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c40495e0 msgr2=0x7f47c416ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.303+0000 7f47cbe00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c40495e0 0x7f47c416ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.303+0000 7f47cbe00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c40dbc50 con 0x7f47c4007910 2024-04-25T12:34:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.303+0000 7f47cb400640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c40495e0 0x7f47c416ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:54.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.308+0000 7f47cbe00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4007910 0x7f47c40db710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47b8000c00 tx=0x7f47b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.309+0000 7f47bfe00640 1 -- 172.21.15.123:0/3739979293 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8037050 con 0x7f47c4007910 2024-04-25T12:34:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.309+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c416f890 con 0x7f47c4007910 2024-04-25T12:34:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.309+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c417db10 con 0x7f47c4007910 2024-04-25T12:34:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.310+0000 7f47bfe00640 1 -- 172.21.15.123:0/3739979293 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b8043070 con 0x7f47c4007910 2024-04-25T12:34:54.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.310+0000 7f47bfe00640 1 -- 172.21.15.123:0/3739979293 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8032a10 con 0x7f47c4007910 2024-04-25T12:34:54.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.315+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a8005290 con 0x7f47c4007910 2024-04-25T12:34:54.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.317+0000 7f47bfe00640 1 -- 172.21.15.123:0/3739979293 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f47b8032bb0 con 0x7f47c4007910 2024-04-25T12:34:54.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.319+0000 7f47bfe00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f47ac074530 0x7f47ac076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:54.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.319+0000 7f47bfe00640 1 -- 172.21.15.123:0/3739979293 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f47b80b5cf0 con 0x7f47c4007910 2024-04-25T12:34:54.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.339+0000 7f47bfe00640 1 -- 172.21.15.123:0/3739979293 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47b8080240 con 0x7f47c4007910 2024-04-25T12:34:54.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.343+0000 7f47cb400640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f47ac074530 0x7f47ac076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:54.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.347+0000 7f47cb400640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f47ac074530 0x7f47ac076a40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e240 tx=0x7f47c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:54.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.373+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7febb0005580 con 0x7febd806cab0 2024-04-25T12:34:54.375 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T12:34:54.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.374+0000 7febcf400640 1 -- 172.21.15.123:0/702742385 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7febd000ae20 con 0x7febd806cab0 2024-04-25T12:34:54.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.377+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febb80745b0 msgr2=0x7febb8076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.377+0000 7febccc00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febb80745b0 0x7febb8076ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febc8000f60 tx=0x7febc8009040 comp rx=0 tx=0).stop 2024-04-25T12:34:54.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.378+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd806cab0 msgr2=0x7febd81df360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.378+0000 7febccc00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd806cab0 0x7febd81df360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7febd000ce10 tx=0x7febd000a650 comp rx=0 tx=0).stop 2024-04-25T12:34:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.378+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 shutdown_connections 2024-04-25T12:34:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.378+0000 7febccc00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febb80745b0 0x7febb8076ac0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.379+0000 7febccc00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febd813e0b0 0x7febd8151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.379+0000 7febccc00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febd80ccf20 0x7febd81e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.379+0000 7febccc00640 1 --2- 172.21.15.123:0/702742385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd806cab0 0x7febd81df360 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.379+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 >> 172.21.15.123:0/702742385 conn(0x7febd80c9f60 msgr2=0x7febd80ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.379+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 shutdown_connections 2024-04-25T12:34:54.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.379+0000 7febccc00640 1 -- 172.21.15.123:0/702742385 wait complete. 2024-04-25T12:34:54.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.396+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f39d0076700 con 0x7f39d00c8520 2024-04-25T12:34:54.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.397+0000 7f39c7e00640 1 -- 172.21.15.123:0/872578973 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f39d0076700 con 0x7f39d00c8520 2024-04-25T12:34:54.402 INFO:teuthology.orchestra.run.smithi123.stdout:60129542156 2024-04-25T12:34:54.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39b00746e0 msgr2=0x7f39b0076bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39b00746e0 0x7f39b0076bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f39d015a4c0 tx=0x7f39c00090e0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8520 msgr2=0x7f39d015e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8520 0x7f39d015e850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39bc031000 tx=0x7f39bc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 shutdown_connections 2024-04-25T12:34:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f39b00746e0 0x7f39b0076bf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39d0158750 0x7f39d0158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d00c8930 0x7f39d0158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 --2- 172.21.15.123:0/872578973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39d00c8520 0x7f39d015e850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.405+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 >> 172.21.15.123:0/872578973 conn(0x7f39d00cd600 msgr2=0x7f39d006f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.406+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 shutdown_connections 2024-04-25T12:34:54.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.406+0000 7f39d5c00640 1 -- 172.21.15.123:0/872578973 wait complete. 2024-04-25T12:34:54.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.422+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f167816ba70 con 0x7f1678050050 2024-04-25T12:34:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.423+0000 7f167cc00640 1 -- 172.21.15.123:0/1790458753 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16800dc4c0 con 0x7f1678050050 2024-04-25T12:34:54.423 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T12:34:54.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.427+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1660074580 msgr2=0x7f1660076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.427+0000 7f1686000640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1660074580 0x7f1660076a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f166c034000 tx=0x7f166c02e000 comp rx=0 tx=0).stop 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1678050050 msgr2=0x7f16780536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1678050050 0x7f16780536b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f168006b4c0 tx=0x7f1680069cf0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 shutdown_connections 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1660074580 0x7f1660076a90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1678050050 0x7f16780536b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16780495e0 0x7f167816e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 --2- 172.21.15.123:0/1790458753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1678007910 0x7f167817a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 >> 172.21.15.123:0/1790458753 conn(0x7f167800e380 msgr2=0x7f16780e2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 shutdown_connections 2024-04-25T12:34:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.431+0000 7f1686000640 1 -- 172.21.15.123:0/1790458753 wait complete. 2024-04-25T12:34:54.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542155 for osd.1 2024-04-25T12:34:54.445 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.445+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7faff4005580 con 0x7fb01004c910 2024-04-25T12:34:54.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.446+0000 7fb00b400640 1 -- 172.21.15.123:0/345795283 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb004062c80 con 0x7fb01004c910 2024-04-25T12:34:54.446 INFO:teuthology.orchestra.run.smithi123.stdout:60129542156 2024-04-25T12:34:54.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.450+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faff8074670 msgr2=0x7faff8076b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.450+0000 7fb01d200640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faff8074670 0x7faff8076b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0180520f0 tx=0x7fb018070040 comp rx=0 tx=0).stop 2024-04-25T12:34:54.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.450+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 msgr2=0x7fb01004e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.450+0000 7fb01d200640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb01004e7f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb010045770 tx=0x7fb00400c870 comp rx=0 tx=0).stop 2024-04-25T12:34:54.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.451+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 shutdown_connections 2024-04-25T12:34:54.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.451+0000 7fb01d200640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faff8074670 0x7faff8076b80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.451+0000 7fb01d200640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0100de8e0 0x7fb01004f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.451+0000 7fb01d200640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01004c910 0x7fb01004e7f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.451+0000 7fb01d200640 1 --2- 172.21.15.123:0/345795283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010047180 0x7fb010048a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.451+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 >> 172.21.15.123:0/345795283 conn(0x7fb010050380 msgr2=0x7fb0100d52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.452+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 shutdown_connections 2024-04-25T12:34:54.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.452+0000 7fb01d200640 1 -- 172.21.15.123:0/345795283 wait complete. 2024-04-25T12:34:54.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.7 2024-04-25T12:34:54.456 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.0 2024-04-25T12:34:54.464 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.5 2024-04-25T12:34:54.485 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.498+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f364413d960 con 0x7f36440cc810 2024-04-25T12:34:54.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.499+0000 7f363a000640 1 -- 172.21.15.123:0/988817973 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f363c0950d0 con 0x7f36440cc810 2024-04-25T12:34:54.499 INFO:teuthology.orchestra.run.smithi123.stdout:60129542156 2024-04-25T12:34:54.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3620074580 msgr2=0x7f3620076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3620074580 0x7f3620076a90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f362c003940 tx=0x7f362c008040 comp rx=0 tx=0).stop 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36440cc810 msgr2=0x7f3644155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36440cc810 0x7f3644155740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f363c00ba10 tx=0x7f363c00bed0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 shutdown_connections 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3620074580 0x7f3620076a90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f36440cc810 0x7f3644155740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36440caa80 0x7f3644155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 --2- 172.21.15.123:0/988817973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364406c570 0x7f364415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 >> 172.21.15.123:0/988817973 conn(0x7f36440c9f60 msgr2=0x7f36440c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 shutdown_connections 2024-04-25T12:34:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.502+0000 7f3643200640 1 -- 172.21.15.123:0/988817973 wait complete. 2024-04-25T12:34:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.507+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f55bc005850 con 0x7f55dc047010 2024-04-25T12:34:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.507+0000 7f55da000640 1 -- 172.21.15.123:0/722540902 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55d4062870 con 0x7f55dc047010 2024-04-25T12:34:54.507 INFO:teuthology.orchestra.run.smithi123.stdout:60129542156 2024-04-25T12:34:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.509+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55c40746b0 msgr2=0x7f55c4076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.509+0000 7f55e8c00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55c40746b0 0x7f55c4076bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55e4065b40 tx=0x7f55e40af040 comp rx=0 tx=0).stop 2024-04-25T12:34:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.509+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc047010 msgr2=0x7f55dc0dafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.509+0000 7f55e8c00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc047010 0x7f55dc0dafc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55dc04e2b0 tx=0x7f55d400c9d0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 shutdown_connections 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55c40746b0 0x7f55c4076bc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55dc047010 0x7f55dc0dafc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55dc045b80 0x7f55dc0daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 --2- 172.21.15.123:0/722540902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55dc045380 0x7f55dc0de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 >> 172.21.15.123:0/722540902 conn(0x7f55dc0503e0 msgr2=0x7f55dc0057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 shutdown_connections 2024-04-25T12:34:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.510+0000 7f55e8c00640 1 -- 172.21.15.123:0/722540902 wait complete. 2024-04-25T12:34:54.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.6 2024-04-25T12:34:54.532 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.4 2024-04-25T12:34:54.537 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.657+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb0d00508c0 con 0x7fb0d00495d0 2024-04-25T12:34:54.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.658+0000 7fb0cbe00640 1 -- 172.21.15.123:0/2613982721 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb0d810d7f0 con 0x7fb0d00495d0 2024-04-25T12:34:54.658 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0b80746e0 msgr2=0x7fb0b8076bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0b80746e0 0x7fb0b8076bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00a790 tx=0x7fb0cc008040 comp rx=0 tx=0).stop 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00495d0 msgr2=0x7fb0d00ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00495d0 0x7fb0d00ceb00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80771e0 tx=0x7fb0d806aa10 comp rx=0 tx=0).stop 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 shutdown_connections 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0b80746e0 0x7fb0b8076bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d004fe70 0x7fb0d00cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00495d0 0x7fb0d00ceb00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 --2- 172.21.15.123:0/2613982721 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb0d0007900 0x7fb0d0006500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.660+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 >> 172.21.15.123:0/2613982721 conn(0x7fb0d000e370 msgr2=0x7fb0d004ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.661+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 shutdown_connections 2024-04-25T12:34:54.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.661+0000 7fb0df000640 1 -- 172.21.15.123:0/2613982721 wait complete. 2024-04-25T12:34:54.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.3 2024-04-25T12:34:54.684 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.714+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f47a8005580 con 0x7f47c4007910 2024-04-25T12:34:54.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.715+0000 7f47bfe00640 1 -- 172.21.15.123:0/3739979293 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47b8088230 con 0x7f47c4007910 2024-04-25T12:34:54.715 INFO:teuthology.orchestra.run.smithi123.stdout:60129542156 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f47ac074530 msgr2=0x7f47ac076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f47ac074530 0x7f47ac076a40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e240 tx=0x7f47c0009040 comp rx=0 tx=0).stop 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4007910 msgr2=0x7f47c40db710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4007910 0x7f47c40db710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47b8000c00 tx=0x7f47b80046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 shutdown_connections 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f47ac074530 0x7f47ac076a40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c4050050 0x7f47c40d85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c40495e0 0x7f47c416ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 --2- 172.21.15.123:0/3739979293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4007910 0x7f47c40db710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 >> 172.21.15.123:0/3739979293 conn(0x7f47c400e380 msgr2=0x7f47c4049b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:54.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 shutdown_connections 2024-04-25T12:34:54.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:54.717+0000 7f47d1c00640 1 -- 172.21.15.123:0/3739979293 wait complete. 2024-04-25T12:34:54.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.2 2024-04-25T12:34:54.739 DEBUG:teuthology.parallel:result is None 2024-04-25T12:34:54.739 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T12:34:54.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:34:55.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.032+0000 7f71c5200640 1 Processor -- start 2024-04-25T12:34:55.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.032+0000 7f71c5200640 1 -- start start 2024-04-25T12:34:55.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.034+0000 7f71c5200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71c015c470 0x7f71c0160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.034+0000 7f71c5200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c00cb250 con 0x7f71c00cc3b0 2024-04-25T12:34:55.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.034+0000 7f71c5200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c00cb530 con 0x7f71c006eaa0 2024-04-25T12:34:55.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.034+0000 7f71c5200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f71c00caa90 con 0x7f71c015c470 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71bf400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71c015c470 0x7f71c0160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71bfe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f71c00cc3b0 0x7f71c0166d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45812/0 (socket says 172.21.15.123:45812) 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71bfe00640 1 -- 172.21.15.123:0/3919367304 learned_addr learned my addr 172.21.15.123:0/3919367304 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71be000640 1 -- 172.21.15.123:0/3919367304 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2643233859 0 0) 0x7f71c00cb250 con 0x7f71c00cc3b0 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71be000640 1 -- 172.21.15.123:0/3919367304 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71a4003520 con 0x7f71c00cc3b0 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71be000640 1 -- 172.21.15.123:0/3919367304 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860493446 0 0) 0x7f71c00cb530 con 0x7f71c006eaa0 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71be000640 1 -- 172.21.15.123:0/3919367304 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71c00cb250 con 0x7f71c006eaa0 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71bf400640 1 -- 172.21.15.123:0/3919367304 >> v1:172.21.15.153:6789/0 conn(0x7f71c006eaa0 legacy=0x7f71c00ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71bf400640 1 -- 172.21.15.123:0/3919367304 >> v1:172.21.15.123:6789/0 conn(0x7f71c00cc3b0 legacy=0x7f71c0166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.035+0000 7f71bf400640 1 -- 172.21.15.123:0/3919367304 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c00cb810 con 0x7f71c015c470 2024-04-25T12:34:55.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.036+0000 7f71bf400640 1 --2- 172.21.15.123:0/3919367304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c015c470 0x7f71c0160280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71b00097a0 tx=0x7f71b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e230b18a4d06763 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.036+0000 7f71be000640 1 -- 172.21.15.123:0/3919367304 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0037070 con 0x7f71c015c470 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.036+0000 7f71be000640 1 -- 172.21.15.123:0/3919367304 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b0032440 con 0x7f71c015c470 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.036+0000 7f71be000640 1 -- 172.21.15.123:0/3919367304 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00327f0 con 0x7f71c015c470 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.037+0000 7f71c5200640 1 -- 172.21.15.123:0/3919367304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c015c470 msgr2=0x7f71c0160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.037+0000 7f71c5200640 1 --2- 172.21.15.123:0/3919367304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c015c470 0x7f71c0160280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71b00097a0 tx=0x7f71b002a270 comp rx=0 tx=0).stop 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.037+0000 7f71c5200640 1 -- 172.21.15.123:0/3919367304 shutdown_connections 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.037+0000 7f71c5200640 1 --2- 172.21.15.123:0/3919367304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c015c470 0x7f71c0160280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.037+0000 7f71c5200640 1 -- 172.21.15.123:0/3919367304 >> 172.21.15.123:0/3919367304 conn(0x7f71c0072cb0 msgr2=0x7f71c00730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.037+0000 7f71c5200640 1 -- 172.21.15.123:0/3919367304 shutdown_connections 2024-04-25T12:34:55.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.037+0000 7f71c5200640 1 -- 172.21.15.123:0/3919367304 wait complete. 2024-04-25T12:34:55.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.039+0000 7f71c5200640 1 Processor -- start 2024-04-25T12:34:55.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.039+0000 7f71c5200640 1 -- start start 2024-04-25T12:34:55.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71c5200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71c006eaa0 0x7f71c014ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71c5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c00cc3b0 0x7f71c01e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71c5200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71c015c470 0x7f71c015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71c5200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71c00cbaf0 con 0x7f71c00cc3b0 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71c5200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71c015fd60 con 0x7f71c006eaa0 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71c5200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f71c01607c0 con 0x7f71c015c470 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71bf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71c006eaa0 0x7f71c014ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71bea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c00cc3b0 0x7f71c01e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71c015c470 0x7f71c015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71bf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71c006eaa0 0x7f71c014ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34076/0 (socket says 172.21.15.123:34076) 2024-04-25T12:34:55.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.041+0000 7f71bf400640 1 -- 172.21.15.123:0/4259631574 learned_addr learned my addr 172.21.15.123:0/4259631574 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:55.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71bf400640 1 -- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71c015c470 msgr2=0x7f71c015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71bf400640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71c015c470 0x7f71c015e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71bf400640 1 -- 172.21.15.123:0/4259631574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c00cc3b0 msgr2=0x7f71c01e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71bf400640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c00cc3b0 0x7f71c01e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71bf400640 1 -- 172.21.15.123:0/4259631574 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c014d340 con 0x7f71c006eaa0 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71bea00640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c00cc3b0 0x7f71c01e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71bf400640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71c006eaa0 0x7f71c014ce00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71b00098c0 tx=0x7f71b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71b7400640 1 -- 172.21.15.123:0/4259631574 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0037050 con 0x7f71c006eaa0 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71b7400640 1 -- 172.21.15.123:0/4259631574 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b0043070 con 0x7f71c006eaa0 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c00cf040 con 0x7f71c006eaa0 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71b7400640 1 -- 172.21.15.123:0/4259631574 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0003ea0 con 0x7f71c006eaa0 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.042+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c015f140 con 0x7f71c006eaa0 2024-04-25T12:34:55.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.043+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0076da0 con 0x7f71c006eaa0 2024-04-25T12:34:55.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.045+0000 7f71b7400640 1 -- 172.21.15.123:0/4259631574 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71b0032a10 con 0x7f71c006eaa0 2024-04-25T12:34:55.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.046+0000 7f71b7400640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71a00744b0 0x7f71a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.046+0000 7f71b7400640 1 -- 172.21.15.123:0/4259631574 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f71b00b6780 con 0x7f71c006eaa0 2024-04-25T12:34:55.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.046+0000 7f71bea00640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71a00744b0 0x7f71a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.047+0000 7f71bea00640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71a00744b0 0x7f71a00769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71c015cfd0 tx=0x7f71ac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:55.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.049+0000 7f71b7400640 1 -- 172.21.15.123:0/4259631574 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71b0080560 con 0x7f71c006eaa0 2024-04-25T12:34:55.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.242+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71c00c6110 con 0x7f71a00744b0 2024-04-25T12:34:55.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.251+0000 7f71b7400640 1 -- 172.21.15.123:0/4259631574 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332640 (secure 0 0 0) 0x7f71c00c6110 con 0x7f71a00744b0 2024-04-25T12:34:55.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:55.261 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.256+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71a00744b0 msgr2=0x7f71a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.256+0000 7f71c5200640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71a00744b0 0x7f71a00769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71c015cfd0 tx=0x7f71ac009040 comp rx=0 tx=0).stop 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.256+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71c006eaa0 msgr2=0x7f71c014ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.256+0000 7f71c5200640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71c006eaa0 0x7f71c014ce00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71b00098c0 tx=0x7f71b00046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 shutdown_connections 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f71a00744b0 0x7f71a00769c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f71c015c470 0x7f71c015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c00cc3b0 0x7f71c01e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 --2- 172.21.15.123:0/4259631574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71c006eaa0 0x7f71c014ce00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 >> 172.21.15.123:0/4259631574 conn(0x7f71c0072cb0 msgr2=0x7f71c00730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:55.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 shutdown_connections 2024-04-25T12:34:55.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.257+0000 7f71c5200640 1 -- 172.21.15.123:0/4259631574 wait complete. 2024-04-25T12:34:55.281 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-25T12:34:54.802406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596414,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":189,"num_write_kb":1659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218384,"kb_used_data":2256,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756336,"statfs":{"total":773094113280,"available":772870488064,"internally_reserved":0,"allocated":2310144,"data_stored":2016292,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003991"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783414+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:34:19.783414+0000","last_peered":"2024-04-25T12:34:19.783414+0000","last_clean":"2024-04-25T12:34:19.783414+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:34:19.783414+0000","last_undegraded":"2024-04-25T12:34:19.783414+0000","last_fullsized":"2024-04-25T12:34:19.783414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782890+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:34:19.782890+0000","last_peered":"2024-04-25T12:34:19.782890+0000","last_clean":"2024-04-25T12:34:19.782890+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:34:19.782890+0000","last_undegraded":"2024-04-25T12:34:19.782890+0000","last_fullsized":"2024-04-25T12:34:19.782890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780793+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:34:19.780793+0000","last_peered":"2024-04-25T12:34:19.780793+0000","last_clean":"2024-04-25T12:34:19.780793+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:34:19.780793+0000","last_undegraded":"2024-04-25T12:34:19.780793+0000","last_fullsized":"2024-04-25T12:34:19.780793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782997+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:34:19.782997+0000","last_peered":"2024-04-25T12:34:19.782997+0000","last_clean":"2024-04-25T12:34:19.782997+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:34:19.782997+0000","last_undegraded":"2024-04-25T12:34:19.782997+0000","last_fullsized":"2024-04-25T12:34:19.782997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782606+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:34:19.782606+0000","last_peered":"2024-04-25T12:34:19.782606+0000","last_clean":"2024-04-25T12:34:19.782606+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:34:19.782606+0000","last_undegraded":"2024-04-25T12:34:19.782606+0000","last_fullsized":"2024-04-25T12:34:19.782606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784500+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:34:19.784500+0000","last_peered":"2024-04-25T12:34:19.784500+0000","last_clean":"2024-04-25T12:34:19.784500+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:34:19.784500+0000","last_undegraded":"2024-04-25T12:34:19.784500+0000","last_fullsized":"2024-04-25T12:34:19.784500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785563+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:34:19.785563+0000","last_peered":"2024-04-25T12:34:19.785563+0000","last_clean":"2024-04-25T12:34:19.785563+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:34:19.785563+0000","last_undegraded":"2024-04-25T12:34:19.785563+0000","last_fullsized":"2024-04-25T12:34:19.785563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.790406+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:34:19.790406+0000","last_peered":"2024-04-25T12:34:19.790406+0000","last_clean":"2024-04-25T12:34:19.790406+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:34:19.790406+0000","last_undegraded":"2024-04-25T12:34:19.790406+0000","last_fullsized":"2024-04-25T12:34:19.790406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788036+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:34:19.788036+0000","last_peered":"2024-04-25T12:34:19.788036+0000","last_clean":"2024-04-25T12:34:19.788036+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:34:19.788036+0000","last_undegraded":"2024-04-25T12:34:19.788036+0000","last_fullsized":"2024-04-25T12:34:19.788036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784075+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:34:19.784075+0000","last_peered":"2024-04-25T12:34:19.784075+0000","last_clean":"2024-04-25T12:34:19.784075+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:34:19.784075+0000","last_undegraded":"2024-04-25T12:34:19.784075+0000","last_fullsized":"2024-04-25T12:34:19.784075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982844+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:34:18.982844+0000","last_peered":"2024-04-25T12:34:18.982844+0000","last_clean":"2024-04-25T12:34:18.982844+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:34:18.982844+0000","last_undegraded":"2024-04-25T12:34:18.982844+0000","last_fullsized":"2024-04-25T12:34:18.982844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786691+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:34:19.786691+0000","last_peered":"2024-04-25T12:34:19.786691+0000","last_clean":"2024-04-25T12:34:19.786691+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:34:19.786691+0000","last_undegraded":"2024-04-25T12:34:19.786691+0000","last_fullsized":"2024-04-25T12:34:19.786691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982590+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:34:18.982590+0000","last_peered":"2024-04-25T12:34:18.982590+0000","last_clean":"2024-04-25T12:34:18.982590+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:34:18.982590+0000","last_undegraded":"2024-04-25T12:34:18.982590+0000","last_fullsized":"2024-04-25T12:34:18.982590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130152+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:34:50.130152+0000","last_peered":"2024-04-25T12:34:50.130152+0000","last_clean":"2024-04-25T12:34:50.130152+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:34:50.130152+0000","last_undegraded":"2024-04-25T12:34:50.130152+0000","last_fullsized":"2024-04-25T12:34:50.130152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982528+0000","last_change":"2024-04-25T12:34:18.007956+0000","last_active":"2024-04-25T12:34:18.982528+0000","last_peered":"2024-04-25T12:34:18.982528+0000","last_clean":"2024-04-25T12:34:18.982528+0000","last_became_active":"2024-04-25T12:34:18.007500+0000","last_became_peered":"2024-04-25T12:34:18.007500+0000","last_unstale":"2024-04-25T12:34:18.982528+0000","last_undegraded":"2024-04-25T12:34:18.982528+0000","last_fullsized":"2024-04-25T12:34:18.982528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:56.400953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782265+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:34:19.782265+0000","last_peered":"2024-04-25T12:34:19.782265+0000","last_clean":"2024-04-25T12:34:19.782265+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:34:19.782265+0000","last_undegraded":"2024-04-25T12:34:19.782265+0000","last_fullsized":"2024-04-25T12:34:19.782265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980152+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:34:18.980152+0000","last_peered":"2024-04-25T12:34:18.980152+0000","last_clean":"2024-04-25T12:34:18.980152+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:34:18.980152+0000","last_undegraded":"2024-04-25T12:34:18.980152+0000","last_fullsized":"2024-04-25T12:34:18.980152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783204+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:34:19.783204+0000","last_peered":"2024-04-25T12:34:19.783204+0000","last_clean":"2024-04-25T12:34:19.783204+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:34:19.783204+0000","last_undegraded":"2024-04-25T12:34:19.783204+0000","last_fullsized":"2024-04-25T12:34:19.783204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782958+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:34:19.782958+0000","last_peered":"2024-04-25T12:34:19.782958+0000","last_clean":"2024-04-25T12:34:19.782958+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:34:19.782958+0000","last_undegraded":"2024-04-25T12:34:19.782958+0000","last_fullsized":"2024-04-25T12:34:19.782958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130938+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:34:50.130938+0000","last_peered":"2024-04-25T12:34:50.130938+0000","last_clean":"2024-04-25T12:34:50.130938+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:34:50.130938+0000","last_undegraded":"2024-04-25T12:34:50.130938+0000","last_fullsized":"2024-04-25T12:34:50.130938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781948+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:34:19.781948+0000","last_peered":"2024-04-25T12:34:19.781948+0000","last_clean":"2024-04-25T12:34:19.781948+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:34:19.781948+0000","last_undegraded":"2024-04-25T12:34:19.781948+0000","last_fullsized":"2024-04-25T12:34:19.781948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981753+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:34:18.981753+0000","last_peered":"2024-04-25T12:34:18.981753+0000","last_clean":"2024-04-25T12:34:18.981753+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:34:18.981753+0000","last_undegraded":"2024-04-25T12:34:18.981753+0000","last_fullsized":"2024-04-25T12:34:18.981753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.131168+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:34:50.131168+0000","last_peered":"2024-04-25T12:34:50.131168+0000","last_clean":"2024-04-25T12:34:50.131168+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:34:50.131168+0000","last_undegraded":"2024-04-25T12:34:50.131168+0000","last_fullsized":"2024-04-25T12:34:50.131168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788041+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:34:19.788041+0000","last_peered":"2024-04-25T12:34:19.788041+0000","last_clean":"2024-04-25T12:34:19.788041+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:34:19.788041+0000","last_undegraded":"2024-04-25T12:34:19.788041+0000","last_fullsized":"2024-04-25T12:34:19.788041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783014+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:34:19.783014+0000","last_peered":"2024-04-25T12:34:19.783014+0000","last_clean":"2024-04-25T12:34:19.783014+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:34:19.783014+0000","last_undegraded":"2024-04-25T12:34:19.783014+0000","last_fullsized":"2024-04-25T12:34:19.783014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.131059+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:34:50.131059+0000","last_peered":"2024-04-25T12:34:50.131059+0000","last_clean":"2024-04-25T12:34:50.131059+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:34:50.131059+0000","last_undegraded":"2024-04-25T12:34:50.131059+0000","last_fullsized":"2024-04-25T12:34:50.131059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787014+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:34:19.787014+0000","last_peered":"2024-04-25T12:34:19.787014+0000","last_clean":"2024-04-25T12:34:19.787014+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:34:19.787014+0000","last_undegraded":"2024-04-25T12:34:19.787014+0000","last_fullsized":"2024-04-25T12:34:19.787014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783329+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:34:19.783329+0000","last_peered":"2024-04-25T12:34:19.783329+0000","last_clean":"2024-04-25T12:34:19.783329+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:34:19.783329+0000","last_undegraded":"2024-04-25T12:34:19.783329+0000","last_fullsized":"2024-04-25T12:34:19.783329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783247+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:34:19.783247+0000","last_peered":"2024-04-25T12:34:19.783247+0000","last_clean":"2024-04-25T12:34:19.783247+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:34:19.783247+0000","last_undegraded":"2024-04-25T12:34:19.783247+0000","last_fullsized":"2024-04-25T12:34:19.783247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781103+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:34:19.781103+0000","last_peered":"2024-04-25T12:34:19.781103+0000","last_clean":"2024-04-25T12:34:19.781103+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:34:19.781103+0000","last_undegraded":"2024-04-25T12:34:19.781103+0000","last_fullsized":"2024-04-25T12:34:19.781103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980759+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:34:18.980759+0000","last_peered":"2024-04-25T12:34:18.980759+0000","last_clean":"2024-04-25T12:34:18.980759+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:34:18.980759+0000","last_undegraded":"2024-04-25T12:34:18.980759+0000","last_fullsized":"2024-04-25T12:34:18.980759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983904+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:34:18.983904+0000","last_peered":"2024-04-25T12:34:18.983904+0000","last_clean":"2024-04-25T12:34:18.983904+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:34:18.983904+0000","last_undegraded":"2024-04-25T12:34:18.983904+0000","last_fullsized":"2024-04-25T12:34:18.983904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980463+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:34:18.980463+0000","last_peered":"2024-04-25T12:34:18.980463+0000","last_clean":"2024-04-25T12:34:18.980463+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:34:18.980463+0000","last_undegraded":"2024-04-25T12:34:18.980463+0000","last_fullsized":"2024-04-25T12:34:18.980463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789572+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:34:19.789572+0000","last_peered":"2024-04-25T12:34:19.789572+0000","last_clean":"2024-04-25T12:34:19.789572+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:34:19.789572+0000","last_undegraded":"2024-04-25T12:34:19.789572+0000","last_fullsized":"2024-04-25T12:34:19.789572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782068+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:34:19.782068+0000","last_peered":"2024-04-25T12:34:19.782068+0000","last_clean":"2024-04-25T12:34:19.782068+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:34:19.782068+0000","last_undegraded":"2024-04-25T12:34:19.782068+0000","last_fullsized":"2024-04-25T12:34:19.782068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979987+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:34:18.979987+0000","last_peered":"2024-04-25T12:34:18.979987+0000","last_clean":"2024-04-25T12:34:18.979987+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:34:18.979987+0000","last_undegraded":"2024-04-25T12:34:18.979987+0000","last_fullsized":"2024-04-25T12:34:18.979987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783310+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:34:19.783310+0000","last_peered":"2024-04-25T12:34:19.783310+0000","last_clean":"2024-04-25T12:34:19.783310+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:34:19.783310+0000","last_undegraded":"2024-04-25T12:34:19.783310+0000","last_fullsized":"2024-04-25T12:34:19.783310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982842+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:34:18.982842+0000","last_peered":"2024-04-25T12:34:18.982842+0000","last_clean":"2024-04-25T12:34:18.982842+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:34:18.982842+0000","last_undegraded":"2024-04-25T12:34:18.982842+0000","last_fullsized":"2024-04-25T12:34:18.982842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980473+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:34:18.980473+0000","last_peered":"2024-04-25T12:34:18.980473+0000","last_clean":"2024-04-25T12:34:18.980473+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:34:18.980473+0000","last_undegraded":"2024-04-25T12:34:18.980473+0000","last_fullsized":"2024-04-25T12:34:18.980473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781093+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:34:19.781093+0000","last_peered":"2024-04-25T12:34:19.781093+0000","last_clean":"2024-04-25T12:34:19.781093+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:34:19.781093+0000","last_undegraded":"2024-04-25T12:34:19.781093+0000","last_fullsized":"2024-04-25T12:34:19.781093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781512+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:34:19.781512+0000","last_peered":"2024-04-25T12:34:19.781512+0000","last_clean":"2024-04-25T12:34:19.781512+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:34:19.781512+0000","last_undegraded":"2024-04-25T12:34:19.781512+0000","last_fullsized":"2024-04-25T12:34:19.781512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980445+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:34:18.980445+0000","last_peered":"2024-04-25T12:34:18.980445+0000","last_clean":"2024-04-25T12:34:18.980445+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:34:18.980445+0000","last_undegraded":"2024-04-25T12:34:18.980445+0000","last_fullsized":"2024-04-25T12:34:18.980445+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979799+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:34:18.979799+0000","last_peered":"2024-04-25T12:34:18.979799+0000","last_clean":"2024-04-25T12:34:18.979799+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:34:18.979799+0000","last_undegraded":"2024-04-25T12:34:18.979799+0000","last_fullsized":"2024-04-25T12:34:18.979799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982530+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:34:18.982530+0000","last_peered":"2024-04-25T12:34:18.982530+0000","last_clean":"2024-04-25T12:34:18.982530+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:34:18.982530+0000","last_undegraded":"2024-04-25T12:34:18.982530+0000","last_fullsized":"2024-04-25T12:34:18.982530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782825+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:34:19.782825+0000","last_peered":"2024-04-25T12:34:19.782825+0000","last_clean":"2024-04-25T12:34:19.782825+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:34:19.782825+0000","last_undegraded":"2024-04-25T12:34:19.782825+0000","last_fullsized":"2024-04-25T12:34:19.782825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783355+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:34:19.783355+0000","last_peered":"2024-04-25T12:34:19.783355+0000","last_clean":"2024-04-25T12:34:19.783355+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:34:19.783355+0000","last_undegraded":"2024-04-25T12:34:19.783355+0000","last_fullsized":"2024-04-25T12:34:19.783355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130560+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:34:50.130560+0000","last_peered":"2024-04-25T12:34:50.130560+0000","last_clean":"2024-04-25T12:34:50.130560+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:34:50.130560+0000","last_undegraded":"2024-04-25T12:34:50.130560+0000","last_fullsized":"2024-04-25T12:34:50.130560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787419+0000","last_change":"2024-04-25T12:34:08.560989+0000","last_active":"2024-04-25T12:34:19.787419+0000","last_peered":"2024-04-25T12:34:19.787419+0000","last_clean":"2024-04-25T12:34:19.787419+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:34:19.787419+0000","last_undegraded":"2024-04-25T12:34:19.787419+0000","last_fullsized":"2024-04-25T12:34:19.787419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:49.130809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983747+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:34:18.983747+0000","last_peered":"2024-04-25T12:34:18.983747+0000","last_clean":"2024-04-25T12:34:18.983747+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:34:18.983747+0000","last_undegraded":"2024-04-25T12:34:18.983747+0000","last_fullsized":"2024-04-25T12:34:18.983747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982942+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:34:18.982942+0000","last_peered":"2024-04-25T12:34:18.982942+0000","last_clean":"2024-04-25T12:34:18.982942+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:34:18.982942+0000","last_undegraded":"2024-04-25T12:34:18.982942+0000","last_fullsized":"2024-04-25T12:34:18.982942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980766+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:34:18.980766+0000","last_peered":"2024-04-25T12:34:18.980766+0000","last_clean":"2024-04-25T12:34:18.980766+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:34:18.980766+0000","last_undegraded":"2024-04-25T12:34:18.980766+0000","last_fullsized":"2024-04-25T12:34:18.980766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786888+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:34:19.786888+0000","last_peered":"2024-04-25T12:34:19.786888+0000","last_clean":"2024-04-25T12:34:19.786888+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:34:19.786888+0000","last_undegraded":"2024-04-25T12:34:19.786888+0000","last_fullsized":"2024-04-25T12:34:19.786888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783906+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:34:19.783906+0000","last_peered":"2024-04-25T12:34:19.783906+0000","last_clean":"2024-04-25T12:34:19.783906+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:34:19.783906+0000","last_undegraded":"2024-04-25T12:34:19.783906+0000","last_fullsized":"2024-04-25T12:34:19.783906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782493+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:34:19.782493+0000","last_peered":"2024-04-25T12:34:19.782493+0000","last_clean":"2024-04-25T12:34:19.782493+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:34:19.782493+0000","last_undegraded":"2024-04-25T12:34:19.782493+0000","last_fullsized":"2024-04-25T12:34:19.782493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781393+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:34:19.781393+0000","last_peered":"2024-04-25T12:34:19.781393+0000","last_clean":"2024-04-25T12:34:19.781393+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:34:19.781393+0000","last_undegraded":"2024-04-25T12:34:19.781393+0000","last_fullsized":"2024-04-25T12:34:19.781393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785529+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:34:19.785529+0000","last_peered":"2024-04-25T12:34:19.785529+0000","last_clean":"2024-04-25T12:34:19.785529+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:34:19.785529+0000","last_undegraded":"2024-04-25T12:34:19.785529+0000","last_fullsized":"2024-04-25T12:34:19.785529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783018+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:34:19.783018+0000","last_peered":"2024-04-25T12:34:19.783018+0000","last_clean":"2024-04-25T12:34:19.783018+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:34:19.783018+0000","last_undegraded":"2024-04-25T12:34:19.783018+0000","last_fullsized":"2024-04-25T12:34:19.783018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981441+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:34:18.981441+0000","last_peered":"2024-04-25T12:34:18.981441+0000","last_clean":"2024-04-25T12:34:18.981441+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:34:18.981441+0000","last_undegraded":"2024-04-25T12:34:18.981441+0000","last_fullsized":"2024-04-25T12:34:18.981441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788784+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:34:19.788784+0000","last_peered":"2024-04-25T12:34:19.788784+0000","last_clean":"2024-04-25T12:34:19.788784+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:34:19.788784+0000","last_undegraded":"2024-04-25T12:34:19.788784+0000","last_fullsized":"2024-04-25T12:34:19.788784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785856+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:34:19.785856+0000","last_peered":"2024-04-25T12:34:19.785856+0000","last_clean":"2024-04-25T12:34:19.785856+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:34:19.785856+0000","last_undegraded":"2024-04-25T12:34:19.785856+0000","last_fullsized":"2024-04-25T12:34:19.785856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788439+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:34:19.788439+0000","last_peered":"2024-04-25T12:34:19.788439+0000","last_clean":"2024-04-25T12:34:19.788439+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:34:19.788439+0000","last_undegraded":"2024-04-25T12:34:19.788439+0000","last_fullsized":"2024-04-25T12:34:19.788439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981709+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:34:18.981709+0000","last_peered":"2024-04-25T12:34:18.981709+0000","last_clean":"2024-04-25T12:34:18.981709+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:34:18.981709+0000","last_undegraded":"2024-04-25T12:34:18.981709+0000","last_fullsized":"2024-04-25T12:34:18.981709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.119769+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:34:50.119769+0000","last_peered":"2024-04-25T12:34:50.119769+0000","last_clean":"2024-04-25T12:34:50.119769+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:34:50.119769+0000","last_undegraded":"2024-04-25T12:34:50.119769+0000","last_fullsized":"2024-04-25T12:34:50.119769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782909+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:34:19.782909+0000","last_peered":"2024-04-25T12:34:19.782909+0000","last_clean":"2024-04-25T12:34:19.782909+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:34:19.782909+0000","last_undegraded":"2024-04-25T12:34:19.782909+0000","last_fullsized":"2024-04-25T12:34:19.782909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786692+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:34:19.786692+0000","last_peered":"2024-04-25T12:34:19.786692+0000","last_clean":"2024-04-25T12:34:19.786692+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:34:19.786692+0000","last_undegraded":"2024-04-25T12:34:19.786692+0000","last_fullsized":"2024-04-25T12:34:19.786692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782692+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:34:19.782692+0000","last_peered":"2024-04-25T12:34:19.782692+0000","last_clean":"2024-04-25T12:34:19.782692+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:34:19.782692+0000","last_undegraded":"2024-04-25T12:34:19.782692+0000","last_fullsized":"2024-04-25T12:34:19.782692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782809+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:34:19.782809+0000","last_peered":"2024-04-25T12:34:19.782809+0000","last_clean":"2024-04-25T12:34:19.782809+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:34:19.782809+0000","last_undegraded":"2024-04-25T12:34:19.782809+0000","last_fullsized":"2024-04-25T12:34:19.782809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983105+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:34:18.983105+0000","last_peered":"2024-04-25T12:34:18.983105+0000","last_clean":"2024-04-25T12:34:18.983105+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:34:18.983105+0000","last_undegraded":"2024-04-25T12:34:18.983105+0000","last_fullsized":"2024-04-25T12:34:18.983105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.128291+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:34:50.128291+0000","last_peered":"2024-04-25T12:34:50.128291+0000","last_clean":"2024-04-25T12:34:50.128291+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:34:50.128291+0000","last_undegraded":"2024-04-25T12:34:50.128291+0000","last_fullsized":"2024-04-25T12:34:50.128291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980674+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:34:18.980674+0000","last_peered":"2024-04-25T12:34:18.980674+0000","last_clean":"2024-04-25T12:34:18.980674+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:34:18.980674+0000","last_undegraded":"2024-04-25T12:34:18.980674+0000","last_fullsized":"2024-04-25T12:34:18.980674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980038+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:34:18.980038+0000","last_peered":"2024-04-25T12:34:18.980038+0000","last_clean":"2024-04-25T12:34:18.980038+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:34:18.980038+0000","last_undegraded":"2024-04-25T12:34:18.980038+0000","last_fullsized":"2024-04-25T12:34:18.980038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.120594+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:34:50.120594+0000","last_peered":"2024-04-25T12:34:50.120594+0000","last_clean":"2024-04-25T12:34:50.120594+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:34:50.120594+0000","last_undegraded":"2024-04-25T12:34:50.120594+0000","last_fullsized":"2024-04-25T12:34:50.120594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982809+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:34:18.982809+0000","last_peered":"2024-04-25T12:34:18.982809+0000","last_clean":"2024-04-25T12:34:18.982809+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:34:18.982809+0000","last_undegraded":"2024-04-25T12:34:18.982809+0000","last_fullsized":"2024-04-25T12:34:18.982809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784989+0000","last_change":"2024-04-25T12:34:13.973917+0000","last_active":"2024-04-25T12:34:19.784989+0000","last_peered":"2024-04-25T12:34:19.784989+0000","last_clean":"2024-04-25T12:34:19.784989+0000","last_became_active":"2024-04-25T12:34:13.973640+0000","last_became_peered":"2024-04-25T12:34:13.973640+0000","last_unstale":"2024-04-25T12:34:19.784989+0000","last_undegraded":"2024-04-25T12:34:19.784989+0000","last_fullsized":"2024-04-25T12:34:19.784989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:21.717728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'12","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.129569+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:34:50.129569+0000","last_peered":"2024-04-25T12:34:50.129569+0000","last_clean":"2024-04-25T12:34:50.129569+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:34:50.129569+0000","last_undegraded":"2024-04-25T12:34:50.129569+0000","last_fullsized":"2024-04-25T12:34:50.129569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783277+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:34:19.783277+0000","last_peered":"2024-04-25T12:34:19.783277+0000","last_clean":"2024-04-25T12:34:19.783277+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:34:19.783277+0000","last_undegraded":"2024-04-25T12:34:19.783277+0000","last_fullsized":"2024-04-25T12:34:19.783277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982241+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:34:18.982241+0000","last_peered":"2024-04-25T12:34:18.982241+0000","last_clean":"2024-04-25T12:34:18.982241+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:34:18.982241+0000","last_undegraded":"2024-04-25T12:34:18.982241+0000","last_fullsized":"2024-04-25T12:34:18.982241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782098+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:34:19.782098+0000","last_peered":"2024-04-25T12:34:19.782098+0000","last_clean":"2024-04-25T12:34:19.782098+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:34:19.782098+0000","last_undegraded":"2024-04-25T12:34:19.782098+0000","last_fullsized":"2024-04-25T12:34:19.782098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788580+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:34:19.788580+0000","last_peered":"2024-04-25T12:34:19.788580+0000","last_clean":"2024-04-25T12:34:19.788580+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:34:19.788580+0000","last_undegraded":"2024-04-25T12:34:19.788580+0000","last_fullsized":"2024-04-25T12:34:19.788580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782777+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:34:19.782777+0000","last_peered":"2024-04-25T12:34:19.782777+0000","last_clean":"2024-04-25T12:34:19.782777+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:34:19.782777+0000","last_undegraded":"2024-04-25T12:34:19.782777+0000","last_fullsized":"2024-04-25T12:34:19.782777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130790+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:34:50.130790+0000","last_peered":"2024-04-25T12:34:50.130790+0000","last_clean":"2024-04-25T12:34:50.130790+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:34:50.130790+0000","last_undegraded":"2024-04-25T12:34:50.130790+0000","last_fullsized":"2024-04-25T12:34:50.130790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780643+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:34:19.780643+0000","last_peered":"2024-04-25T12:34:19.780643+0000","last_clean":"2024-04-25T12:34:19.780643+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:34:19.780643+0000","last_undegraded":"2024-04-25T12:34:19.780643+0000","last_fullsized":"2024-04-25T12:34:19.780643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788514+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:34:19.788514+0000","last_peered":"2024-04-25T12:34:19.788514+0000","last_clean":"2024-04-25T12:34:19.788514+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:34:19.788514+0000","last_undegraded":"2024-04-25T12:34:19.788514+0000","last_fullsized":"2024-04-25T12:34:19.788514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.129489+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:34:50.129489+0000","last_peered":"2024-04-25T12:34:50.129489+0000","last_clean":"2024-04-25T12:34:50.129489+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:34:50.129489+0000","last_undegraded":"2024-04-25T12:34:50.129489+0000","last_fullsized":"2024-04-25T12:34:50.129489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781414+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:34:19.781414+0000","last_peered":"2024-04-25T12:34:19.781414+0000","last_clean":"2024-04-25T12:34:19.781414+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:34:19.781414+0000","last_undegraded":"2024-04-25T12:34:19.781414+0000","last_fullsized":"2024-04-25T12:34:19.781414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782882+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:34:19.782882+0000","last_peered":"2024-04-25T12:34:19.782882+0000","last_clean":"2024-04-25T12:34:19.782882+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:34:19.782882+0000","last_undegraded":"2024-04-25T12:34:19.782882+0000","last_fullsized":"2024-04-25T12:34:19.782882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781481+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:34:19.781481+0000","last_peered":"2024-04-25T12:34:19.781481+0000","last_clean":"2024-04-25T12:34:19.781481+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:34:19.781481+0000","last_undegraded":"2024-04-25T12:34:19.781481+0000","last_fullsized":"2024-04-25T12:34:19.781481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982350+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:34:18.982350+0000","last_peered":"2024-04-25T12:34:18.982350+0000","last_clean":"2024-04-25T12:34:18.982350+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:34:18.982350+0000","last_undegraded":"2024-04-25T12:34:18.982350+0000","last_fullsized":"2024-04-25T12:34:18.982350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983399+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:34:18.983399+0000","last_peered":"2024-04-25T12:34:18.983399+0000","last_clean":"2024-04-25T12:34:18.983399+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:34:18.983399+0000","last_undegraded":"2024-04-25T12:34:18.983399+0000","last_fullsized":"2024-04-25T12:34:18.983399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782668+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:34:19.782668+0000","last_peered":"2024-04-25T12:34:19.782668+0000","last_clean":"2024-04-25T12:34:19.782668+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:34:19.782668+0000","last_undegraded":"2024-04-25T12:34:19.782668+0000","last_fullsized":"2024-04-25T12:34:19.782668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979833+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:34:18.979833+0000","last_peered":"2024-04-25T12:34:18.979833+0000","last_clean":"2024-04-25T12:34:18.979833+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:34:18.979833+0000","last_undegraded":"2024-04-25T12:34:18.979833+0000","last_fullsized":"2024-04-25T12:34:18.979833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982033+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:34:18.982033+0000","last_peered":"2024-04-25T12:34:18.982033+0000","last_clean":"2024-04-25T12:34:18.982033+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:34:18.982033+0000","last_undegraded":"2024-04-25T12:34:18.982033+0000","last_fullsized":"2024-04-25T12:34:18.982033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783137+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:34:19.783137+0000","last_peered":"2024-04-25T12:34:19.783137+0000","last_clean":"2024-04-25T12:34:19.783137+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:34:19.783137+0000","last_undegraded":"2024-04-25T12:34:19.783137+0000","last_fullsized":"2024-04-25T12:34:19.783137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782539+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:34:19.782539+0000","last_peered":"2024-04-25T12:34:19.782539+0000","last_clean":"2024-04-25T12:34:19.782539+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:34:19.782539+0000","last_undegraded":"2024-04-25T12:34:19.782539+0000","last_fullsized":"2024-04-25T12:34:19.782539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.121039+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:34:50.121039+0000","last_peered":"2024-04-25T12:34:50.121039+0000","last_clean":"2024-04-25T12:34:50.121039+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:34:50.121039+0000","last_undegraded":"2024-04-25T12:34:50.121039+0000","last_fullsized":"2024-04-25T12:34:50.121039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789165+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:34:19.789165+0000","last_peered":"2024-04-25T12:34:19.789165+0000","last_clean":"2024-04-25T12:34:19.789165+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:34:19.789165+0000","last_undegraded":"2024-04-25T12:34:19.789165+0000","last_fullsized":"2024-04-25T12:34:19.789165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781753+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:34:19.781753+0000","last_peered":"2024-04-25T12:34:19.781753+0000","last_clean":"2024-04-25T12:34:19.781753+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:34:19.781753+0000","last_undegraded":"2024-04-25T12:34:19.781753+0000","last_fullsized":"2024-04-25T12:34:19.781753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785020+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:34:19.785020+0000","last_peered":"2024-04-25T12:34:19.785020+0000","last_clean":"2024-04-25T12:34:19.785020+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:34:19.785020+0000","last_undegraded":"2024-04-25T12:34:19.785020+0000","last_fullsized":"2024-04-25T12:34:19.785020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781340+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:34:19.781340+0000","last_peered":"2024-04-25T12:34:19.781340+0000","last_clean":"2024-04-25T12:34:19.781340+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:34:19.781340+0000","last_undegraded":"2024-04-25T12:34:19.781340+0000","last_fullsized":"2024-04-25T12:34:19.781340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981734+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:34:18.981734+0000","last_peered":"2024-04-25T12:34:18.981734+0000","last_clean":"2024-04-25T12:34:18.981734+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:34:18.981734+0000","last_undegraded":"2024-04-25T12:34:18.981734+0000","last_fullsized":"2024-04-25T12:34:18.981734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781427+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:34:19.781427+0000","last_peered":"2024-04-25T12:34:19.781427+0000","last_clean":"2024-04-25T12:34:19.781427+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:34:19.781427+0000","last_undegraded":"2024-04-25T12:34:19.781427+0000","last_fullsized":"2024-04-25T12:34:19.781427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980630+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:34:18.980630+0000","last_peered":"2024-04-25T12:34:18.980630+0000","last_clean":"2024-04-25T12:34:18.980630+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:34:18.980630+0000","last_undegraded":"2024-04-25T12:34:18.980630+0000","last_fullsized":"2024-04-25T12:34:18.980630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980186+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:34:18.980186+0000","last_peered":"2024-04-25T12:34:18.980186+0000","last_clean":"2024-04-25T12:34:18.980186+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:34:18.980186+0000","last_undegraded":"2024-04-25T12:34:18.980186+0000","last_fullsized":"2024-04-25T12:34:18.980186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788210+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:34:19.788210+0000","last_peered":"2024-04-25T12:34:19.788210+0000","last_clean":"2024-04-25T12:34:19.788210+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:34:19.788210+0000","last_undegraded":"2024-04-25T12:34:19.788210+0000","last_fullsized":"2024-04-25T12:34:19.788210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782446+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:34:19.782446+0000","last_peered":"2024-04-25T12:34:19.782446+0000","last_clean":"2024-04-25T12:34:19.782446+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:34:19.782446+0000","last_undegraded":"2024-04-25T12:34:19.782446+0000","last_fullsized":"2024-04-25T12:34:19.782446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982148+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:34:18.982148+0000","last_peered":"2024-04-25T12:34:18.982148+0000","last_clean":"2024-04-25T12:34:18.982148+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:34:18.982148+0000","last_undegraded":"2024-04-25T12:34:18.982148+0000","last_fullsized":"2024-04-25T12:34:18.982148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781207+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:34:19.781207+0000","last_peered":"2024-04-25T12:34:19.781207+0000","last_clean":"2024-04-25T12:34:19.781207+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:34:19.781207+0000","last_undegraded":"2024-04-25T12:34:19.781207+0000","last_fullsized":"2024-04-25T12:34:19.781207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789184+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:34:19.789184+0000","last_peered":"2024-04-25T12:34:19.789184+0000","last_clean":"2024-04-25T12:34:19.789184+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:34:19.789184+0000","last_undegraded":"2024-04-25T12:34:19.789184+0000","last_fullsized":"2024-04-25T12:34:19.789184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783090+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:34:19.783090+0000","last_peered":"2024-04-25T12:34:19.783090+0000","last_clean":"2024-04-25T12:34:19.783090+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:34:19.783090+0000","last_undegraded":"2024-04-25T12:34:19.783090+0000","last_fullsized":"2024-04-25T12:34:19.783090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783799+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:34:19.783799+0000","last_peered":"2024-04-25T12:34:19.783799+0000","last_clean":"2024-04-25T12:34:19.783799+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:34:19.783799+0000","last_undegraded":"2024-04-25T12:34:19.783799+0000","last_fullsized":"2024-04-25T12:34:19.783799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983704+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:34:18.983704+0000","last_peered":"2024-04-25T12:34:18.983704+0000","last_clean":"2024-04-25T12:34:18.983704+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:34:18.983704+0000","last_undegraded":"2024-04-25T12:34:18.983704+0000","last_fullsized":"2024-04-25T12:34:18.983704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782288+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:34:19.782288+0000","last_peered":"2024-04-25T12:34:19.782288+0000","last_clean":"2024-04-25T12:34:19.782288+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:34:19.782288+0000","last_undegraded":"2024-04-25T12:34:19.782288+0000","last_fullsized":"2024-04-25T12:34:19.782288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983392+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:34:18.983392+0000","last_peered":"2024-04-25T12:34:18.983392+0000","last_clean":"2024-04-25T12:34:18.983392+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:34:18.983392+0000","last_undegraded":"2024-04-25T12:34:18.983392+0000","last_fullsized":"2024-04-25T12:34:18.983392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981957+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:34:18.981957+0000","last_peered":"2024-04-25T12:34:18.981957+0000","last_clean":"2024-04-25T12:34:18.981957+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:34:18.981957+0000","last_undegraded":"2024-04-25T12:34:18.981957+0000","last_fullsized":"2024-04-25T12:34:18.981957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780879+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:34:19.780879+0000","last_peered":"2024-04-25T12:34:19.780879+0000","last_clean":"2024-04-25T12:34:19.780879+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:34:19.780879+0000","last_undegraded":"2024-04-25T12:34:19.780879+0000","last_fullsized":"2024-04-25T12:34:19.780879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979899+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:34:18.979899+0000","last_peered":"2024-04-25T12:34:18.979899+0000","last_clean":"2024-04-25T12:34:18.979899+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:34:18.979899+0000","last_undegraded":"2024-04-25T12:34:18.979899+0000","last_fullsized":"2024-04-25T12:34:18.979899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980556+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:34:18.980556+0000","last_peered":"2024-04-25T12:34:18.980556+0000","last_clean":"2024-04-25T12:34:18.980556+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:34:18.980556+0000","last_undegraded":"2024-04-25T12:34:18.980556+0000","last_fullsized":"2024-04-25T12:34:18.980556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783694+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:34:19.783694+0000","last_peered":"2024-04-25T12:34:19.783694+0000","last_clean":"2024-04-25T12:34:19.783694+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:34:19.783694+0000","last_undegraded":"2024-04-25T12:34:19.783694+0000","last_fullsized":"2024-04-25T12:34:19.783694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784671+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:34:19.784671+0000","last_peered":"2024-04-25T12:34:19.784671+0000","last_clean":"2024-04-25T12:34:19.784671+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:34:19.784671+0000","last_undegraded":"2024-04-25T12:34:19.784671+0000","last_fullsized":"2024-04-25T12:34:19.784671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784485+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:34:19.784485+0000","last_peered":"2024-04-25T12:34:19.784485+0000","last_clean":"2024-04-25T12:34:19.784485+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:34:19.784485+0000","last_undegraded":"2024-04-25T12:34:19.784485+0000","last_fullsized":"2024-04-25T12:34:19.784485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.791100+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:34:19.791100+0000","last_peered":"2024-04-25T12:34:19.791100+0000","last_clean":"2024-04-25T12:34:19.791100+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:34:19.791100+0000","last_undegraded":"2024-04-25T12:34:19.791100+0000","last_fullsized":"2024-04-25T12:34:19.791100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781305+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:34:19.781305+0000","last_peered":"2024-04-25T12:34:19.781305+0000","last_clean":"2024-04-25T12:34:19.781305+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:34:19.781305+0000","last_undegraded":"2024-04-25T12:34:19.781305+0000","last_fullsized":"2024-04-25T12:34:19.781305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782773+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:34:19.782773+0000","last_peered":"2024-04-25T12:34:19.782773+0000","last_clean":"2024-04-25T12:34:19.782773+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:34:19.782773+0000","last_undegraded":"2024-04-25T12:34:19.782773+0000","last_fullsized":"2024-04-25T12:34:19.782773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786973+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:34:19.786973+0000","last_peered":"2024-04-25T12:34:19.786973+0000","last_clean":"2024-04-25T12:34:19.786973+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:34:19.786973+0000","last_undegraded":"2024-04-25T12:34:19.786973+0000","last_fullsized":"2024-04-25T12:34:19.786973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783422+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:34:19.783422+0000","last_peered":"2024-04-25T12:34:19.783422+0000","last_clean":"2024-04-25T12:34:19.783422+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:34:19.783422+0000","last_undegraded":"2024-04-25T12:34:19.783422+0000","last_fullsized":"2024-04-25T12:34:19.783422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788118+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:34:19.788118+0000","last_peered":"2024-04-25T12:34:19.788118+0000","last_clean":"2024-04-25T12:34:19.788118+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:34:19.788118+0000","last_undegraded":"2024-04-25T12:34:19.788118+0000","last_fullsized":"2024-04-25T12:34:19.788118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782309+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:34:19.782309+0000","last_peered":"2024-04-25T12:34:19.782309+0000","last_clean":"2024-04-25T12:34:19.782309+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:34:19.782309+0000","last_undegraded":"2024-04-25T12:34:19.782309+0000","last_fullsized":"2024-04-25T12:34:19.782309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782643+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:34:19.782643+0000","last_peered":"2024-04-25T12:34:19.782643+0000","last_clean":"2024-04-25T12:34:19.782643+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:34:19.782643+0000","last_undegraded":"2024-04-25T12:34:19.782643+0000","last_fullsized":"2024-04-25T12:34:19.782643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785344+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:34:19.785344+0000","last_peered":"2024-04-25T12:34:19.785344+0000","last_clean":"2024-04-25T12:34:19.785344+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:34:19.785344+0000","last_undegraded":"2024-04-25T12:34:19.785344+0000","last_fullsized":"2024-04-25T12:34:19.785344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6046,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":103079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":103639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":694015,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":108231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":331776,"data_stored":693461,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:34:55.286 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:34:55.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.574+0000 7f773f000640 1 Processor -- start 2024-04-25T12:34:55.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.574+0000 7f773f000640 1 -- start start 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.576+0000 7f773f000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f773815d170 0x7f773815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.576+0000 7f773f000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f77380ccf10 0x7f77380cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.576+0000 7f773f000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f773815e270 con 0x7f77380d1d80 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.576+0000 7f773f000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f77380d2760 con 0x7f77380ccf10 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.576+0000 7f773f000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f77380ceb90 con 0x7f773815d170 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773d200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f773815d170 0x7f773815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773dc00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f77380d1d80 0x7f773815db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:40904/0 (socket says 172.21.15.123:40904) 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773e600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f77380ccf10 0x7f77380cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773dc00640 1 -- 172.21.15.123:0/3027372736 learned_addr learned my addr 172.21.15.123:0/3027372736 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:55.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773d200640 1 -- 172.21.15.123:0/3027372736 >> v2:172.21.15.153:3300/0 conn(0x7f77380ccf10 msgr2=0x7f77380cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:55.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773d200640 1 --2- 172.21.15.123:0/3027372736 >> v2:172.21.15.153:3300/0 conn(0x7f77380ccf10 0x7f77380cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f7737e00640 1 -- 172.21.15.123:0/3027372736 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880539004 0 0) 0x7f773815e270 con 0x7f77380d1d80 2024-04-25T12:34:55.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773d200640 1 -- 172.21.15.123:0/3027372736 >> v1:172.21.15.153:6789/0 conn(0x7f77380d1d80 legacy=0x7f773815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.577+0000 7f773d200640 1 -- 172.21.15.123:0/3027372736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77380d2480 con 0x7f773815d170 2024-04-25T12:34:55.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.578+0000 7f773d200640 1 --2- 172.21.15.123:0/3027372736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f773815d170 0x7f773815c060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77240097a0 tx=0x7f772402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b56c306cce35fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:55.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.578+0000 7f7737e00640 1 -- 172.21.15.123:0/3027372736 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724037070 con 0x7f773815d170 2024-04-25T12:34:55.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f7737e00640 1 -- 172.21.15.123:0/3027372736 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7724032440 con 0x7f773815d170 2024-04-25T12:34:55.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f7737e00640 1 -- 172.21.15.123:0/3027372736 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724032810 con 0x7f773815d170 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 -- 172.21.15.123:0/3027372736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f773815d170 msgr2=0x7f773815c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 --2- 172.21.15.123:0/3027372736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f773815d170 0x7f773815c060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77240097a0 tx=0x7f772402a210 comp rx=0 tx=0).stop 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 -- 172.21.15.123:0/3027372736 shutdown_connections 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 --2- 172.21.15.123:0/3027372736 >> v2:172.21.15.153:3300/0 conn(0x7f77380ccf10 0x7f77380cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 --2- 172.21.15.123:0/3027372736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f773815d170 0x7f773815c060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 -- 172.21.15.123:0/3027372736 >> 172.21.15.123:0/3027372736 conn(0x7f773808da20 msgr2=0x7f773808de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 -- 172.21.15.123:0/3027372736 shutdown_connections 2024-04-25T12:34:55.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.579+0000 7f773f000640 1 -- 172.21.15.123:0/3027372736 wait complete. 2024-04-25T12:34:55.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.582+0000 7f773f000640 1 Processor -- start 2024-04-25T12:34:55.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.582+0000 7f773f000640 1 -- start start 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773f000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77380ccf10 0x7f77381f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77380d1d80 0x7f77381e3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773f000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f773815d170 0x7f77380d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f77380ce540 con 0x7f77380d1d80 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773f000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f773815c5a0 con 0x7f773815d170 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773f000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f77380ce820 con 0x7f77380ccf10 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77380ccf10 0x7f77381f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77380d1d80 0x7f77381e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773e600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f773815d170 0x7f77380d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77380ccf10 0x7f77381f2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38300/0 (socket says 172.21.15.123:38300) 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773dc00640 1 -- 172.21.15.123:0/2171674532 learned_addr learned my addr 172.21.15.123:0/2171674532 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:55.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.583+0000 7f773d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77380d1d80 0x7f77381e3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39574/0 (socket says 172.21.15.123:39574) 2024-04-25T12:34:55.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773dc00640 1 -- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f773815d170 msgr2=0x7f77380d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773dc00640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f773815d170 0x7f77380d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773dc00640 1 -- 172.21.15.123:0/2171674532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77380d1d80 msgr2=0x7f77381e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773dc00640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77380d1d80 0x7f77381e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773dc00640 1 -- 172.21.15.123:0/2171674532 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77381e3eb0 con 0x7f77380ccf10 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773d200640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77380d1d80 0x7f77381e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773e600640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f773815d170 0x7f77380d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.584+0000 7f773dc00640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77380ccf10 0x7f77381f2260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c430 tx=0x7f772c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.585+0000 7f7735600640 1 -- 172.21.15.123:0/2171674532 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c0074e0 con 0x7f77380ccf10 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.585+0000 7f7735600640 1 -- 172.21.15.123:0/2171674532 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c0047e0 con 0x7f77380ccf10 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.585+0000 7f7735600640 1 -- 172.21.15.123:0/2171674532 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c004bd0 con 0x7f77380ccf10 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.585+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773815b570 con 0x7f77380ccf10 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.585+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77381f2e60 con 0x7f77380ccf10 2024-04-25T12:34:55.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.586+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77381e4080 con 0x7f77380ccf10 2024-04-25T12:34:55.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.587+0000 7f7735600640 1 -- 172.21.15.123:0/2171674532 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f772c011070 con 0x7f77380ccf10 2024-04-25T12:34:55.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.588+0000 7f7735600640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f77180744b0 0x7f77180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:55.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.588+0000 7f7735600640 1 -- 172.21.15.123:0/2171674532 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f772c095100 con 0x7f77380ccf10 2024-04-25T12:34:55.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.588+0000 7f773d200640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f77180744b0 0x7f77180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:55.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.589+0000 7f773d200640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f77180744b0 0x7f77180769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7724034000 tx=0x7f772402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.592+0000 7f7735600640 1 -- 172.21.15.123:0/2171674532 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f772c05eee0 con 0x7f77380ccf10 2024-04-25T12:34:55.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.778+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7738076da0 con 0x7f77180744b0 2024-04-25T12:34:55.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.788+0000 7f7735600640 1 -- 172.21.15.123:0/2171674532 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332640 (secure 0 0 0) 0x7f7738076da0 con 0x7f77180744b0 2024-04-25T12:34:55.792 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:55.794 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.793+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f77180744b0 msgr2=0x7f77180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.793+0000 7f773f000640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f77180744b0 0x7f77180769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7724034000 tx=0x7f772402e000 comp rx=0 tx=0).stop 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.793+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77380ccf10 msgr2=0x7f77381f2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.793+0000 7f773f000640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77380ccf10 0x7f77381f2260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c430 tx=0x7f772c00c8f0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 shutdown_connections 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f77180744b0 0x7f77180769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f773815d170 0x7f77380d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77380d1d80 0x7f77381e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 --2- 172.21.15.123:0/2171674532 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f77380ccf10 0x7f77381f2260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 >> 172.21.15.123:0/2171674532 conn(0x7f773808da20 msgr2=0x7f773808de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:55.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 shutdown_connections 2024-04-25T12:34:55.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:55.794+0000 7f773f000640 1 -- 172.21.15.123:0/2171674532 wait complete. 2024-04-25T12:34:55.818 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-25T12:34:54.802406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596414,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":189,"num_write_kb":1659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218384,"kb_used_data":2256,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756336,"statfs":{"total":773094113280,"available":772870488064,"internally_reserved":0,"allocated":2310144,"data_stored":2016292,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003991"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783414+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:34:19.783414+0000","last_peered":"2024-04-25T12:34:19.783414+0000","last_clean":"2024-04-25T12:34:19.783414+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:34:19.783414+0000","last_undegraded":"2024-04-25T12:34:19.783414+0000","last_fullsized":"2024-04-25T12:34:19.783414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782890+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:34:19.782890+0000","last_peered":"2024-04-25T12:34:19.782890+0000","last_clean":"2024-04-25T12:34:19.782890+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:34:19.782890+0000","last_undegraded":"2024-04-25T12:34:19.782890+0000","last_fullsized":"2024-04-25T12:34:19.782890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780793+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:34:19.780793+0000","last_peered":"2024-04-25T12:34:19.780793+0000","last_clean":"2024-04-25T12:34:19.780793+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:34:19.780793+0000","last_undegraded":"2024-04-25T12:34:19.780793+0000","last_fullsized":"2024-04-25T12:34:19.780793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782997+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:34:19.782997+0000","last_peered":"2024-04-25T12:34:19.782997+0000","last_clean":"2024-04-25T12:34:19.782997+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:34:19.782997+0000","last_undegraded":"2024-04-25T12:34:19.782997+0000","last_fullsized":"2024-04-25T12:34:19.782997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782606+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:34:19.782606+0000","last_peered":"2024-04-25T12:34:19.782606+0000","last_clean":"2024-04-25T12:34:19.782606+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:34:19.782606+0000","last_undegraded":"2024-04-25T12:34:19.782606+0000","last_fullsized":"2024-04-25T12:34:19.782606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784500+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:34:19.784500+0000","last_peered":"2024-04-25T12:34:19.784500+0000","last_clean":"2024-04-25T12:34:19.784500+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:34:19.784500+0000","last_undegraded":"2024-04-25T12:34:19.784500+0000","last_fullsized":"2024-04-25T12:34:19.784500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785563+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:34:19.785563+0000","last_peered":"2024-04-25T12:34:19.785563+0000","last_clean":"2024-04-25T12:34:19.785563+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:34:19.785563+0000","last_undegraded":"2024-04-25T12:34:19.785563+0000","last_fullsized":"2024-04-25T12:34:19.785563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.790406+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:34:19.790406+0000","last_peered":"2024-04-25T12:34:19.790406+0000","last_clean":"2024-04-25T12:34:19.790406+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:34:19.790406+0000","last_undegraded":"2024-04-25T12:34:19.790406+0000","last_fullsized":"2024-04-25T12:34:19.790406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788036+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:34:19.788036+0000","last_peered":"2024-04-25T12:34:19.788036+0000","last_clean":"2024-04-25T12:34:19.788036+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:34:19.788036+0000","last_undegraded":"2024-04-25T12:34:19.788036+0000","last_fullsized":"2024-04-25T12:34:19.788036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784075+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:34:19.784075+0000","last_peered":"2024-04-25T12:34:19.784075+0000","last_clean":"2024-04-25T12:34:19.784075+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:34:19.784075+0000","last_undegraded":"2024-04-25T12:34:19.784075+0000","last_fullsized":"2024-04-25T12:34:19.784075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982844+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:34:18.982844+0000","last_peered":"2024-04-25T12:34:18.982844+0000","last_clean":"2024-04-25T12:34:18.982844+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:34:18.982844+0000","last_undegraded":"2024-04-25T12:34:18.982844+0000","last_fullsized":"2024-04-25T12:34:18.982844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786691+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:34:19.786691+0000","last_peered":"2024-04-25T12:34:19.786691+0000","last_clean":"2024-04-25T12:34:19.786691+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:34:19.786691+0000","last_undegraded":"2024-04-25T12:34:19.786691+0000","last_fullsized":"2024-04-25T12:34:19.786691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982590+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:34:18.982590+0000","last_peered":"2024-04-25T12:34:18.982590+0000","last_clean":"2024-04-25T12:34:18.982590+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:34:18.982590+0000","last_undegraded":"2024-04-25T12:34:18.982590+0000","last_fullsized":"2024-04-25T12:34:18.982590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130152+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:34:50.130152+0000","last_peered":"2024-04-25T12:34:50.130152+0000","last_clean":"2024-04-25T12:34:50.130152+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:34:50.130152+0000","last_undegraded":"2024-04-25T12:34:50.130152+0000","last_fullsized":"2024-04-25T12:34:50.130152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982528+0000","last_change":"2024-04-25T12:34:18.007956+0000","last_active":"2024-04-25T12:34:18.982528+0000","last_peered":"2024-04-25T12:34:18.982528+0000","last_clean":"2024-04-25T12:34:18.982528+0000","last_became_active":"2024-04-25T12:34:18.007500+0000","last_became_peered":"2024-04-25T12:34:18.007500+0000","last_unstale":"2024-04-25T12:34:18.982528+0000","last_undegraded":"2024-04-25T12:34:18.982528+0000","last_fullsized":"2024-04-25T12:34:18.982528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:56.400953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782265+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:34:19.782265+0000","last_peered":"2024-04-25T12:34:19.782265+0000","last_clean":"2024-04-25T12:34:19.782265+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:34:19.782265+0000","last_undegraded":"2024-04-25T12:34:19.782265+0000","last_fullsized":"2024-04-25T12:34:19.782265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980152+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:34:18.980152+0000","last_peered":"2024-04-25T12:34:18.980152+0000","last_clean":"2024-04-25T12:34:18.980152+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:34:18.980152+0000","last_undegraded":"2024-04-25T12:34:18.980152+0000","last_fullsized":"2024-04-25T12:34:18.980152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783204+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:34:19.783204+0000","last_peered":"2024-04-25T12:34:19.783204+0000","last_clean":"2024-04-25T12:34:19.783204+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:34:19.783204+0000","last_undegraded":"2024-04-25T12:34:19.783204+0000","last_fullsized":"2024-04-25T12:34:19.783204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782958+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:34:19.782958+0000","last_peered":"2024-04-25T12:34:19.782958+0000","last_clean":"2024-04-25T12:34:19.782958+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:34:19.782958+0000","last_undegraded":"2024-04-25T12:34:19.782958+0000","last_fullsized":"2024-04-25T12:34:19.782958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130938+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:34:50.130938+0000","last_peered":"2024-04-25T12:34:50.130938+0000","last_clean":"2024-04-25T12:34:50.130938+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:34:50.130938+0000","last_undegraded":"2024-04-25T12:34:50.130938+0000","last_fullsized":"2024-04-25T12:34:50.130938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781948+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:34:19.781948+0000","last_peered":"2024-04-25T12:34:19.781948+0000","last_clean":"2024-04-25T12:34:19.781948+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:34:19.781948+0000","last_undegraded":"2024-04-25T12:34:19.781948+0000","last_fullsized":"2024-04-25T12:34:19.781948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981753+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:34:18.981753+0000","last_peered":"2024-04-25T12:34:18.981753+0000","last_clean":"2024-04-25T12:34:18.981753+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:34:18.981753+0000","last_undegraded":"2024-04-25T12:34:18.981753+0000","last_fullsized":"2024-04-25T12:34:18.981753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.131168+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:34:50.131168+0000","last_peered":"2024-04-25T12:34:50.131168+0000","last_clean":"2024-04-25T12:34:50.131168+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:34:50.131168+0000","last_undegraded":"2024-04-25T12:34:50.131168+0000","last_fullsized":"2024-04-25T12:34:50.131168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788041+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:34:19.788041+0000","last_peered":"2024-04-25T12:34:19.788041+0000","last_clean":"2024-04-25T12:34:19.788041+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:34:19.788041+0000","last_undegraded":"2024-04-25T12:34:19.788041+0000","last_fullsized":"2024-04-25T12:34:19.788041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783014+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:34:19.783014+0000","last_peered":"2024-04-25T12:34:19.783014+0000","last_clean":"2024-04-25T12:34:19.783014+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:34:19.783014+0000","last_undegraded":"2024-04-25T12:34:19.783014+0000","last_fullsized":"2024-04-25T12:34:19.783014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.131059+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:34:50.131059+0000","last_peered":"2024-04-25T12:34:50.131059+0000","last_clean":"2024-04-25T12:34:50.131059+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:34:50.131059+0000","last_undegraded":"2024-04-25T12:34:50.131059+0000","last_fullsized":"2024-04-25T12:34:50.131059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787014+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:34:19.787014+0000","last_peered":"2024-04-25T12:34:19.787014+0000","last_clean":"2024-04-25T12:34:19.787014+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:34:19.787014+0000","last_undegraded":"2024-04-25T12:34:19.787014+0000","last_fullsized":"2024-04-25T12:34:19.787014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783329+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:34:19.783329+0000","last_peered":"2024-04-25T12:34:19.783329+0000","last_clean":"2024-04-25T12:34:19.783329+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:34:19.783329+0000","last_undegraded":"2024-04-25T12:34:19.783329+0000","last_fullsized":"2024-04-25T12:34:19.783329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783247+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:34:19.783247+0000","last_peered":"2024-04-25T12:34:19.783247+0000","last_clean":"2024-04-25T12:34:19.783247+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:34:19.783247+0000","last_undegraded":"2024-04-25T12:34:19.783247+0000","last_fullsized":"2024-04-25T12:34:19.783247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781103+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:34:19.781103+0000","last_peered":"2024-04-25T12:34:19.781103+0000","last_clean":"2024-04-25T12:34:19.781103+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:34:19.781103+0000","last_undegraded":"2024-04-25T12:34:19.781103+0000","last_fullsized":"2024-04-25T12:34:19.781103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980759+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:34:18.980759+0000","last_peered":"2024-04-25T12:34:18.980759+0000","last_clean":"2024-04-25T12:34:18.980759+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:34:18.980759+0000","last_undegraded":"2024-04-25T12:34:18.980759+0000","last_fullsized":"2024-04-25T12:34:18.980759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983904+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:34:18.983904+0000","last_peered":"2024-04-25T12:34:18.983904+0000","last_clean":"2024-04-25T12:34:18.983904+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:34:18.983904+0000","last_undegraded":"2024-04-25T12:34:18.983904+0000","last_fullsized":"2024-04-25T12:34:18.983904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980463+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:34:18.980463+0000","last_peered":"2024-04-25T12:34:18.980463+0000","last_clean":"2024-04-25T12:34:18.980463+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:34:18.980463+0000","last_undegraded":"2024-04-25T12:34:18.980463+0000","last_fullsized":"2024-04-25T12:34:18.980463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789572+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:34:19.789572+0000","last_peered":"2024-04-25T12:34:19.789572+0000","last_clean":"2024-04-25T12:34:19.789572+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:34:19.789572+0000","last_undegraded":"2024-04-25T12:34:19.789572+0000","last_fullsized":"2024-04-25T12:34:19.789572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782068+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:34:19.782068+0000","last_peered":"2024-04-25T12:34:19.782068+0000","last_clean":"2024-04-25T12:34:19.782068+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:34:19.782068+0000","last_undegraded":"2024-04-25T12:34:19.782068+0000","last_fullsized":"2024-04-25T12:34:19.782068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979987+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:34:18.979987+0000","last_peered":"2024-04-25T12:34:18.979987+0000","last_clean":"2024-04-25T12:34:18.979987+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:34:18.979987+0000","last_undegraded":"2024-04-25T12:34:18.979987+0000","last_fullsized":"2024-04-25T12:34:18.979987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783310+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:34:19.783310+0000","last_peered":"2024-04-25T12:34:19.783310+0000","last_clean":"2024-04-25T12:34:19.783310+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:34:19.783310+0000","last_undegraded":"2024-04-25T12:34:19.783310+0000","last_fullsized":"2024-04-25T12:34:19.783310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982842+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:34:18.982842+0000","last_peered":"2024-04-25T12:34:18.982842+0000","last_clean":"2024-04-25T12:34:18.982842+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:34:18.982842+0000","last_undegraded":"2024-04-25T12:34:18.982842+0000","last_fullsized":"2024-04-25T12:34:18.982842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980473+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:34:18.980473+0000","last_peered":"2024-04-25T12:34:18.980473+0000","last_clean":"2024-04-25T12:34:18.980473+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:34:18.980473+0000","last_undegraded":"2024-04-25T12:34:18.980473+0000","last_fullsized":"2024-04-25T12:34:18.980473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781093+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:34:19.781093+0000","last_peered":"2024-04-25T12:34:19.781093+0000","last_clean":"2024-04-25T12:34:19.781093+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:34:19.781093+0000","last_undegraded":"2024-04-25T12:34:19.781093+0000","last_fullsized":"2024-04-25T12:34:19.781093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781512+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:34:19.781512+0000","last_peered":"2024-04-25T12:34:19.781512+0000","last_clean":"2024-04-25T12:34:19.781512+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:34:19.781512+0000","last_undegraded":"2024-04-25T12:34:19.781512+0000","last_fullsized":"2024-04-25T12:34:19.781512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980445+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:34:18.980445+0000","last_peered":"2024-04-25T12:34:18.980445+0000","last_clean":"2024-04-25T12:34:18.980445+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:34:18.980445+0000","last_undegraded":"2024-04-25T12:34:18.980445+0000","last_fullsized":"2024-04-25T12:34:18.980445+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979799+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:34:18.979799+0000","last_peered":"2024-04-25T12:34:18.979799+0000","last_clean":"2024-04-25T12:34:18.979799+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:34:18.979799+0000","last_undegraded":"2024-04-25T12:34:18.979799+0000","last_fullsized":"2024-04-25T12:34:18.979799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982530+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:34:18.982530+0000","last_peered":"2024-04-25T12:34:18.982530+0000","last_clean":"2024-04-25T12:34:18.982530+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:34:18.982530+0000","last_undegraded":"2024-04-25T12:34:18.982530+0000","last_fullsized":"2024-04-25T12:34:18.982530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782825+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:34:19.782825+0000","last_peered":"2024-04-25T12:34:19.782825+0000","last_clean":"2024-04-25T12:34:19.782825+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:34:19.782825+0000","last_undegraded":"2024-04-25T12:34:19.782825+0000","last_fullsized":"2024-04-25T12:34:19.782825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783355+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:34:19.783355+0000","last_peered":"2024-04-25T12:34:19.783355+0000","last_clean":"2024-04-25T12:34:19.783355+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:34:19.783355+0000","last_undegraded":"2024-04-25T12:34:19.783355+0000","last_fullsized":"2024-04-25T12:34:19.783355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130560+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:34:50.130560+0000","last_peered":"2024-04-25T12:34:50.130560+0000","last_clean":"2024-04-25T12:34:50.130560+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:34:50.130560+0000","last_undegraded":"2024-04-25T12:34:50.130560+0000","last_fullsized":"2024-04-25T12:34:50.130560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.787419+0000","last_change":"2024-04-25T12:34:08.560989+0000","last_active":"2024-04-25T12:34:19.787419+0000","last_peered":"2024-04-25T12:34:19.787419+0000","last_clean":"2024-04-25T12:34:19.787419+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:34:19.787419+0000","last_undegraded":"2024-04-25T12:34:19.787419+0000","last_fullsized":"2024-04-25T12:34:19.787419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:49.130809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983747+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:34:18.983747+0000","last_peered":"2024-04-25T12:34:18.983747+0000","last_clean":"2024-04-25T12:34:18.983747+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:34:18.983747+0000","last_undegraded":"2024-04-25T12:34:18.983747+0000","last_fullsized":"2024-04-25T12:34:18.983747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982942+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:34:18.982942+0000","last_peered":"2024-04-25T12:34:18.982942+0000","last_clean":"2024-04-25T12:34:18.982942+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:34:18.982942+0000","last_undegraded":"2024-04-25T12:34:18.982942+0000","last_fullsized":"2024-04-25T12:34:18.982942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980766+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:34:18.980766+0000","last_peered":"2024-04-25T12:34:18.980766+0000","last_clean":"2024-04-25T12:34:18.980766+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:34:18.980766+0000","last_undegraded":"2024-04-25T12:34:18.980766+0000","last_fullsized":"2024-04-25T12:34:18.980766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786888+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:34:19.786888+0000","last_peered":"2024-04-25T12:34:19.786888+0000","last_clean":"2024-04-25T12:34:19.786888+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:34:19.786888+0000","last_undegraded":"2024-04-25T12:34:19.786888+0000","last_fullsized":"2024-04-25T12:34:19.786888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783906+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:34:19.783906+0000","last_peered":"2024-04-25T12:34:19.783906+0000","last_clean":"2024-04-25T12:34:19.783906+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:34:19.783906+0000","last_undegraded":"2024-04-25T12:34:19.783906+0000","last_fullsized":"2024-04-25T12:34:19.783906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782493+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:34:19.782493+0000","last_peered":"2024-04-25T12:34:19.782493+0000","last_clean":"2024-04-25T12:34:19.782493+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:34:19.782493+0000","last_undegraded":"2024-04-25T12:34:19.782493+0000","last_fullsized":"2024-04-25T12:34:19.782493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781393+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:34:19.781393+0000","last_peered":"2024-04-25T12:34:19.781393+0000","last_clean":"2024-04-25T12:34:19.781393+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:34:19.781393+0000","last_undegraded":"2024-04-25T12:34:19.781393+0000","last_fullsized":"2024-04-25T12:34:19.781393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785529+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:34:19.785529+0000","last_peered":"2024-04-25T12:34:19.785529+0000","last_clean":"2024-04-25T12:34:19.785529+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:34:19.785529+0000","last_undegraded":"2024-04-25T12:34:19.785529+0000","last_fullsized":"2024-04-25T12:34:19.785529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783018+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:34:19.783018+0000","last_peered":"2024-04-25T12:34:19.783018+0000","last_clean":"2024-04-25T12:34:19.783018+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:34:19.783018+0000","last_undegraded":"2024-04-25T12:34:19.783018+0000","last_fullsized":"2024-04-25T12:34:19.783018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981441+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:34:18.981441+0000","last_peered":"2024-04-25T12:34:18.981441+0000","last_clean":"2024-04-25T12:34:18.981441+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:34:18.981441+0000","last_undegraded":"2024-04-25T12:34:18.981441+0000","last_fullsized":"2024-04-25T12:34:18.981441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788784+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:34:19.788784+0000","last_peered":"2024-04-25T12:34:19.788784+0000","last_clean":"2024-04-25T12:34:19.788784+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:34:19.788784+0000","last_undegraded":"2024-04-25T12:34:19.788784+0000","last_fullsized":"2024-04-25T12:34:19.788784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785856+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:34:19.785856+0000","last_peered":"2024-04-25T12:34:19.785856+0000","last_clean":"2024-04-25T12:34:19.785856+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:34:19.785856+0000","last_undegraded":"2024-04-25T12:34:19.785856+0000","last_fullsized":"2024-04-25T12:34:19.785856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788439+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:34:19.788439+0000","last_peered":"2024-04-25T12:34:19.788439+0000","last_clean":"2024-04-25T12:34:19.788439+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:34:19.788439+0000","last_undegraded":"2024-04-25T12:34:19.788439+0000","last_fullsized":"2024-04-25T12:34:19.788439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981709+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:34:18.981709+0000","last_peered":"2024-04-25T12:34:18.981709+0000","last_clean":"2024-04-25T12:34:18.981709+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:34:18.981709+0000","last_undegraded":"2024-04-25T12:34:18.981709+0000","last_fullsized":"2024-04-25T12:34:18.981709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.119769+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:34:50.119769+0000","last_peered":"2024-04-25T12:34:50.119769+0000","last_clean":"2024-04-25T12:34:50.119769+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:34:50.119769+0000","last_undegraded":"2024-04-25T12:34:50.119769+0000","last_fullsized":"2024-04-25T12:34:50.119769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782909+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:34:19.782909+0000","last_peered":"2024-04-25T12:34:19.782909+0000","last_clean":"2024-04-25T12:34:19.782909+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:34:19.782909+0000","last_undegraded":"2024-04-25T12:34:19.782909+0000","last_fullsized":"2024-04-25T12:34:19.782909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786692+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:34:19.786692+0000","last_peered":"2024-04-25T12:34:19.786692+0000","last_clean":"2024-04-25T12:34:19.786692+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:34:19.786692+0000","last_undegraded":"2024-04-25T12:34:19.786692+0000","last_fullsized":"2024-04-25T12:34:19.786692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782692+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:34:19.782692+0000","last_peered":"2024-04-25T12:34:19.782692+0000","last_clean":"2024-04-25T12:34:19.782692+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:34:19.782692+0000","last_undegraded":"2024-04-25T12:34:19.782692+0000","last_fullsized":"2024-04-25T12:34:19.782692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782809+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:34:19.782809+0000","last_peered":"2024-04-25T12:34:19.782809+0000","last_clean":"2024-04-25T12:34:19.782809+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:34:19.782809+0000","last_undegraded":"2024-04-25T12:34:19.782809+0000","last_fullsized":"2024-04-25T12:34:19.782809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983105+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:34:18.983105+0000","last_peered":"2024-04-25T12:34:18.983105+0000","last_clean":"2024-04-25T12:34:18.983105+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:34:18.983105+0000","last_undegraded":"2024-04-25T12:34:18.983105+0000","last_fullsized":"2024-04-25T12:34:18.983105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.128291+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:34:50.128291+0000","last_peered":"2024-04-25T12:34:50.128291+0000","last_clean":"2024-04-25T12:34:50.128291+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:34:50.128291+0000","last_undegraded":"2024-04-25T12:34:50.128291+0000","last_fullsized":"2024-04-25T12:34:50.128291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980674+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:34:18.980674+0000","last_peered":"2024-04-25T12:34:18.980674+0000","last_clean":"2024-04-25T12:34:18.980674+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:34:18.980674+0000","last_undegraded":"2024-04-25T12:34:18.980674+0000","last_fullsized":"2024-04-25T12:34:18.980674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980038+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:34:18.980038+0000","last_peered":"2024-04-25T12:34:18.980038+0000","last_clean":"2024-04-25T12:34:18.980038+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:34:18.980038+0000","last_undegraded":"2024-04-25T12:34:18.980038+0000","last_fullsized":"2024-04-25T12:34:18.980038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.120594+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:34:50.120594+0000","last_peered":"2024-04-25T12:34:50.120594+0000","last_clean":"2024-04-25T12:34:50.120594+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:34:50.120594+0000","last_undegraded":"2024-04-25T12:34:50.120594+0000","last_fullsized":"2024-04-25T12:34:50.120594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982809+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:34:18.982809+0000","last_peered":"2024-04-25T12:34:18.982809+0000","last_clean":"2024-04-25T12:34:18.982809+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:34:18.982809+0000","last_undegraded":"2024-04-25T12:34:18.982809+0000","last_fullsized":"2024-04-25T12:34:18.982809+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784989+0000","last_change":"2024-04-25T12:34:13.973917+0000","last_active":"2024-04-25T12:34:19.784989+0000","last_peered":"2024-04-25T12:34:19.784989+0000","last_clean":"2024-04-25T12:34:19.784989+0000","last_became_active":"2024-04-25T12:34:13.973640+0000","last_became_peered":"2024-04-25T12:34:13.973640+0000","last_unstale":"2024-04-25T12:34:19.784989+0000","last_undegraded":"2024-04-25T12:34:19.784989+0000","last_fullsized":"2024-04-25T12:34:19.784989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:21.717728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'12","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.129569+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:34:50.129569+0000","last_peered":"2024-04-25T12:34:50.129569+0000","last_clean":"2024-04-25T12:34:50.129569+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:34:50.129569+0000","last_undegraded":"2024-04-25T12:34:50.129569+0000","last_fullsized":"2024-04-25T12:34:50.129569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783277+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:34:19.783277+0000","last_peered":"2024-04-25T12:34:19.783277+0000","last_clean":"2024-04-25T12:34:19.783277+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:34:19.783277+0000","last_undegraded":"2024-04-25T12:34:19.783277+0000","last_fullsized":"2024-04-25T12:34:19.783277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982241+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:34:18.982241+0000","last_peered":"2024-04-25T12:34:18.982241+0000","last_clean":"2024-04-25T12:34:18.982241+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:34:18.982241+0000","last_undegraded":"2024-04-25T12:34:18.982241+0000","last_fullsized":"2024-04-25T12:34:18.982241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782098+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:34:19.782098+0000","last_peered":"2024-04-25T12:34:19.782098+0000","last_clean":"2024-04-25T12:34:19.782098+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:34:19.782098+0000","last_undegraded":"2024-04-25T12:34:19.782098+0000","last_fullsized":"2024-04-25T12:34:19.782098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788580+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:34:19.788580+0000","last_peered":"2024-04-25T12:34:19.788580+0000","last_clean":"2024-04-25T12:34:19.788580+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:34:19.788580+0000","last_undegraded":"2024-04-25T12:34:19.788580+0000","last_fullsized":"2024-04-25T12:34:19.788580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782777+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:34:19.782777+0000","last_peered":"2024-04-25T12:34:19.782777+0000","last_clean":"2024-04-25T12:34:19.782777+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:34:19.782777+0000","last_undegraded":"2024-04-25T12:34:19.782777+0000","last_fullsized":"2024-04-25T12:34:19.782777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.130790+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:34:50.130790+0000","last_peered":"2024-04-25T12:34:50.130790+0000","last_clean":"2024-04-25T12:34:50.130790+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:34:50.130790+0000","last_undegraded":"2024-04-25T12:34:50.130790+0000","last_fullsized":"2024-04-25T12:34:50.130790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780643+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:34:19.780643+0000","last_peered":"2024-04-25T12:34:19.780643+0000","last_clean":"2024-04-25T12:34:19.780643+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:34:19.780643+0000","last_undegraded":"2024-04-25T12:34:19.780643+0000","last_fullsized":"2024-04-25T12:34:19.780643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788514+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:34:19.788514+0000","last_peered":"2024-04-25T12:34:19.788514+0000","last_clean":"2024-04-25T12:34:19.788514+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:34:19.788514+0000","last_undegraded":"2024-04-25T12:34:19.788514+0000","last_fullsized":"2024-04-25T12:34:19.788514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.129489+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:34:50.129489+0000","last_peered":"2024-04-25T12:34:50.129489+0000","last_clean":"2024-04-25T12:34:50.129489+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:34:50.129489+0000","last_undegraded":"2024-04-25T12:34:50.129489+0000","last_fullsized":"2024-04-25T12:34:50.129489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781414+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:34:19.781414+0000","last_peered":"2024-04-25T12:34:19.781414+0000","last_clean":"2024-04-25T12:34:19.781414+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:34:19.781414+0000","last_undegraded":"2024-04-25T12:34:19.781414+0000","last_fullsized":"2024-04-25T12:34:19.781414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782882+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:34:19.782882+0000","last_peered":"2024-04-25T12:34:19.782882+0000","last_clean":"2024-04-25T12:34:19.782882+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:34:19.782882+0000","last_undegraded":"2024-04-25T12:34:19.782882+0000","last_fullsized":"2024-04-25T12:34:19.782882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781481+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:34:19.781481+0000","last_peered":"2024-04-25T12:34:19.781481+0000","last_clean":"2024-04-25T12:34:19.781481+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:34:19.781481+0000","last_undegraded":"2024-04-25T12:34:19.781481+0000","last_fullsized":"2024-04-25T12:34:19.781481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982350+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:34:18.982350+0000","last_peered":"2024-04-25T12:34:18.982350+0000","last_clean":"2024-04-25T12:34:18.982350+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:34:18.982350+0000","last_undegraded":"2024-04-25T12:34:18.982350+0000","last_fullsized":"2024-04-25T12:34:18.982350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983399+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:34:18.983399+0000","last_peered":"2024-04-25T12:34:18.983399+0000","last_clean":"2024-04-25T12:34:18.983399+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:34:18.983399+0000","last_undegraded":"2024-04-25T12:34:18.983399+0000","last_fullsized":"2024-04-25T12:34:18.983399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782668+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:34:19.782668+0000","last_peered":"2024-04-25T12:34:19.782668+0000","last_clean":"2024-04-25T12:34:19.782668+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:34:19.782668+0000","last_undegraded":"2024-04-25T12:34:19.782668+0000","last_fullsized":"2024-04-25T12:34:19.782668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979833+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:34:18.979833+0000","last_peered":"2024-04-25T12:34:18.979833+0000","last_clean":"2024-04-25T12:34:18.979833+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:34:18.979833+0000","last_undegraded":"2024-04-25T12:34:18.979833+0000","last_fullsized":"2024-04-25T12:34:18.979833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982033+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:34:18.982033+0000","last_peered":"2024-04-25T12:34:18.982033+0000","last_clean":"2024-04-25T12:34:18.982033+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:34:18.982033+0000","last_undegraded":"2024-04-25T12:34:18.982033+0000","last_fullsized":"2024-04-25T12:34:18.982033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783137+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:34:19.783137+0000","last_peered":"2024-04-25T12:34:19.783137+0000","last_clean":"2024-04-25T12:34:19.783137+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:34:19.783137+0000","last_undegraded":"2024-04-25T12:34:19.783137+0000","last_fullsized":"2024-04-25T12:34:19.783137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782539+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:34:19.782539+0000","last_peered":"2024-04-25T12:34:19.782539+0000","last_clean":"2024-04-25T12:34:19.782539+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:34:19.782539+0000","last_undegraded":"2024-04-25T12:34:19.782539+0000","last_fullsized":"2024-04-25T12:34:19.782539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:50.121039+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:34:50.121039+0000","last_peered":"2024-04-25T12:34:50.121039+0000","last_clean":"2024-04-25T12:34:50.121039+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:34:50.121039+0000","last_undegraded":"2024-04-25T12:34:50.121039+0000","last_fullsized":"2024-04-25T12:34:50.121039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789165+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:34:19.789165+0000","last_peered":"2024-04-25T12:34:19.789165+0000","last_clean":"2024-04-25T12:34:19.789165+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:34:19.789165+0000","last_undegraded":"2024-04-25T12:34:19.789165+0000","last_fullsized":"2024-04-25T12:34:19.789165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781753+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:34:19.781753+0000","last_peered":"2024-04-25T12:34:19.781753+0000","last_clean":"2024-04-25T12:34:19.781753+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:34:19.781753+0000","last_undegraded":"2024-04-25T12:34:19.781753+0000","last_fullsized":"2024-04-25T12:34:19.781753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785020+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:34:19.785020+0000","last_peered":"2024-04-25T12:34:19.785020+0000","last_clean":"2024-04-25T12:34:19.785020+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:34:19.785020+0000","last_undegraded":"2024-04-25T12:34:19.785020+0000","last_fullsized":"2024-04-25T12:34:19.785020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781340+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:34:19.781340+0000","last_peered":"2024-04-25T12:34:19.781340+0000","last_clean":"2024-04-25T12:34:19.781340+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:34:19.781340+0000","last_undegraded":"2024-04-25T12:34:19.781340+0000","last_fullsized":"2024-04-25T12:34:19.781340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981734+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:34:18.981734+0000","last_peered":"2024-04-25T12:34:18.981734+0000","last_clean":"2024-04-25T12:34:18.981734+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:34:18.981734+0000","last_undegraded":"2024-04-25T12:34:18.981734+0000","last_fullsized":"2024-04-25T12:34:18.981734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781427+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:34:19.781427+0000","last_peered":"2024-04-25T12:34:19.781427+0000","last_clean":"2024-04-25T12:34:19.781427+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:34:19.781427+0000","last_undegraded":"2024-04-25T12:34:19.781427+0000","last_fullsized":"2024-04-25T12:34:19.781427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980630+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:34:18.980630+0000","last_peered":"2024-04-25T12:34:18.980630+0000","last_clean":"2024-04-25T12:34:18.980630+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:34:18.980630+0000","last_undegraded":"2024-04-25T12:34:18.980630+0000","last_fullsized":"2024-04-25T12:34:18.980630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980186+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:34:18.980186+0000","last_peered":"2024-04-25T12:34:18.980186+0000","last_clean":"2024-04-25T12:34:18.980186+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:34:18.980186+0000","last_undegraded":"2024-04-25T12:34:18.980186+0000","last_fullsized":"2024-04-25T12:34:18.980186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788210+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:34:19.788210+0000","last_peered":"2024-04-25T12:34:19.788210+0000","last_clean":"2024-04-25T12:34:19.788210+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:34:19.788210+0000","last_undegraded":"2024-04-25T12:34:19.788210+0000","last_fullsized":"2024-04-25T12:34:19.788210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782446+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:34:19.782446+0000","last_peered":"2024-04-25T12:34:19.782446+0000","last_clean":"2024-04-25T12:34:19.782446+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:34:19.782446+0000","last_undegraded":"2024-04-25T12:34:19.782446+0000","last_fullsized":"2024-04-25T12:34:19.782446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.982148+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:34:18.982148+0000","last_peered":"2024-04-25T12:34:18.982148+0000","last_clean":"2024-04-25T12:34:18.982148+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:34:18.982148+0000","last_undegraded":"2024-04-25T12:34:18.982148+0000","last_fullsized":"2024-04-25T12:34:18.982148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781207+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:34:19.781207+0000","last_peered":"2024-04-25T12:34:19.781207+0000","last_clean":"2024-04-25T12:34:19.781207+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:34:19.781207+0000","last_undegraded":"2024-04-25T12:34:19.781207+0000","last_fullsized":"2024-04-25T12:34:19.781207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.789184+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:34:19.789184+0000","last_peered":"2024-04-25T12:34:19.789184+0000","last_clean":"2024-04-25T12:34:19.789184+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:34:19.789184+0000","last_undegraded":"2024-04-25T12:34:19.789184+0000","last_fullsized":"2024-04-25T12:34:19.789184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783090+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:34:19.783090+0000","last_peered":"2024-04-25T12:34:19.783090+0000","last_clean":"2024-04-25T12:34:19.783090+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:34:19.783090+0000","last_undegraded":"2024-04-25T12:34:19.783090+0000","last_fullsized":"2024-04-25T12:34:19.783090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783799+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:34:19.783799+0000","last_peered":"2024-04-25T12:34:19.783799+0000","last_clean":"2024-04-25T12:34:19.783799+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:34:19.783799+0000","last_undegraded":"2024-04-25T12:34:19.783799+0000","last_fullsized":"2024-04-25T12:34:19.783799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983704+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:34:18.983704+0000","last_peered":"2024-04-25T12:34:18.983704+0000","last_clean":"2024-04-25T12:34:18.983704+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:34:18.983704+0000","last_undegraded":"2024-04-25T12:34:18.983704+0000","last_fullsized":"2024-04-25T12:34:18.983704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782288+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:34:19.782288+0000","last_peered":"2024-04-25T12:34:19.782288+0000","last_clean":"2024-04-25T12:34:19.782288+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:34:19.782288+0000","last_undegraded":"2024-04-25T12:34:19.782288+0000","last_fullsized":"2024-04-25T12:34:19.782288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.983392+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:34:18.983392+0000","last_peered":"2024-04-25T12:34:18.983392+0000","last_clean":"2024-04-25T12:34:18.983392+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:34:18.983392+0000","last_undegraded":"2024-04-25T12:34:18.983392+0000","last_fullsized":"2024-04-25T12:34:18.983392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.981957+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:34:18.981957+0000","last_peered":"2024-04-25T12:34:18.981957+0000","last_clean":"2024-04-25T12:34:18.981957+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:34:18.981957+0000","last_undegraded":"2024-04-25T12:34:18.981957+0000","last_fullsized":"2024-04-25T12:34:18.981957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.780879+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:34:19.780879+0000","last_peered":"2024-04-25T12:34:19.780879+0000","last_clean":"2024-04-25T12:34:19.780879+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:34:19.780879+0000","last_undegraded":"2024-04-25T12:34:19.780879+0000","last_fullsized":"2024-04-25T12:34:19.780879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.979899+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:34:18.979899+0000","last_peered":"2024-04-25T12:34:18.979899+0000","last_clean":"2024-04-25T12:34:18.979899+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:34:18.979899+0000","last_undegraded":"2024-04-25T12:34:18.979899+0000","last_fullsized":"2024-04-25T12:34:18.979899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:18.980556+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:34:18.980556+0000","last_peered":"2024-04-25T12:34:18.980556+0000","last_clean":"2024-04-25T12:34:18.980556+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:34:18.980556+0000","last_undegraded":"2024-04-25T12:34:18.980556+0000","last_fullsized":"2024-04-25T12:34:18.980556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783694+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:34:19.783694+0000","last_peered":"2024-04-25T12:34:19.783694+0000","last_clean":"2024-04-25T12:34:19.783694+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:34:19.783694+0000","last_undegraded":"2024-04-25T12:34:19.783694+0000","last_fullsized":"2024-04-25T12:34:19.783694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784671+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:34:19.784671+0000","last_peered":"2024-04-25T12:34:19.784671+0000","last_clean":"2024-04-25T12:34:19.784671+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:34:19.784671+0000","last_undegraded":"2024-04-25T12:34:19.784671+0000","last_fullsized":"2024-04-25T12:34:19.784671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.784485+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:34:19.784485+0000","last_peered":"2024-04-25T12:34:19.784485+0000","last_clean":"2024-04-25T12:34:19.784485+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:34:19.784485+0000","last_undegraded":"2024-04-25T12:34:19.784485+0000","last_fullsized":"2024-04-25T12:34:19.784485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.791100+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:34:19.791100+0000","last_peered":"2024-04-25T12:34:19.791100+0000","last_clean":"2024-04-25T12:34:19.791100+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:34:19.791100+0000","last_undegraded":"2024-04-25T12:34:19.791100+0000","last_fullsized":"2024-04-25T12:34:19.791100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.781305+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:34:19.781305+0000","last_peered":"2024-04-25T12:34:19.781305+0000","last_clean":"2024-04-25T12:34:19.781305+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:34:19.781305+0000","last_undegraded":"2024-04-25T12:34:19.781305+0000","last_fullsized":"2024-04-25T12:34:19.781305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782773+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:34:19.782773+0000","last_peered":"2024-04-25T12:34:19.782773+0000","last_clean":"2024-04-25T12:34:19.782773+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:34:19.782773+0000","last_undegraded":"2024-04-25T12:34:19.782773+0000","last_fullsized":"2024-04-25T12:34:19.782773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.786973+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:34:19.786973+0000","last_peered":"2024-04-25T12:34:19.786973+0000","last_clean":"2024-04-25T12:34:19.786973+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:34:19.786973+0000","last_undegraded":"2024-04-25T12:34:19.786973+0000","last_fullsized":"2024-04-25T12:34:19.786973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.783422+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:34:19.783422+0000","last_peered":"2024-04-25T12:34:19.783422+0000","last_clean":"2024-04-25T12:34:19.783422+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:34:19.783422+0000","last_undegraded":"2024-04-25T12:34:19.783422+0000","last_fullsized":"2024-04-25T12:34:19.783422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.788118+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:34:19.788118+0000","last_peered":"2024-04-25T12:34:19.788118+0000","last_clean":"2024-04-25T12:34:19.788118+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:34:19.788118+0000","last_undegraded":"2024-04-25T12:34:19.788118+0000","last_fullsized":"2024-04-25T12:34:19.788118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782309+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:34:19.782309+0000","last_peered":"2024-04-25T12:34:19.782309+0000","last_clean":"2024-04-25T12:34:19.782309+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:34:19.782309+0000","last_undegraded":"2024-04-25T12:34:19.782309+0000","last_fullsized":"2024-04-25T12:34:19.782309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.782643+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:34:19.782643+0000","last_peered":"2024-04-25T12:34:19.782643+0000","last_clean":"2024-04-25T12:34:19.782643+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:34:19.782643+0000","last_undegraded":"2024-04-25T12:34:19.782643+0000","last_fullsized":"2024-04-25T12:34:19.782643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T12:34:19.785344+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:34:19.785344+0000","last_peered":"2024-04-25T12:34:19.785344+0000","last_clean":"2024-04-25T12:34:19.785344+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:34:19.785344+0000","last_undegraded":"2024-04-25T12:34:19.785344+0000","last_fullsized":"2024-04-25T12:34:19.785344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6046,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":103079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":103639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":694015,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":108231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":331776,"data_stored":693461,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:34:55.823 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T12:34:55.823 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T12:34:55.823 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T12:34:55.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T12:34:56.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.118+0000 7f6812000640 1 Processor -- start 2024-04-25T12:34:56.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.119+0000 7f6812000640 1 -- start start 2024-04-25T12:34:56.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.120+0000 7f6812000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.120+0000 7f6812000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680c0cedf0 con 0x7f680c0cdca0 2024-04-25T12:34:56.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.120+0000 7f6812000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680c0c7df0 con 0x7f680c1568c0 2024-04-25T12:34:56.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.120+0000 7f6812000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f680c0cebd0 con 0x7f680c0c6800 2024-04-25T12:34:56.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f6803e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f680c0cdca0 0x7f680c15c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45828/0 (socket says 172.21.15.123:45828) 2024-04-25T12:34:56.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f6803e00640 1 -- 172.21.15.123:0/3977542267 learned_addr learned my addr 172.21.15.123:0/3977542267 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:56.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f6811600640 1 --2- 172.21.15.123:0/3977542267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f680be00640 1 -- 172.21.15.123:0/3977542267 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356604826 0 0) 0x7f680c0cedf0 con 0x7f680c0cdca0 2024-04-25T12:34:56.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f680be00640 1 -- 172.21.15.123:0/3977542267 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67f0003520 con 0x7f680c0cdca0 2024-04-25T12:34:56.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f680be00640 1 -- 172.21.15.123:0/3977542267 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310216153 0 0) 0x7f680c0c7df0 con 0x7f680c1568c0 2024-04-25T12:34:56.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f6811600640 1 -- 172.21.15.123:0/3977542267 >> v1:172.21.15.153:6789/0 conn(0x7f680c1568c0 legacy=0x7f680c0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f6811600640 1 -- 172.21.15.123:0/3977542267 >> v1:172.21.15.123:6789/0 conn(0x7f680c0cdca0 legacy=0x7f680c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.121+0000 7f6811600640 1 -- 172.21.15.123:0/3977542267 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c0c80d0 con 0x7f680c0c6800 2024-04-25T12:34:56.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.122+0000 7f6811600640 1 --2- 172.21.15.123:0/3977542267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c0ce690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6804009dc0 tx=0x7f680402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6be3189869433c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:56.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f680be00640 1 -- 172.21.15.123:0/3977542267 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804007740 con 0x7f680c0c6800 2024-04-25T12:34:56.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f680be00640 1 -- 172.21.15.123:0/3977542267 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68040078e0 con 0x7f680c0c6800 2024-04-25T12:34:56.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f680be00640 1 -- 172.21.15.123:0/3977542267 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804007c90 con 0x7f680c0c6800 2024-04-25T12:34:56.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f6812000640 1 -- 172.21.15.123:0/3977542267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 msgr2=0x7f680c0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f6812000640 1 --2- 172.21.15.123:0/3977542267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c0ce690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6804009dc0 tx=0x7f680402a6d0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f6812000640 1 -- 172.21.15.123:0/3977542267 shutdown_connections 2024-04-25T12:34:56.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f6812000640 1 --2- 172.21.15.123:0/3977542267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c0ce690 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f6812000640 1 -- 172.21.15.123:0/3977542267 >> 172.21.15.123:0/3977542267 conn(0x7f680c06dcd0 msgr2=0x7f680c06e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:56.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.123+0000 7f6812000640 1 -- 172.21.15.123:0/3977542267 shutdown_connections 2024-04-25T12:34:56.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.124+0000 7f6812000640 1 -- 172.21.15.123:0/3977542267 wait complete. 2024-04-25T12:34:56.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.126+0000 7f6812000640 1 Processor -- start 2024-04-25T12:34:56.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.126+0000 7f6812000640 1 -- start start 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6812000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6812000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f680c0cdca0 0x7f680c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6812000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c15a8b0 0x7f680c15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6812000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f680c0c8720 con 0x7f680c15a8b0 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6812000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f680c0cf0d0 con 0x7f680c0cdca0 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6812000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f680c0cd630 con 0x7f680c0c6800 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6811600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c15a8b0 0x7f680c15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6803e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f680c0cdca0 0x7f680c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6811600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c15a8b0 0x7f680c15acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39582/0 (socket says 172.21.15.123:39582) 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6811600640 1 -- 172.21.15.123:0/3047895985 learned_addr learned my addr 172.21.15.123:0/3047895985 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:56.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.127+0000 7f6810c00640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.128+0000 7f6803e00640 1 -- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 msgr2=0x7f680c1584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.128+0000 7f6803e00640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c1584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.128+0000 7f6803e00640 1 -- 172.21.15.123:0/3047895985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c15a8b0 msgr2=0x7f680c15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.128+0000 7f6803e00640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c15a8b0 0x7f680c15acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.128+0000 7f6803e00640 1 -- 172.21.15.123:0/3047895985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c1589f0 con 0x7f680c0cdca0 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.128+0000 7f6811600640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c15a8b0 0x7f680c15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.128+0000 7f6803e00640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f680c0cdca0 0x7f680c15ba80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c390 tx=0x7f67f4001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.129+0000 7f6809600640 1 -- 172.21.15.123:0/3047895985 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4017070 con 0x7f680c0cdca0 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.129+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c1e70c0 con 0x7f680c0cdca0 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.129+0000 7f6809600640 1 -- 172.21.15.123:0/3047895985 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f4005ce0 con 0x7f680c0cdca0 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.129+0000 7f6809600640 1 -- 172.21.15.123:0/3047895985 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f40072f0 con 0x7f680c0cdca0 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.129+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c156ce0 con 0x7f680c0cdca0 2024-04-25T12:34:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.130+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680c076da0 con 0x7f680c0cdca0 2024-04-25T12:34:56.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.131+0000 7f6809600640 1 -- 172.21.15.123:0/3047895985 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f67f401a070 con 0x7f680c0cdca0 2024-04-25T12:34:56.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.132+0000 7f6809600640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f67e80744b0 0x7f67e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.132+0000 7f6809600640 1 -- 172.21.15.123:0/3047895985 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f67f4094860 con 0x7f680c0cdca0 2024-04-25T12:34:56.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.132+0000 7f6810c00640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f67e80744b0 0x7f67e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.133+0000 7f6810c00640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f67e80744b0 0x7f67e80769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000f60 tx=0x7f67fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:56.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.136+0000 7f6809600640 1 -- 172.21.15.123:0/3047895985 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67f405e650 con 0x7f680c0cdca0 2024-04-25T12:34:56.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.366+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f680c0cc700 con 0x7f680c0cdca0 2024-04-25T12:34:56.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.367+0000 7f6809600640 1 -- 172.21.15.123:0/3047895985 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f67f4062950 con 0x7f680c0cdca0 2024-04-25T12:34:56.367 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T12:34:56.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f67e80744b0 msgr2=0x7f67e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f67e80744b0 0x7f67e80769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000f60 tx=0x7f67fc009040 comp rx=0 tx=0).stop 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f680c0cdca0 msgr2=0x7f680c15ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f680c0cdca0 0x7f680c15ba80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c390 tx=0x7f67f4001e70 comp rx=0 tx=0).stop 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 shutdown_connections 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f67e80744b0 0x7f67e80769c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c15a8b0 0x7f680c15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f680c0cdca0 0x7f680c15ba80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 --2- 172.21.15.123:0/3047895985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f680c0c6800 0x7f680c1584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 >> 172.21.15.123:0/3047895985 conn(0x7f680c06dcd0 msgr2=0x7f680c0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 shutdown_connections 2024-04-25T12:34:56.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.370+0000 7f6812000640 1 -- 172.21.15.123:0/3047895985 wait complete. 2024-04-25T12:34:56.405 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T12:34:56.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:56.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.659+0000 7fcecea00640 1 Processor -- start 2024-04-25T12:34:56.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.660+0000 7fcecea00640 1 -- start start 2024-04-25T12:34:56.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.662+0000 7fcecea00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcec81546d0 0x7fcec815e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.662+0000 7fcecea00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec815a330 con 0x7fcec80c9080 2024-04-25T12:34:56.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.662+0000 7fcecea00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec8073b10 con 0x7fcec815e590 2024-04-25T12:34:56.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.662+0000 7fcecea00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcec8073df0 con 0x7fcec81546d0 2024-04-25T12:34:56.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fcece000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcec815e590 0x7fcec806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:40922/0 (socket says 172.21.15.123:40922) 2024-04-25T12:34:56.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fcece000640 1 -- 172.21.15.123:0/3243655464 learned_addr learned my addr 172.21.15.123:0/3243655464 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:56.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fceccc00640 1 --2- 172.21.15.123:0/3243655464 >> v2:172.21.15.153:3300/0 conn(0x7fcec81546d0 0x7fcec815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fcec7e00640 1 -- 172.21.15.123:0/3243655464 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1811147620 0 0) 0x7fcec8073b10 con 0x7fcec815e590 2024-04-25T12:34:56.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fcec7e00640 1 -- 172.21.15.123:0/3243655464 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb0003520 con 0x7fcec815e590 2024-04-25T12:34:56.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fceccc00640 1 -- 172.21.15.123:0/3243655464 >> v1:172.21.15.153:6789/0 conn(0x7fcec815e590 legacy=0x7fcec806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fceccc00640 1 -- 172.21.15.123:0/3243655464 >> v1:172.21.15.123:6789/0 conn(0x7fcec80c9080 legacy=0x7fcec80c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fcec7e00640 1 -- 172.21.15.123:0/3243655464 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033715790 0 0) 0x7fcec815a330 con 0x7fcec80c9080 2024-04-25T12:34:56.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.663+0000 7fceccc00640 1 -- 172.21.15.123:0/3243655464 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec80740d0 con 0x7fcec81546d0 2024-04-25T12:34:56.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.664+0000 7fceccc00640 1 --2- 172.21.15.123:0/3243655464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec81546d0 0x7fcec815e050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fceb40097a0 tx=0x7fceb402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b606ad9e08e2767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:56.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcec7e00640 1 -- 172.21.15.123:0/3243655464 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4037070 con 0x7fcec81546d0 2024-04-25T12:34:56.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcec7e00640 1 -- 172.21.15.123:0/3243655464 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb4032640 con 0x7fcec81546d0 2024-04-25T12:34:56.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcec7e00640 1 -- 172.21.15.123:0/3243655464 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4032a10 con 0x7fcec81546d0 2024-04-25T12:34:56.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcecea00640 1 -- 172.21.15.123:0/3243655464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec81546d0 msgr2=0x7fcec815e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcecea00640 1 --2- 172.21.15.123:0/3243655464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec81546d0 0x7fcec815e050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fceb40097a0 tx=0x7fceb402a2c0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcecea00640 1 -- 172.21.15.123:0/3243655464 shutdown_connections 2024-04-25T12:34:56.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcecea00640 1 --2- 172.21.15.123:0/3243655464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec81546d0 0x7fcec815e050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcecea00640 1 -- 172.21.15.123:0/3243655464 >> 172.21.15.123:0/3243655464 conn(0x7fcec80cd600 msgr2=0x7fcec80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:56.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.665+0000 7fcecea00640 1 -- 172.21.15.123:0/3243655464 shutdown_connections 2024-04-25T12:34:56.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.666+0000 7fcecea00640 1 -- 172.21.15.123:0/3243655464 wait complete. 2024-04-25T12:34:56.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.668+0000 7fcecea00640 1 Processor -- start 2024-04-25T12:34:56.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.668+0000 7fcecea00640 1 -- start start 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fcecea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec80c9080 0x7fcec815c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fcecea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec81546d0 0x7fcec80c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fcecea00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815e590 0x7fcec8148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fcecea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8073760 con 0x7fcec81546d0 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fcecea00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcec815db30 con 0x7fcec80c9080 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fcecea00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcec815ce90 con 0x7fcec815e590 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fceccc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec81546d0 0x7fcec80c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.669+0000 7fceccc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec81546d0 0x7fcec80c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39594/0 (socket says 172.21.15.123:39594) 2024-04-25T12:34:56.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fcece000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815e590 0x7fcec8148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fceccc00640 1 -- 172.21.15.123:0/1403899102 learned_addr learned my addr 172.21.15.123:0/1403899102 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:56.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fcecd600640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec80c9080 0x7fcec815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fceccc00640 1 -- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815e590 msgr2=0x7fcec8148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fceccc00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815e590 0x7fcec8148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fceccc00640 1 -- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec80c9080 msgr2=0x7fcec815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fceccc00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec80c9080 0x7fcec815c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fceccc00640 1 -- 172.21.15.123:0/1403899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec815c9c0 con 0x7fcec81546d0 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.670+0000 7fceccc00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec81546d0 0x7fcec80c6f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fceb40098c0 tx=0x7fceb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.671+0000 7fcec5600640 1 -- 172.21.15.123:0/1403899102 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4041020 con 0x7fcec81546d0 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.671+0000 7fcec5600640 1 -- 172.21.15.123:0/1403899102 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb40306f0 con 0x7fcec81546d0 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.671+0000 7fcec5600640 1 -- 172.21.15.123:0/1403899102 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4037040 con 0x7fcec81546d0 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.671+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec80cb8a0 con 0x7fcec81546d0 2024-04-25T12:34:56.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.671+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec81f66e0 con 0x7fcec81546d0 2024-04-25T12:34:56.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.672+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec8076da0 con 0x7fcec81546d0 2024-04-25T12:34:56.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.673+0000 7fcec5600640 1 -- 172.21.15.123:0/1403899102 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fceb4044020 con 0x7fcec81546d0 2024-04-25T12:34:56.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.674+0000 7fcec5600640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcea80744b0 0x7fcea80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:56.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.674+0000 7fcec5600640 1 -- 172.21.15.123:0/1403899102 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fceb40b5b10 con 0x7fcec81546d0 2024-04-25T12:34:56.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.675+0000 7fcecd600640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcea80744b0 0x7fcea80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:56.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.676+0000 7fcecd600640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcea80744b0 0x7fcea80769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000f60 tx=0x7fcebc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:56.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.680+0000 7fcec5600640 1 -- 172.21.15.123:0/1403899102 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fceb404b070 con 0x7fcec81546d0 2024-04-25T12:34:56.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.879+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcec80734c0 con 0x7fcec81546d0 2024-04-25T12:34:56.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.883+0000 7fcec5600640 1 -- 172.21.15.123:0/1403899102 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18578 (secure 0 0 0) 0x7fceb407f870 con 0x7fcec81546d0 2024-04-25T12:34:56.883 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:56.883 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:18.535783+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcea80744b0 msgr2=0x7fcea80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcea80744b0 0x7fcea80769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000f60 tx=0x7fcebc009040 comp rx=0 tx=0).stop 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec81546d0 msgr2=0x7fcec80c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec81546d0 0x7fcec80c6f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fceb40098c0 tx=0x7fceb40046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 shutdown_connections 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcea80744b0 0x7fcea80769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcec815e590 0x7fcec8148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec81546d0 0x7fcec80c6f00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 --2- 172.21.15.123:0/1403899102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec80c9080 0x7fcec815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 >> 172.21.15.123:0/1403899102 conn(0x7fcec80cd600 msgr2=0x7fcec80ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 shutdown_connections 2024-04-25T12:34:56.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:56.885+0000 7fcecea00640 1 -- 172.21.15.123:0/1403899102 wait complete. 2024-04-25T12:34:56.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T12:34:56.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:34:57.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.165+0000 7fd4ea400640 1 Processor -- start 2024-04-25T12:34:57.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.166+0000 7fd4ea400640 1 -- start start 2024-04-25T12:34:57.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4ea400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd4e40cdd60 0x7fd4e40b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4ea400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd4e40c8c10 0x7fd4e4156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e406d9b0 0x7fd4e406ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4ea400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd4e4156d40 con 0x7fd4e40c8c10 2024-04-25T12:34:57.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4ea400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd4e415c760 con 0x7fd4e40cdd60 2024-04-25T12:34:57.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4ea400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e415ca40 con 0x7fd4e406d9b0 2024-04-25T12:34:57.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4e9000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd4e40cdd60 0x7fd4e40b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4e9a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e406d9b0 0x7fd4e406ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4dbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd4e40c8c10 0x7fd4e4156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4e9000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd4e40cdd60 0x7fd4e40b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34134/0 (socket says 172.21.15.123:34134) 2024-04-25T12:34:57.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.168+0000 7fd4e9000640 1 -- 172.21.15.123:0/3623738834 learned_addr learned my addr 172.21.15.123:0/3623738834 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:57.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.169+0000 7fd4e9000640 1 -- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e406d9b0 msgr2=0x7fd4e406ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:57.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.169+0000 7fd4e9000640 1 --2- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e406d9b0 0x7fd4e406ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.169+0000 7fd4e9000640 1 -- 172.21.15.123:0/3623738834 >> v2:172.21.15.123:3300/0 conn(0x7fd4e40c8c10 msgr2=0x7fd4e4156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:57.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.169+0000 7fd4e9000640 1 --2- 172.21.15.123:0/3623738834 >> v2:172.21.15.123:3300/0 conn(0x7fd4e40c8c10 0x7fd4e4156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.169+0000 7fd4e9000640 1 -- 172.21.15.123:0/3623738834 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e415cd20 con 0x7fd4e40cdd60 2024-04-25T12:34:57.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.169+0000 7fd4e9a00640 1 --2- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e406d9b0 0x7fd4e406ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:57.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.170+0000 7fd4e9000640 1 --2- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40cdd60 0x7fd4e40b7300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4e415c970 tx=0x7fd4cc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b66bd9b9ebb17dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:57.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.170+0000 7fd4e3e00640 1 -- 172.21.15.123:0/3623738834 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc004030 con 0x7fd4e40cdd60 2024-04-25T12:34:57.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.170+0000 7fd4e3e00640 1 -- 172.21.15.123:0/3623738834 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4cc0041d0 con 0x7fd4e40cdd60 2024-04-25T12:34:57.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.170+0000 7fd4e3e00640 1 -- 172.21.15.123:0/3623738834 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc003610 con 0x7fd4e40cdd60 2024-04-25T12:34:57.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 -- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40cdd60 msgr2=0x7fd4e40b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 --2- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40cdd60 0x7fd4e40b7300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4e415c970 tx=0x7fd4cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 -- 172.21.15.123:0/3623738834 shutdown_connections 2024-04-25T12:34:57.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 --2- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e406d9b0 0x7fd4e406ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 --2- 172.21.15.123:0/3623738834 >> v2:172.21.15.123:3300/0 conn(0x7fd4e40c8c10 0x7fd4e4156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 --2- 172.21.15.123:0/3623738834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40cdd60 0x7fd4e40b7300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 -- 172.21.15.123:0/3623738834 >> 172.21.15.123:0/3623738834 conn(0x7fd4e40d1810 msgr2=0x7fd4e40d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:57.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 -- 172.21.15.123:0/3623738834 shutdown_connections 2024-04-25T12:34:57.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.171+0000 7fd4ea400640 1 -- 172.21.15.123:0/3623738834 wait complete. 2024-04-25T12:34:57.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.174+0000 7fd4ea400640 1 Processor -- start 2024-04-25T12:34:57.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.174+0000 7fd4ea400640 1 -- start start 2024-04-25T12:34:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e406d9b0 0x7fd4e41f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40c8c10 0x7fd4e4151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e40cdd60 0x7fd4e415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4ea400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e40c9ea0 con 0x7fd4e406d9b0 2024-04-25T12:34:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4ea400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e415d2e0 con 0x7fd4e40c8c10 2024-04-25T12:34:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4ea400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e40ca020 con 0x7fd4e40cdd60 2024-04-25T12:34:57.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4e9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e406d9b0 0x7fd4e41f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4e9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e406d9b0 0x7fd4e41f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39608/0 (socket says 172.21.15.123:39608) 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4e9000640 1 -- 172.21.15.123:0/2036690807 learned_addr learned my addr 172.21.15.123:0/2036690807 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4e9a00640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e40cdd60 0x7fd4e415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.176+0000 7fd4dbe00640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40c8c10 0x7fd4e4151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.177+0000 7fd4e9000640 1 -- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e40cdd60 msgr2=0x7fd4e415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.177+0000 7fd4e9000640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e40cdd60 0x7fd4e415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.177+0000 7fd4e9000640 1 -- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40c8c10 msgr2=0x7fd4e4151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.177+0000 7fd4e9000640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40c8c10 0x7fd4e4151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.177+0000 7fd4e9000640 1 -- 172.21.15.123:0/2036690807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e41f29b0 con 0x7fd4e406d9b0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.177+0000 7fd4e9000640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e406d9b0 0x7fd4e41f2470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc02a1e0 tx=0x7fd4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.178+0000 7fd4e1600640 1 -- 172.21.15.123:0/2036690807 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc041020 con 0x7fd4e406d9b0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.178+0000 7fd4e1600640 1 -- 172.21.15.123:0/2036690807 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4cc005e10 con 0x7fd4e406d9b0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.178+0000 7fd4e1600640 1 -- 172.21.15.123:0/2036690807 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc004030 con 0x7fd4e406d9b0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.178+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e4159370 con 0x7fd4e406d9b0 2024-04-25T12:34:57.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.178+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e4159930 con 0x7fd4e406d9b0 2024-04-25T12:34:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.179+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e4076da0 con 0x7fd4e406d9b0 2024-04-25T12:34:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.180+0000 7fd4e1600640 1 -- 172.21.15.123:0/2036690807 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4cc044020 con 0x7fd4e406d9b0 2024-04-25T12:34:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.181+0000 7fd4e1600640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd4c00744e0 0x7fd4c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.181+0000 7fd4e1600640 1 -- 172.21.15.123:0/2036690807 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd4cc0b5e40 con 0x7fd4e406d9b0 2024-04-25T12:34:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.182+0000 7fd4dbe00640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd4c00744e0 0x7fd4c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.182+0000 7fd4dbe00640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd4c00744e0 0x7fd4c00769f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd4e41efd50 tx=0x7fd4d4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:57.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.186+0000 7fd4e1600640 1 -- 172.21.15.123:0/2036690807 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4cc04b030 con 0x7fd4e406d9b0 2024-04-25T12:34:57.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.384+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4e40cdad0 con 0x7fd4e406d9b0 2024-04-25T12:34:57.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.385+0000 7fd4e1600640 1 -- 172.21.15.123:0/2036690807 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fd4cc080310 con 0x7fd4e406d9b0 2024-04-25T12:34:57.386 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd4c00744e0 msgr2=0x7fd4c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd4c00744e0 0x7fd4c00769f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd4e41efd50 tx=0x7fd4d4009190 comp rx=0 tx=0).stop 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e406d9b0 msgr2=0x7fd4e41f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e406d9b0 0x7fd4e41f2470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc02a1e0 tx=0x7fd4cc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 shutdown_connections 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd4c00744e0 0x7fd4c00769f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd4e40cdd60 0x7fd4e415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4e40c8c10 0x7fd4e4151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 --2- 172.21.15.123:0/2036690807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e406d9b0 0x7fd4e41f2470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 >> 172.21.15.123:0/2036690807 conn(0x7fd4e40d1810 msgr2=0x7fd4e40c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 shutdown_connections 2024-04-25T12:34:57.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.388+0000 7fd4ea400640 1 -- 172.21.15.123:0/2036690807 wait complete. 2024-04-25T12:34:57.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:34:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.681+0000 7f3aa8a00640 1 Processor -- start 2024-04-25T12:34:57.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.682+0000 7f3aa8a00640 1 -- start start 2024-04-25T12:34:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa8a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3aa40cdd60 0x7f3aa40b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40c8c10 0x7f3aa4156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa8a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa415c760 con 0x7f3aa406d9b0 2024-04-25T12:34:57.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa8a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa4156d40 con 0x7f3aa40cdd60 2024-04-25T12:34:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa415ca40 con 0x7f3aa40c8c10 2024-04-25T12:34:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa3200640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3aa406d9b0 0x7f3aa40c9ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:40930/0 (socket says 172.21.15.123:40930) 2024-04-25T12:34:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa2800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3aa40cdd60 0x7f3aa40b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa3200640 1 -- 172.21.15.123:0/4172308139 learned_addr learned my addr 172.21.15.123:0/4172308139 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:57.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.684+0000 7f3aa1e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40c8c10 0x7f3aa4156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.685+0000 7f3aa1400640 1 -- 172.21.15.123:0/4172308139 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464763166 0 0) 0x7f3aa415c760 con 0x7f3aa406d9b0 2024-04-25T12:34:57.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.685+0000 7f3aa1400640 1 -- 172.21.15.123:0/4172308139 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a88003520 con 0x7f3aa406d9b0 2024-04-25T12:34:57.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.685+0000 7f3aa2800640 1 -- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40c8c10 msgr2=0x7f3aa4156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:57.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.685+0000 7f3aa2800640 1 --2- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40c8c10 0x7f3aa4156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.685+0000 7f3aa2800640 1 -- 172.21.15.123:0/4172308139 >> v1:172.21.15.153:6789/0 conn(0x7f3aa406d9b0 legacy=0x7f3aa40c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.685+0000 7f3aa2800640 1 -- 172.21.15.123:0/4172308139 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa415cd20 con 0x7f3aa40cdd60 2024-04-25T12:34:57.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.685+0000 7f3aa1e00640 1 --2- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40c8c10 0x7f3aa4156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:57.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.686+0000 7f3aa2800640 1 --2- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40cdd60 0x7f3aa40b7300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009790 tx=0x7f3a9402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48f6e7b1702ad8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:57.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.686+0000 7f3aa1400640 1 -- 172.21.15.123:0/4172308139 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94037070 con 0x7f3aa40cdd60 2024-04-25T12:34:57.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.686+0000 7f3aa1400640 1 -- 172.21.15.123:0/4172308139 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94032440 con 0x7f3aa40cdd60 2024-04-25T12:34:57.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.686+0000 7f3aa1400640 1 -- 172.21.15.123:0/4172308139 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94032810 con 0x7f3aa40cdd60 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 -- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40cdd60 msgr2=0x7f3aa40b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40cdd60 0x7f3aa40b7300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009790 tx=0x7f3a9402a180 comp rx=0 tx=0).stop 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 -- 172.21.15.123:0/4172308139 shutdown_connections 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40c8c10 0x7f3aa4156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/4172308139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40cdd60 0x7f3aa40b7300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 -- 172.21.15.123:0/4172308139 >> 172.21.15.123:0/4172308139 conn(0x7f3aa40d1810 msgr2=0x7f3aa40d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 -- 172.21.15.123:0/4172308139 shutdown_connections 2024-04-25T12:34:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.687+0000 7f3aa8a00640 1 -- 172.21.15.123:0/4172308139 wait complete. 2024-04-25T12:34:57.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.689+0000 7f3aa8a00640 1 Processor -- start 2024-04-25T12:34:57.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.689+0000 7f3aa8a00640 1 -- start start 2024-04-25T12:34:57.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa406d9b0 0x7f3aa41f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40c8c10 0x7f3aa41f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa40cdd60 0x7f3aa40d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa40ca5b0 con 0x7f3aa40cdd60 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa415d2e0 con 0x7f3aa40c8c10 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa40ca730 con 0x7f3aa406d9b0 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa2800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa406d9b0 0x7f3aa41f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40c8c10 0x7f3aa41f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa40cdd60 0x7f3aa40d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa2800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa406d9b0 0x7f3aa41f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38362/0 (socket says 172.21.15.123:38362) 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.691+0000 7f3aa2800640 1 -- 172.21.15.123:0/1837881743 learned_addr learned my addr 172.21.15.123:0/1837881743 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa2800640 1 -- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40c8c10 msgr2=0x7f3aa41f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa2800640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40c8c10 0x7f3aa41f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa2800640 1 -- 172.21.15.123:0/1837881743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa40cdd60 msgr2=0x7f3aa40d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa2800640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa40cdd60 0x7f3aa40d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa2800640 1 -- 172.21.15.123:0/1837881743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa41e3d20 con 0x7f3aa406d9b0 2024-04-25T12:34:57.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa2800640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa406d9b0 0x7f3aa41f5f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a94035040 tx=0x7f3a940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:57.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3a9aa00640 1 -- 172.21.15.123:0/1837881743 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94037040 con 0x7f3aa406d9b0 2024-04-25T12:34:57.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3a9aa00640 1 -- 172.21.15.123:0/1837881743 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94042070 con 0x7f3aa406d9b0 2024-04-25T12:34:57.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3a9aa00640 1 -- 172.21.15.123:0/1837881743 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94006030 con 0x7f3aa406d9b0 2024-04-25T12:34:57.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa41f64a0 con 0x7f3aa406d9b0 2024-04-25T12:34:57.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.692+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa4153d80 con 0x7f3aa406d9b0 2024-04-25T12:34:57.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.693+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa41e3ef0 con 0x7f3aa406d9b0 2024-04-25T12:34:57.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.695+0000 7f3a9aa00640 1 -- 172.21.15.123:0/1837881743 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a940061d0 con 0x7f3aa406d9b0 2024-04-25T12:34:57.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.696+0000 7f3a9aa00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a840744e0 0x7f3a840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:57.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.696+0000 7f3a9aa00640 1 -- 172.21.15.123:0/1837881743 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3a940b6810 con 0x7f3aa406d9b0 2024-04-25T12:34:57.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.696+0000 7f3aa1e00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a840744e0 0x7f3a840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:57.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.697+0000 7f3aa1e00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a840744e0 0x7f3a840769f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3aa41efc60 tx=0x7f3a90009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:57.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.699+0000 7f3a9aa00640 1 -- 172.21.15.123:0/1837881743 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a94080690 con 0x7f3aa406d9b0 2024-04-25T12:34:57.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.896+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3aa4076da0 con 0x7f3aa406d9b0 2024-04-25T12:34:57.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.897+0000 7f3a9aa00640 1 -- 172.21.15.123:0/1837881743 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f3a94084990 con 0x7f3aa406d9b0 2024-04-25T12:34:57.898 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:34:57.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a840744e0 msgr2=0x7f3a840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a840744e0 0x7f3a840769f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3aa41efc60 tx=0x7f3a90009190 comp rx=0 tx=0).stop 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa406d9b0 msgr2=0x7f3aa41f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa406d9b0 0x7f3aa41f5f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a94035040 tx=0x7f3a940046c0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 shutdown_connections 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a840744e0 0x7f3a840769f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aa40cdd60 0x7f3aa40d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40c8c10 0x7f3aa41f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 --2- 172.21.15.123:0/1837881743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa406d9b0 0x7f3aa41f5f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 >> 172.21.15.123:0/1837881743 conn(0x7f3aa40d1810 msgr2=0x7f3aa40c6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 shutdown_connections 2024-04-25T12:34:57.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:57.900+0000 7f3aa8a00640 1 -- 172.21.15.123:0/1837881743 wait complete. 2024-04-25T12:34:57.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:34:58.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.190+0000 7f1bfd800640 1 Processor -- start 2024-04-25T12:34:58.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.190+0000 7f1bfd800640 1 -- start start 2024-04-25T12:34:58.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bfd800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7e90 0x7f1bf81640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bfd800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf80c5300 con 0x7f1bf80c7a80 2024-04-25T12:34:58.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bfd800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf80c55e0 con 0x7f1bf8152930 2024-04-25T12:34:58.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bfd800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf80c58c0 con 0x7f1bf80c7e90 2024-04-25T12:34:58.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bf7400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1bf80c7a80 0x7f1bf80c9a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45850/0 (socket says 172.21.15.123:45850) 2024-04-25T12:34:58.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bf7400640 1 -- 172.21.15.123:0/1458736143 learned_addr learned my addr 172.21.15.123:0/1458736143 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:58.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bfce00640 1 --2- 172.21.15.123:0/1458736143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7e90 0x7f1bf81640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.193+0000 7f1bf6a00640 1 -- 172.21.15.123:0/1458736143 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497395479 0 0) 0x7f1bf80c5300 con 0x7f1bf80c7a80 2024-04-25T12:34:58.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.194+0000 7f1bf6a00640 1 -- 172.21.15.123:0/1458736143 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bdc003520 con 0x7f1bf80c7a80 2024-04-25T12:34:58.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.194+0000 7f1bf6a00640 1 -- 172.21.15.123:0/1458736143 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541580097 0 0) 0x7f1bf80c55e0 con 0x7f1bf8152930 2024-04-25T12:34:58.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.194+0000 7f1bf6a00640 1 -- 172.21.15.123:0/1458736143 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bf80c5300 con 0x7f1bf8152930 2024-04-25T12:34:58.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.194+0000 7f1bfce00640 1 -- 172.21.15.123:0/1458736143 >> v1:172.21.15.153:6789/0 conn(0x7f1bf8152930 legacy=0x7f1bf806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.194+0000 7f1bfce00640 1 -- 172.21.15.123:0/1458736143 >> v1:172.21.15.123:6789/0 conn(0x7f1bf80c7a80 legacy=0x7f1bf80c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.194+0000 7f1bfce00640 1 -- 172.21.15.123:0/1458736143 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf80c5ba0 con 0x7f1bf80c7e90 2024-04-25T12:34:58.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.195+0000 7f1bfce00640 1 --2- 172.21.15.123:0/1458736143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7e90 0x7f1bf81640f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009dc0 tx=0x7f1bf002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1386a1c209b15dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:58.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.195+0000 7f1bf6a00640 1 -- 172.21.15.123:0/1458736143 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0007740 con 0x7f1bf80c7e90 2024-04-25T12:34:58.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.195+0000 7f1bf6a00640 1 -- 172.21.15.123:0/1458736143 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf00078e0 con 0x7f1bf80c7e90 2024-04-25T12:34:58.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.195+0000 7f1bf6a00640 1 -- 172.21.15.123:0/1458736143 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0007cb0 con 0x7f1bf80c7e90 2024-04-25T12:34:58.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.196+0000 7f1bfd800640 1 -- 172.21.15.123:0/1458736143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7e90 msgr2=0x7f1bf81640f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.196+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1458736143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7e90 0x7f1bf81640f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009dc0 tx=0x7f1bf002a6d0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.196+0000 7f1bfd800640 1 -- 172.21.15.123:0/1458736143 shutdown_connections 2024-04-25T12:34:58.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.196+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1458736143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7e90 0x7f1bf81640f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.196+0000 7f1bfd800640 1 -- 172.21.15.123:0/1458736143 >> 172.21.15.123:0/1458736143 conn(0x7f1bf80cae20 msgr2=0x7f1bf8073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:58.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.196+0000 7f1bfd800640 1 -- 172.21.15.123:0/1458736143 shutdown_connections 2024-04-25T12:34:58.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.196+0000 7f1bfd800640 1 -- 172.21.15.123:0/1458736143 wait complete. 2024-04-25T12:34:58.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.198+0000 7f1bfd800640 1 Processor -- start 2024-04-25T12:34:58.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.198+0000 7f1bfd800640 1 -- start start 2024-04-25T12:34:58.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfd800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7a80 0x7f1bf81e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfd800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf80c7e90 0x7f1bf81e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf8152930 0x7f1bf81df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf8071b70 con 0x7f1bf8152930 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfd800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf80c8c00 con 0x7f1bf80c7e90 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfd800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf80c5e80 con 0x7f1bf80c7a80 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf8152930 0x7f1bf81df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7a80 0x7f1bf81e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bf7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf80c7e90 0x7f1bf81e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf8152930 0x7f1bf81df8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39626/0 (socket says 172.21.15.123:39626) 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7a80 0x7f1bf81e6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:38380/0 (socket says 172.21.15.123:38380) 2024-04-25T12:34:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.200+0000 7f1bfce00640 1 -- 172.21.15.123:0/1660008824 learned_addr learned my addr 172.21.15.123:0/1660008824 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bf7e00640 1 -- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf80c7e90 msgr2=0x7f1bf81e79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bf7e00640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf80c7e90 0x7f1bf81e79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bf7e00640 1 -- 172.21.15.123:0/1660008824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf8152930 msgr2=0x7f1bf81df8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bf7e00640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf8152930 0x7f1bf81df8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bf7e00640 1 -- 172.21.15.123:0/1660008824 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf81dfde0 con 0x7f1bf80c7a80 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bfce00640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf8152930 0x7f1bf81df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bf7e00640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7a80 0x7f1bf81e6f40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1be800c3c0 tx=0x7f1be8001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1befe00640 1 -- 172.21.15.123:0/1660008824 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8017070 con 0x7f1bf80c7a80 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1befe00640 1 -- 172.21.15.123:0/1660008824 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be80052d0 con 0x7f1bf80c7a80 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1befe00640 1 -- 172.21.15.123:0/1660008824 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8003830 con 0x7f1bf80c7a80 2024-04-25T12:34:58.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.201+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf81f6990 con 0x7f1bf80c7a80 2024-04-25T12:34:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.202+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf81f6f50 con 0x7f1bf80c7a80 2024-04-25T12:34:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.203+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf8076da0 con 0x7f1bf80c7a80 2024-04-25T12:34:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.204+0000 7f1befe00640 1 -- 172.21.15.123:0/1660008824 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1be801a070 con 0x7f1bf80c7a80 2024-04-25T12:34:58.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.205+0000 7f1befe00640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1bd80744e0 0x7f1bd80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.205+0000 7f1befe00640 1 -- 172.21.15.123:0/1660008824 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1be8094770 con 0x7f1bf80c7a80 2024-04-25T12:34:58.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.205+0000 7f1bf7400640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1bd80744e0 0x7f1bd80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.206+0000 7f1bf7400640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1bd80744e0 0x7f1bd80769f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1bf81e4d30 tx=0x7f1be4007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:58.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.208+0000 7f1befe00640 1 -- 172.21.15.123:0/1660008824 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1be805e670 con 0x7f1bf80c7a80 2024-04-25T12:34:58.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.404+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1bf8075dd0 con 0x7f1bf80c7a80 2024-04-25T12:34:58.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.405+0000 7f1befe00640 1 -- 172.21.15.123:0/1660008824 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f1be8062970 con 0x7f1bf80c7a80 2024-04-25T12:34:58.405 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1bd80744e0 msgr2=0x7f1bd80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1bd80744e0 0x7f1bd80769f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1bf81e4d30 tx=0x7f1be4007d50 comp rx=0 tx=0).stop 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7a80 msgr2=0x7f1bf81e6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7a80 0x7f1bf81e6f40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1be800c3c0 tx=0x7f1be8001e70 comp rx=0 tx=0).stop 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 shutdown_connections 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1bd80744e0 0x7f1bd80769f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf8152930 0x7f1bf81df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf80c7e90 0x7f1bf81e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.407+0000 7f1bfd800640 1 --2- 172.21.15.123:0/1660008824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bf80c7a80 0x7f1bf81e6f40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.408+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 >> 172.21.15.123:0/1660008824 conn(0x7f1bf80cae20 msgr2=0x7f1bf81e3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:58.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.408+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 shutdown_connections 2024-04-25T12:34:58.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.408+0000 7f1bfd800640 1 -- 172.21.15.123:0/1660008824 wait complete. 2024-04-25T12:34:58.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T12:34:58.551 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-25T12:34:58.551+0000 7f1a9b000640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T12:34:58.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.682+0000 7fabbf200640 1 Processor -- start 2024-04-25T12:34:58.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.682+0000 7fabbf200640 1 -- start start 2024-04-25T12:34:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.684+0000 7fabbf200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabbf200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fabb80c8170 0x7fabb80ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabbf200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb8074400 con 0x7fabb80c8fd0 2024-04-25T12:34:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabbf200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fabb81595e0 con 0x7fabb80c8170 2024-04-25T12:34:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabbf200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fabb8074670 con 0x7fabb81426a0 2024-04-25T12:34:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabb7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fabb80c8fd0 0x7fabb8153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45856/0 (socket says 172.21.15.123:45856) 2024-04-25T12:34:58.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabb7e00640 1 -- 172.21.15.123:0/4108900248 learned_addr learned my addr 172.21.15.123:0/4108900248 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:34:58.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabbd200640 1 --2- 172.21.15.123:0/4108900248 >> v2:172.21.15.123:3300/0 conn(0x7fabb80c8170 0x7fabb80ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabb7400640 1 --2- 172.21.15.123:0/4108900248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabbd200640 1 -- 172.21.15.123:0/4108900248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 msgr2=0x7fabb8159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:58.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabbd200640 1 --2- 172.21.15.123:0/4108900248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb8159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.686+0000 7fabbd200640 1 -- 172.21.15.123:0/4108900248 >> v1:172.21.15.123:6789/0 conn(0x7fabb80c8fd0 legacy=0x7fabb8153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.685+0000 7fabb6a00640 1 -- 172.21.15.123:0/4108900248 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 477851926 0 0) 0x7fabb8074400 con 0x7fabb80c8fd0 2024-04-25T12:34:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.686+0000 7fabbd200640 1 -- 172.21.15.123:0/4108900248 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb8074950 con 0x7fabb80c8170 2024-04-25T12:34:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.686+0000 7fabb7400640 1 --2- 172.21.15.123:0/4108900248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:34:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.687+0000 7fabbd200640 1 --2- 172.21.15.123:0/4108900248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8170 0x7fabb80ca200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d90 tx=0x7fabb002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56b19d87f32ad7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.687+0000 7fabb6a00640 1 -- 172.21.15.123:0/4108900248 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0007740 con 0x7fabb80c8170 2024-04-25T12:34:58.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.687+0000 7fabb6a00640 1 -- 172.21.15.123:0/4108900248 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb00078e0 con 0x7fabb80c8170 2024-04-25T12:34:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.687+0000 7fabb6a00640 1 -- 172.21.15.123:0/4108900248 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0007cb0 con 0x7fabb80c8170 2024-04-25T12:34:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 -- 172.21.15.123:0/4108900248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8170 msgr2=0x7fabb80ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 --2- 172.21.15.123:0/4108900248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8170 0x7fabb80ca200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d90 tx=0x7fabb002a660 comp rx=0 tx=0).stop 2024-04-25T12:34:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 -- 172.21.15.123:0/4108900248 shutdown_connections 2024-04-25T12:34:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 --2- 172.21.15.123:0/4108900248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8170 0x7fabb80ca200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 --2- 172.21.15.123:0/4108900248 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 -- 172.21.15.123:0/4108900248 >> 172.21.15.123:0/4108900248 conn(0x7fabb806f440 msgr2=0x7fabb806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 -- 172.21.15.123:0/4108900248 shutdown_connections 2024-04-25T12:34:58.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.688+0000 7fabbf200640 1 -- 172.21.15.123:0/4108900248 wait complete. 2024-04-25T12:34:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.692+0000 7fabbf200640 1 Processor -- start 2024-04-25T12:34:58.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.692+0000 7fabbf200640 1 -- start start 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabbf200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb80c8170 0x7fabb81df470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabbf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8fd0 0x7fabb80c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabbf200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb81e31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabbf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabb80c4880 con 0x7fabb80c8fd0 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabbf200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabb815a300 con 0x7fabb80c8170 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabbf200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fabb80c8580 con 0x7fabb81426a0 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8fd0 0x7fabb80c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb80c8170 0x7fabb81df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabbd200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb81e31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8fd0 0x7fabb80c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39642/0 (socket says 172.21.15.123:39642) 2024-04-25T12:34:58.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.694+0000 7fabb7400640 1 -- 172.21.15.123:0/1433674637 learned_addr learned my addr 172.21.15.123:0/1433674637 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:58.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.695+0000 7fabb7400640 1 -- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 msgr2=0x7fabb81e31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.695+0000 7fabb7400640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb81e31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.695+0000 7fabb7400640 1 -- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb80c8170 msgr2=0x7fabb81df470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.695+0000 7fabb7400640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb80c8170 0x7fabb81df470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.695+0000 7fabb7400640 1 -- 172.21.15.123:0/1433674637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb8070950 con 0x7fabb80c8fd0 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.695+0000 7fabb7400640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8fd0 0x7fabb80c4b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faba400e960 tx=0x7faba400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.696+0000 7fabafe00640 1 -- 172.21.15.123:0/1433674637 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4017070 con 0x7fabb80c8fd0 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.696+0000 7fabafe00640 1 -- 172.21.15.123:0/1433674637 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba400ca00 con 0x7fabb80c8fd0 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.696+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb81e3dc0 con 0x7fabb80c8fd0 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.696+0000 7fabafe00640 1 -- 172.21.15.123:0/1433674637 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba400cdd0 con 0x7fabb80c8fd0 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.696+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb80c6400 con 0x7fabb80c8fd0 2024-04-25T12:34:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.697+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb81e42e0 con 0x7fabb80c8fd0 2024-04-25T12:34:58.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.699+0000 7fabafe00640 1 -- 172.21.15.123:0/1433674637 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faba401a070 con 0x7fabb80c8fd0 2024-04-25T12:34:58.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.700+0000 7fabafe00640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab980744b0 0x7fab980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:58.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.700+0000 7fabafe00640 1 -- 172.21.15.123:0/1433674637 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faba40957a0 con 0x7fabb80c8fd0 2024-04-25T12:34:58.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.701+0000 7fabb7e00640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab980744b0 0x7fab980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:58.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.701+0000 7fabb7e00640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab980744b0 0x7fab980769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faba8007fd0 tx=0x7faba8007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:58.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.710+0000 7fabafe00640 1 -- 172.21.15.123:0/1433674637 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fabb81e42e0 con 0x7fabb80c8fd0 2024-04-25T12:34:58.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.908+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fabb8076700 con 0x7fabb80c8fd0 2024-04-25T12:34:58.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.908+0000 7fabafe00640 1 -- 172.21.15.123:0/1433674637 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7faba405f590 con 0x7fabb80c8fd0 2024-04-25T12:34:58.909 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:34:58.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab980744b0 msgr2=0x7fab980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab980744b0 0x7fab980769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faba8007fd0 tx=0x7faba8007de0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8fd0 msgr2=0x7fabb80c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:58.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8fd0 0x7fabb80c4b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faba400e960 tx=0x7faba400ee20 comp rx=0 tx=0).stop 2024-04-25T12:34:58.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 shutdown_connections 2024-04-25T12:34:58.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab980744b0 0x7fab980769c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabb81426a0 0x7fabb81e31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb80c8fd0 0x7fabb80c4b20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 --2- 172.21.15.123:0/1433674637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb80c8170 0x7fabb81df470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:58.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 >> 172.21.15.123:0/1433674637 conn(0x7fabb806f440 msgr2=0x7fabb8153640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:58.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 shutdown_connections 2024-04-25T12:34:58.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:58.911+0000 7fabbf200640 1 -- 172.21.15.123:0/1433674637 wait complete. 2024-04-25T12:34:58.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:34:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.178+0000 7f8a2ca00640 1 Processor -- start 2024-04-25T12:34:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.178+0000 7f8a2ca00640 1 -- start start 2024-04-25T12:34:59.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a2ca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a2ca00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8a28071ee0 0x7f8a280c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a2ca00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8a280c9ae0 0x7f8a2815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a2ca00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8a280c59d0 con 0x7f8a280c9ae0 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a2ca00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8a280c90e0 con 0x7f8a28071ee0 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a2ca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8a280c5450 con 0x7f8a280ca110 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a26800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8a280c9ae0 0x7f8a2815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a26800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8a280c9ae0 0x7f8a2815fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39646/0 (socket says 172.21.15.123:39646) 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a1fe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8a28071ee0 0x7f8a280c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a26800640 1 -- 172.21.15.123:0/3873446403 learned_addr learned my addr 172.21.15.123:0/3873446403 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:59.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.181+0000 7f8a25e00640 1 --2- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.182+0000 7f8a1fe00640 1 -- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 msgr2=0x7f8a280719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:59.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.182+0000 7f8a1fe00640 1 --2- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.182+0000 7f8a1fe00640 1 -- 172.21.15.123:0/3873446403 >> v2:172.21.15.123:3300/0 conn(0x7f8a280c9ae0 msgr2=0x7f8a2815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:59.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.182+0000 7f8a1fe00640 1 --2- 172.21.15.123:0/3873446403 >> v2:172.21.15.123:3300/0 conn(0x7f8a280c9ae0 0x7f8a2815fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.182+0000 7f8a1fe00640 1 -- 172.21.15.123:0/3873446403 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a280c56f0 con 0x7f8a28071ee0 2024-04-25T12:34:59.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.183+0000 7f8a1fe00640 1 --2- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28071ee0 0x7f8a280c95a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a28073190 tx=0x7f8a1002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8e2e1246ace8312 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:59.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.183+0000 7f8a25400640 1 -- 172.21.15.123:0/3873446403 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10037070 con 0x7f8a28071ee0 2024-04-25T12:34:59.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.183+0000 7f8a25400640 1 -- 172.21.15.123:0/3873446403 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a10032440 con 0x7f8a28071ee0 2024-04-25T12:34:59.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.183+0000 7f8a25400640 1 -- 172.21.15.123:0/3873446403 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10032810 con 0x7f8a28071ee0 2024-04-25T12:34:59.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 -- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28071ee0 msgr2=0x7f8a280c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28071ee0 0x7f8a280c95a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a28073190 tx=0x7f8a1002a1a0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 -- 172.21.15.123:0/3873446403 shutdown_connections 2024-04-25T12:34:59.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/3873446403 >> v2:172.21.15.123:3300/0 conn(0x7f8a280c9ae0 0x7f8a2815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a28071ee0 0x7f8a280c95a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/3873446403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 -- 172.21.15.123:0/3873446403 >> 172.21.15.123:0/3873446403 conn(0x7f8a28070e80 msgr2=0x7f8a28071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:59.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 -- 172.21.15.123:0/3873446403 shutdown_connections 2024-04-25T12:34:59.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.184+0000 7f8a2ca00640 1 -- 172.21.15.123:0/3873446403 wait complete. 2024-04-25T12:34:59.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.186+0000 7f8a2ca00640 1 Processor -- start 2024-04-25T12:34:59.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.187+0000 7f8a2ca00640 1 -- start start 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a2ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a28071ee0 0x7f8a28158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a2ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a280c9ae0 0x7f8a28159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a2ca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a2ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28156c70 con 0x7f8a28071ee0 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a2ca00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a280c8c50 con 0x7f8a280c9ae0 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a2ca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8a28068d50 con 0x7f8a280ca110 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a25e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a28071ee0 0x7f8a28158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a25e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a28071ee0 0x7f8a28158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39660/0 (socket says 172.21.15.123:39660) 2024-04-25T12:34:59.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a25e00640 1 -- 172.21.15.123:0/2789935473 learned_addr learned my addr 172.21.15.123:0/2789935473 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.188+0000 7f8a1fe00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a280c9ae0 0x7f8a28159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a26800640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a25e00640 1 -- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 msgr2=0x7f8a280cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a25e00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a25e00640 1 -- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a280c9ae0 msgr2=0x7f8a28159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a25e00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a280c9ae0 0x7f8a28159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a25e00640 1 -- 172.21.15.123:0/2789935473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2815a120 con 0x7f8a28071ee0 2024-04-25T12:34:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a25e00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a28071ee0 0x7f8a28158b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800e870 tx=0x7f8a1800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a1e000640 1 -- 172.21.15.123:0/2789935473 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18017070 con 0x7f8a28071ee0 2024-04-25T12:34:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a1e000640 1 -- 172.21.15.123:0/2789935473 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1800c940 con 0x7f8a28071ee0 2024-04-25T12:34:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a1e000640 1 -- 172.21.15.123:0/2789935473 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800cd30 con 0x7f8a28071ee0 2024-04-25T12:34:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.189+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2815a400 con 0x7f8a28071ee0 2024-04-25T12:34:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.190+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a28158660 con 0x7f8a28071ee0 2024-04-25T12:34:59.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.191+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a28076700 con 0x7f8a28071ee0 2024-04-25T12:34:59.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.192+0000 7f8a1e000640 1 -- 172.21.15.123:0/2789935473 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8a1801a070 con 0x7f8a28071ee0 2024-04-25T12:34:59.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.193+0000 7f8a1e000640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8a040744b0 0x7f8a040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.193+0000 7f8a1e000640 1 -- 172.21.15.123:0/2789935473 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8a180957b0 con 0x7f8a28071ee0 2024-04-25T12:34:59.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.193+0000 7f8a1fe00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8a040744b0 0x7f8a040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.194+0000 7f8a1fe00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8a040744b0 0x7f8a040769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:59.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.202+0000 7f8a1e000640 1 -- 172.21.15.123:0/2789935473 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a1805f520 con 0x7f8a28071ee0 2024-04-25T12:34:59.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.445+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a280c96a0 con 0x7f8a28071ee0 2024-04-25T12:34:59.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.445+0000 7f8a1e000640 1 -- 172.21.15.123:0/2789935473 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6881 (secure 0 0 0) 0x7f8a18063820 con 0x7f8a28071ee0 2024-04-25T12:34:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:59.446 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:55.199669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4270,"mds_1":4264,"mds_2":4258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4258":{"gid":4258,"name":"d","rank":2,"incarnation":13,"state":"up:active","state_seq":13,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4264":{"gid":4264,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":12,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4258,4264,4270]},"id":1}]} 2024-04-25T12:34:59.446 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 14 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.448+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8a040744b0 msgr2=0x7f8a040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.448+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8a040744b0 0x7f8a040769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).stop 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.448+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a28071ee0 msgr2=0x7f8a28158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.448+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a28071ee0 0x7f8a28158b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800e870 tx=0x7f8a1800ed30 comp rx=0 tx=0).stop 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.448+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 shutdown_connections 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.448+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8a040744b0 0x7f8a040769c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.448+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a280ca110 0x7f8a280cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.449+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a280c9ae0 0x7f8a28159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.449+0000 7f8a2ca00640 1 --2- 172.21.15.123:0/2789935473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a28071ee0 0x7f8a28158b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.449+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 >> 172.21.15.123:0/2789935473 conn(0x7f8a28070e80 msgr2=0x7f8a2806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:59.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.449+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 shutdown_connections 2024-04-25T12:34:59.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.449+0000 7f8a2ca00640 1 -- 172.21.15.123:0/2789935473 wait complete. 2024-04-25T12:34:59.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:34:59.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.729+0000 7f26c1c00640 1 Processor -- start 2024-04-25T12:34:59.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.729+0000 7f26c1c00640 1 -- start start 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.731+0000 7f26c1c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26bc0c6ee0 0x7f26bc156730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.731+0000 7f26c1c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26bc0c7c80 0x7f26bc0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.731+0000 7f26c1c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26bc156c70 con 0x7f26bc0c65c0 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.731+0000 7f26c1c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f26bc0c77f0 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.731+0000 7f26c1c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f26bc0c85d0 con 0x7f26bc0c7c80 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26bbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26bc0c6ee0 0x7f26bc156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26bbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26bc0c6ee0 0x7f26bc156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39672/0 (socket says 172.21.15.123:39672) 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26bbe00640 1 -- 172.21.15.123:0/1672365550 learned_addr learned my addr 172.21.15.123:0/1672365550 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:59.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26c1200640 1 --2- 172.21.15.123:0/1672365550 >> v2:172.21.15.153:3300/0 conn(0x7f26bc0c7c80 0x7f26bc0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26bbe00640 1 -- 172.21.15.123:0/1672365550 >> v2:172.21.15.153:3300/0 conn(0x7f26bc0c7c80 msgr2=0x7f26bc0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:34:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26bbe00640 1 --2- 172.21.15.123:0/1672365550 >> v2:172.21.15.153:3300/0 conn(0x7f26bc0c7c80 0x7f26bc0c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26bbe00640 1 -- 172.21.15.123:0/1672365550 >> v1:172.21.15.123:6789/0 conn(0x7f26bc0c65c0 legacy=0x7f26bc0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26bbe00640 1 -- 172.21.15.123:0/1672365550 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc15c960 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.732+0000 7f26baa00640 1 -- 172.21.15.123:0/1672365550 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3627057919 0 0) 0x7f26bc156c70 con 0x7f26bc0c65c0 2024-04-25T12:34:59.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.734+0000 7f26bbe00640 1 --2- 172.21.15.123:0/1672365550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc156730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0cf240 tx=0x7f26ac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd90440edec544f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:59.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.734+0000 7f26baa00640 1 -- 172.21.15.123:0/1672365550 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac037070 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.734+0000 7f26baa00640 1 -- 172.21.15.123:0/1672365550 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ac02ae50 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.734+0000 7f26baa00640 1 -- 172.21.15.123:0/1672365550 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac032860 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 -- 172.21.15.123:0/1672365550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 msgr2=0x7f26bc156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1672365550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc156730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0cf240 tx=0x7f26ac02a300 comp rx=0 tx=0).stop 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 -- 172.21.15.123:0/1672365550 shutdown_connections 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1672365550 >> v2:172.21.15.153:3300/0 conn(0x7f26bc0c7c80 0x7f26bc0c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1672365550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc156730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 -- 172.21.15.123:0/1672365550 >> 172.21.15.123:0/1672365550 conn(0x7f26bc06dbe0 msgr2=0x7f26bc06dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 -- 172.21.15.123:0/1672365550 shutdown_connections 2024-04-25T12:34:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.735+0000 7f26c1c00640 1 -- 172.21.15.123:0/1672365550 wait complete. 2024-04-25T12:34:59.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.739+0000 7f26c1c00640 1 Processor -- start 2024-04-25T12:34:59.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.739+0000 7f26c1c00640 1 -- start start 2024-04-25T12:34:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.741+0000 7f26c1c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26bc0c65c0 0x7f26bc1f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.741+0000 7f26c1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc0d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.741+0000 7f26c1c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26bc0c7c80 0x7f26bc1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.741+0000 7f26c1c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc15d200 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.741+0000 7f26c1c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc15cf20 con 0x7f26bc0c65c0 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.741+0000 7f26c1c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f26bc0c8b20 con 0x7f26bc0c7c80 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc0d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc0d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39676/0 (socket says 172.21.15.123:39676) 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 -- 172.21.15.123:0/1936051251 learned_addr learned my addr 172.21.15.123:0/1936051251 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26c1200640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26bc0c7c80 0x7f26bc1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bbe00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26bc0c65c0 0x7f26bc1f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 -- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26bc0c7c80 msgr2=0x7f26bc1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26bc0c7c80 0x7f26bc1500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 -- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26bc0c65c0 msgr2=0x7f26bc1f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26bc0c65c0 0x7f26bc1f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 -- 172.21.15.123:0/1936051251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc1f2ca0 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.742+0000 7f26bb400640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc0d0de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c410 tx=0x7f26a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.743+0000 7f26b3e00640 1 -- 172.21.15.123:0/1936051251 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8017070 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.743+0000 7f26b3e00640 1 -- 172.21.15.123:0/1936051251 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a80035f0 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.743+0000 7f26b3e00640 1 -- 172.21.15.123:0/1936051251 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8003a00 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.743+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc155e90 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.743+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc1f5fe0 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.744+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26bc076da0 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.745+0000 7f26b3e00640 1 -- 172.21.15.123:0/1936051251 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f26a801a070 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.746+0000 7f26b3e00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f269c0744b0 0x7f269c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:34:59.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.746+0000 7f26b3e00640 1 -- 172.21.15.123:0/1936051251 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6590+0+0 (secure 0 0 0) 0x7f26a8094980 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.746+0000 7f26bbe00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f269c0744b0 0x7f269c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:34:59.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.747+0000 7f26bbe00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f269c0744b0 0x7f269c0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26ac034000 tx=0x7f26ac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:34:59.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.751+0000 7f26b3e00640 1 -- 172.21.15.123:0/1936051251 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26a805e6b0 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.962+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26bc151d40 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.965+0000 7f26b3e00640 1 -- 172.21.15.123:0/1936051251 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18673 (secure 0 0 0) 0x7f26a80629b0 con 0x7f26bc0c6ee0 2024-04-25T12:34:59.966 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:34:59.966 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:34:59.221520+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:34:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f269c0744b0 msgr2=0x7f269c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f269c0744b0 0x7f269c0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26ac034000 tx=0x7f26ac02e000 comp rx=0 tx=0).stop 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 msgr2=0x7f26bc0d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc0d0de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c410 tx=0x7f26a800c8d0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 shutdown_connections 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f269c0744b0 0x7f269c0769c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26bc0c7c80 0x7f26bc1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26bc0c6ee0 0x7f26bc0d0de0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 --2- 172.21.15.123:0/1936051251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26bc0c65c0 0x7f26bc1f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 >> 172.21.15.123:0/1936051251 conn(0x7f26bc06dbe0 msgr2=0x7f26bc0cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 shutdown_connections 2024-04-25T12:34:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:34:59.968+0000 7f26c1c00640 1 -- 172.21.15.123:0/1936051251 wait complete. 2024-04-25T12:34:59.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T12:34:59.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:35:00.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.254+0000 7fa881600640 1 Processor -- start 2024-04-25T12:35:00.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.254+0000 7fa881600640 1 -- start start 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.256+0000 7fa881600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa87c0ca510 0x7fa87c15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.256+0000 7fa881600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c54a0 0x7fa87c166d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa881600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c06dde0 con 0x7fa87c0c5090 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa881600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa87c15d490 con 0x7fa87c0ca510 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa881600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c06e9d0 con 0x7fa87c0c54a0 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa87c0ca510 0x7fa87c15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa87c0ca510 0x7fa87c15cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39692/0 (socket says 172.21.15.123:39692) 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87be00640 1 -- 172.21.15.123:0/1743218809 learned_addr learned my addr 172.21.15.123:0/1743218809 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87b400640 1 --1- 172.21.15.123:0/1743218809 >> v1:172.21.15.123:6789/0 conn(0x7fa87c0c5090 0x7fa87c0cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45886/0 (socket says 172.21.15.123:45886) 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa880c00640 1 --2- 172.21.15.123:0/1743218809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c54a0 0x7fa87c166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87be00640 1 -- 172.21.15.123:0/1743218809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c54a0 msgr2=0x7fa87c166d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87be00640 1 --2- 172.21.15.123:0/1743218809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c54a0 0x7fa87c166d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87be00640 1 -- 172.21.15.123:0/1743218809 >> v1:172.21.15.123:6789/0 conn(0x7fa87c0c5090 legacy=0x7fa87c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87aa00640 1 -- 172.21.15.123:0/1743218809 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836779871 0 0) 0x7fa87c06dde0 con 0x7fa87c0c5090 2024-04-25T12:35:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa87be00640 1 -- 172.21.15.123:0/1743218809 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c0cab00 con 0x7fa87c0ca510 2024-04-25T12:35:00.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.257+0000 7fa880c00640 1 --2- 172.21.15.123:0/1743218809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c54a0 0x7fa87c166d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.258+0000 7fa87be00640 1 --2- 172.21.15.123:0/1743218809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c15cf50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03ac70324d76589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa87aa00640 1 -- 172.21.15.123:0/1743218809 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c037070 con 0x7fa87c0ca510 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa87aa00640 1 -- 172.21.15.123:0/1743218809 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c032640 con 0x7fa87c0ca510 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa87aa00640 1 -- 172.21.15.123:0/1743218809 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c032a10 con 0x7fa87c0ca510 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 -- 172.21.15.123:0/1743218809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 msgr2=0x7fa87c15cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 --2- 172.21.15.123:0/1743218809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c15cf50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02a2e0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 -- 172.21.15.123:0/1743218809 shutdown_connections 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 --2- 172.21.15.123:0/1743218809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c54a0 0x7fa87c166d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 --2- 172.21.15.123:0/1743218809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c15cf50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 -- 172.21.15.123:0/1743218809 >> 172.21.15.123:0/1743218809 conn(0x7fa87c06cba0 msgr2=0x7fa87c068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:00.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 -- 172.21.15.123:0/1743218809 shutdown_connections 2024-04-25T12:35:00.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.259+0000 7fa881600640 1 -- 172.21.15.123:0/1743218809 wait complete. 2024-04-25T12:35:00.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.262+0000 7fa881600640 1 Processor -- start 2024-04-25T12:35:00.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.262+0000 7fa881600640 1 -- start start 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.263+0000 7fa881600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c5090 0x7fa87c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.263+0000 7fa881600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa87c0c54a0 0x7fa87c151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.263+0000 7fa881600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.263+0000 7fa881600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c06e380 con 0x7fa87c0ca510 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.263+0000 7fa881600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0c58b0 con 0x7fa87c0c54a0 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.263+0000 7fa881600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c06e660 con 0x7fa87c0c5090 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.263+0000 7fa880c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c14abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39700/0 (socket says 172.21.15.123:39700) 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa87b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa87c0c54a0 0x7fa87c151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 -- 172.21.15.123:0/1163184168 learned_addr learned my addr 172.21.15.123:0/1163184168 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:00.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa87be00640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c5090 0x7fa87c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 -- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c5090 msgr2=0x7fa87c157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c5090 0x7fa87c157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 -- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa87c0c54a0 msgr2=0x7fa87c151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa87c0c54a0 0x7fa87c151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 -- 172.21.15.123:0/1163184168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c158430 con 0x7fa87c0ca510 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa880c00640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c14abe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa874007e60 tx=0x7fa87400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.264+0000 7fa873e00640 1 -- 172.21.15.123:0/1163184168 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87401a070 con 0x7fa87c0ca510 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.265+0000 7fa873e00640 1 -- 172.21.15.123:0/1163184168 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa874015400 con 0x7fa87c0ca510 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.265+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c159630 con 0x7fa87c0ca510 2024-04-25T12:35:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.265+0000 7fa873e00640 1 -- 172.21.15.123:0/1163184168 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740157d0 con 0x7fa87c0ca510 2024-04-25T12:35:00.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.265+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c159bf0 con 0x7fa87c0ca510 2024-04-25T12:35:00.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.266+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87c076da0 con 0x7fa87c0ca510 2024-04-25T12:35:00.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.267+0000 7fa873e00640 1 -- 172.21.15.123:0/1163184168 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa87401d070 con 0x7fa87c0ca510 2024-04-25T12:35:00.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.268+0000 7fa873e00640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa85c0744b0 0x7fa85c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.268+0000 7fa873e00640 1 -- 172.21.15.123:0/1163184168 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fa8740956e0 con 0x7fa87c0ca510 2024-04-25T12:35:00.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.268+0000 7fa87be00640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa85c0744b0 0x7fa85c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.269+0000 7fa87be00640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa85c0744b0 0x7fa85c0769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034000 tx=0x7fa86c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:00.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.273+0000 7fa873e00640 1 -- 172.21.15.123:0/1163184168 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa874010070 con 0x7fa87c0ca510 2024-04-25T12:35:00.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.458+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa87c06d9b0 con 0x7fa87c0ca510 2024-04-25T12:35:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.459+0000 7fa873e00640 1 -- 172.21.15.123:0/1163184168 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fa87405f530 con 0x7fa87c0ca510 2024-04-25T12:35:00.459 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa85c0744b0 msgr2=0x7fa85c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa85c0744b0 0x7fa85c0769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034000 tx=0x7fa86c02e000 comp rx=0 tx=0).stop 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 msgr2=0x7fa87c14abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c14abe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa874007e60 tx=0x7fa87400d4a0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 shutdown_connections 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa85c0744b0 0x7fa85c0769c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa87c0ca510 0x7fa87c14abe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa87c0c54a0 0x7fa87c151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 --2- 172.21.15.123:0/1163184168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa87c0c5090 0x7fa87c157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 >> 172.21.15.123:0/1163184168 conn(0x7fa87c06cba0 msgr2=0x7fa87c1555b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 shutdown_connections 2024-04-25T12:35:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.461+0000 7fa881600640 1 -- 172.21.15.123:0/1163184168 wait complete. 2024-04-25T12:35:00.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:35:00.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.753+0000 7fcb0d000640 1 Processor -- start 2024-04-25T12:35:00.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.753+0000 7fcb0d000640 1 -- start start 2024-04-25T12:35:00.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.755+0000 7fcb0d000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb080c7dc0 0x7fcb081565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb0d000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb080c81d0 con 0x7fcb080c91c0 2024-04-25T12:35:00.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb0d000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb080c8450 con 0x7fcb080c2720 2024-04-25T12:35:00.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb0d000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcb08159310 con 0x7fcb080c7dc0 2024-04-25T12:35:00.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb07e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcb080c91c0 0x7fcb080c95d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45894/0 (socket says 172.21.15.123:45894) 2024-04-25T12:35:00.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb07e00640 1 -- 172.21.15.123:0/1679698723 learned_addr learned my addr 172.21.15.123:0/1679698723 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:00.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb07400640 1 --2- 172.21.15.123:0/1679698723 >> v2:172.21.15.153:3300/0 conn(0x7fcb080c7dc0 0x7fcb081565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb06a00640 1 -- 172.21.15.123:0/1679698723 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030973181 0 0) 0x7fcb00001490 con 0x7fcb080c91c0 2024-04-25T12:35:00.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb06a00640 1 -- 172.21.15.123:0/1679698723 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaec003520 con 0x7fcb080c91c0 2024-04-25T12:35:00.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb06a00640 1 -- 172.21.15.123:0/1679698723 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177386081 0 0) 0x7fcb080c8450 con 0x7fcb080c2720 2024-04-25T12:35:00.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.756+0000 7fcb06a00640 1 -- 172.21.15.123:0/1679698723 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb00001490 con 0x7fcb080c2720 2024-04-25T12:35:00.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.757+0000 7fcb07400640 1 -- 172.21.15.123:0/1679698723 >> v1:172.21.15.153:6789/0 conn(0x7fcb080c2720 legacy=0x7fcb0806c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.757+0000 7fcb07400640 1 -- 172.21.15.123:0/1679698723 >> v1:172.21.15.123:6789/0 conn(0x7fcb080c91c0 legacy=0x7fcb080c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.757+0000 7fcb07400640 1 -- 172.21.15.123:0/1679698723 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb081595f0 con 0x7fcb080c7dc0 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.758+0000 7fcb07400640 1 --2- 172.21.15.123:0/1679698723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c7dc0 0x7fcb081565f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80097a0 tx=0x7fcaf802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3048c4e431ca4abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.758+0000 7fcb06a00640 1 -- 172.21.15.123:0/1679698723 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8037070 con 0x7fcb080c7dc0 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.758+0000 7fcb06a00640 1 -- 172.21.15.123:0/1679698723 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf8032640 con 0x7fcb080c7dc0 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.758+0000 7fcb06a00640 1 -- 172.21.15.123:0/1679698723 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80329f0 con 0x7fcb080c7dc0 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.759+0000 7fcb0d000640 1 -- 172.21.15.123:0/1679698723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c7dc0 msgr2=0x7fcb081565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.759+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1679698723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c7dc0 0x7fcb081565f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80097a0 tx=0x7fcaf802a2c0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.759+0000 7fcb0d000640 1 -- 172.21.15.123:0/1679698723 shutdown_connections 2024-04-25T12:35:00.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.759+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1679698723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c7dc0 0x7fcb081565f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.759+0000 7fcb0d000640 1 -- 172.21.15.123:0/1679698723 >> 172.21.15.123:0/1679698723 conn(0x7fcb08070b70 msgr2=0x7fcb08070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:00.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.759+0000 7fcb0d000640 1 -- 172.21.15.123:0/1679698723 shutdown_connections 2024-04-25T12:35:00.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.759+0000 7fcb0d000640 1 -- 172.21.15.123:0/1679698723 wait complete. 2024-04-25T12:35:00.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.761+0000 7fcb0d000640 1 Processor -- start 2024-04-25T12:35:00.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.761+0000 7fcb0d000640 1 -- start start 2024-04-25T12:35:00.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb0d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c2720 0x7fcb0814dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb0d000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb080c7dc0 0x7fcb080c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb0d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb080c91c0 0x7fcb08150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb0d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb080c4cb0 con 0x7fcb080c91c0 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb0d000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08159bb0 con 0x7fcb080c2720 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb0d000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcb08158fc0 con 0x7fcb080c7dc0 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb07e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb080c91c0 0x7fcb08150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb07e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb080c91c0 0x7fcb08150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39710/0 (socket says 172.21.15.123:39710) 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb07e00640 1 -- 172.21.15.123:0/1887030705 learned_addr learned my addr 172.21.15.123:0/1887030705 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcb07400640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c2720 0x7fcb0814dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.763+0000 7fcaffe00640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb080c7dc0 0x7fcb080c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb07e00640 1 -- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb080c7dc0 msgr2=0x7fcb080c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb07e00640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb080c7dc0 0x7fcb080c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb07e00640 1 -- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c2720 msgr2=0x7fcb0814dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb07e00640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c2720 0x7fcb0814dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb07e00640 1 -- 172.21.15.123:0/1887030705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb080c6180 con 0x7fcb080c91c0 2024-04-25T12:35:00.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb07400640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c2720 0x7fcb0814dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:00.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb07e00640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb080c91c0 0x7fcb08150fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb00001ed0 tx=0x7fcb0000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:00.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcaff400640 1 -- 172.21.15.123:0/1887030705 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00017070 con 0x7fcb080c91c0 2024-04-25T12:35:00.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcaff400640 1 -- 172.21.15.123:0/1887030705 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb00012400 con 0x7fcb080c91c0 2024-04-25T12:35:00.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb08152570 con 0x7fcb080c91c0 2024-04-25T12:35:00.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcaff400640 1 -- 172.21.15.123:0/1887030705 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb000127f0 con 0x7fcb080c91c0 2024-04-25T12:35:00.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.764+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb080c5160 con 0x7fcb080c91c0 2024-04-25T12:35:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.766+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb08076da0 con 0x7fcb080c91c0 2024-04-25T12:35:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.767+0000 7fcaff400640 1 -- 172.21.15.123:0/1887030705 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb0001a070 con 0x7fcb080c91c0 2024-04-25T12:35:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.768+0000 7fcaff400640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcae40744b0 0x7fcae40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.768+0000 7fcaff400640 1 -- 172.21.15.123:0/1887030705 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fcb00095360 con 0x7fcb080c91c0 2024-04-25T12:35:00.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.768+0000 7fcb07400640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcae40744b0 0x7fcae40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:00.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.769+0000 7fcb07400640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcae40744b0 0x7fcae40769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8034000 tx=0x7fcaf802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:00.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.772+0000 7fcaff400640 1 -- 172.21.15.123:0/1887030705 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb080c81d0 con 0x7fcb080c91c0 2024-04-25T12:35:00.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.965+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcb080c8d90 con 0x7fcb080c91c0 2024-04-25T12:35:00.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.967+0000 7fcaff400640 1 -- 172.21.15.123:0/1887030705 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fcb0005f1b0 con 0x7fcb080c91c0 2024-04-25T12:35:00.967 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:35:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.969+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcae40744b0 msgr2=0x7fcae40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.969+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcae40744b0 0x7fcae40769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8034000 tx=0x7fcaf802e000 comp rx=0 tx=0).stop 2024-04-25T12:35:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.969+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb080c91c0 msgr2=0x7fcb08150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.969+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb080c91c0 0x7fcb08150fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb00001ed0 tx=0x7fcb0000a430 comp rx=0 tx=0).stop 2024-04-25T12:35:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 shutdown_connections 2024-04-25T12:35:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcae40744b0 0x7fcae40769c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb080c91c0 0x7fcb08150fd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb080c7dc0 0x7fcb080c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 --2- 172.21.15.123:0/1887030705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb080c2720 0x7fcb0814dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 >> 172.21.15.123:0/1887030705 conn(0x7fcb08070b70 msgr2=0x7fcb08072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 shutdown_connections 2024-04-25T12:35:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:00.970+0000 7fcb0d000640 1 -- 172.21.15.123:0/1887030705 wait complete. 2024-04-25T12:35:00.996 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:35:01.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.260+0000 7fa285000640 1 Processor -- start 2024-04-25T12:35:01.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.260+0000 7fa285000640 1 -- start start 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.262+0000 7fa285000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2800b3be0 0x7fa2800cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.262+0000 7fa285000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa280071350 0x7fa2800cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.262+0000 7fa285000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa28015fcd0 con 0x7fa28006fec0 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.262+0000 7fa285000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa2801608e0 con 0x7fa280071350 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.262+0000 7fa285000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa28015ffb0 con 0x7fa2800b3be0 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27e800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa28006fec0 0x7fa2800c47c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45910/0 (socket says 172.21.15.123:45910) 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27e800640 1 -- 172.21.15.123:0/2427153088 learned_addr learned my addr 172.21.15.123:0/2427153088 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa275400640 1 --2- 172.21.15.123:0/2427153088 >> v2:172.21.15.153:3300/0 conn(0x7fa280071350 0x7fa2800cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27de00640 1 --2- 172.21.15.123:0/2427153088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2800b3be0 0x7fa2800cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27d400640 1 -- 172.21.15.123:0/2427153088 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823311150 0 0) 0x7fa28015fcd0 con 0x7fa28006fec0 2024-04-25T12:35:01.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27d400640 1 -- 172.21.15.123:0/2427153088 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa264003520 con 0x7fa28006fec0 2024-04-25T12:35:01.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27de00640 1 -- 172.21.15.123:0/2427153088 >> v2:172.21.15.153:3300/0 conn(0x7fa280071350 msgr2=0x7fa2800cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:01.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27de00640 1 --2- 172.21.15.123:0/2427153088 >> v2:172.21.15.153:3300/0 conn(0x7fa280071350 0x7fa2800cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27de00640 1 -- 172.21.15.123:0/2427153088 >> v1:172.21.15.123:6789/0 conn(0x7fa28006fec0 legacy=0x7fa2800c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa27de00640 1 -- 172.21.15.123:0/2427153088 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa280160290 con 0x7fa2800b3be0 2024-04-25T12:35:01.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.263+0000 7fa275400640 1 --2- 172.21.15.123:0/2427153088 >> v2:172.21.15.153:3300/0 conn(0x7fa280071350 0x7fa2800cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:01.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.265+0000 7fa27de00640 1 --2- 172.21.15.123:0/2427153088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2800b3be0 0x7fa2800cb0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2700097a0 tx=0x7fa27002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c521642d11ed56b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:01.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.265+0000 7fa27d400640 1 -- 172.21.15.123:0/2427153088 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270037070 con 0x7fa2800b3be0 2024-04-25T12:35:01.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.265+0000 7fa27d400640 1 -- 172.21.15.123:0/2427153088 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa270032440 con 0x7fa2800b3be0 2024-04-25T12:35:01.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.265+0000 7fa27d400640 1 -- 172.21.15.123:0/2427153088 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270032810 con 0x7fa2800b3be0 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 -- 172.21.15.123:0/2427153088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2800b3be0 msgr2=0x7fa2800cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 --2- 172.21.15.123:0/2427153088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2800b3be0 0x7fa2800cb0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2700097a0 tx=0x7fa27002a210 comp rx=0 tx=0).stop 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 -- 172.21.15.123:0/2427153088 shutdown_connections 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 --2- 172.21.15.123:0/2427153088 >> v2:172.21.15.153:3300/0 conn(0x7fa280071350 0x7fa2800cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 --2- 172.21.15.123:0/2427153088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2800b3be0 0x7fa2800cb0d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 -- 172.21.15.123:0/2427153088 >> 172.21.15.123:0/2427153088 conn(0x7fa28006dd90 msgr2=0x7fa280071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 -- 172.21.15.123:0/2427153088 shutdown_connections 2024-04-25T12:35:01.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.266+0000 7fa285000640 1 -- 172.21.15.123:0/2427153088 wait complete. 2024-04-25T12:35:01.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.269+0000 7fa285000640 1 Processor -- start 2024-04-25T12:35:01.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.269+0000 7fa285000640 1 -- start start 2024-04-25T12:35:01.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa285000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28006fec0 0x7fa2801f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa285000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa280071350 0x7fa280158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa285000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2800b3be0 0x7fa280151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa285000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa28006f720 con 0x7fa2800b3be0 2024-04-25T12:35:01.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa285000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa28015d580 con 0x7fa28006fec0 2024-04-25T12:35:01.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa285000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa28015d140 con 0x7fa280071350 2024-04-25T12:35:01.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa27e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2800b3be0 0x7fa280151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa27de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28006fec0 0x7fa2801f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa275400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa280071350 0x7fa280158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa27e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2800b3be0 0x7fa280151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39720/0 (socket says 172.21.15.123:39720) 2024-04-25T12:35:01.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa27e800640 1 -- 172.21.15.123:0/4263237541 learned_addr learned my addr 172.21.15.123:0/4263237541 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa27de00640 1 -- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa280071350 msgr2=0x7fa280158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.271+0000 7fa27de00640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa280071350 0x7fa280158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa27de00640 1 -- 172.21.15.123:0/4263237541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2800b3be0 msgr2=0x7fa280151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa27de00640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2800b3be0 0x7fa280151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa27de00640 1 -- 172.21.15.123:0/4263237541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2801f3110 con 0x7fa28006fec0 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa27e800640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2800b3be0 0x7fa280151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa27de00640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28006fec0 0x7fa2801f2bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa270031000 tx=0x7fa2700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa276a00640 1 -- 172.21.15.123:0/4263237541 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270041030 con 0x7fa28006fec0 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa276a00640 1 -- 172.21.15.123:0/4263237541 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27003d4e0 con 0x7fa28006fec0 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa276a00640 1 -- 172.21.15.123:0/4263237541 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270037040 con 0x7fa28006fec0 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2801f26d0 con 0x7fa28006fec0 2024-04-25T12:35:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.272+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa280156f10 con 0x7fa28006fec0 2024-04-25T12:35:01.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.273+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa280076700 con 0x7fa28006fec0 2024-04-25T12:35:01.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.274+0000 7fa276a00640 1 -- 172.21.15.123:0/4263237541 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa270044070 con 0x7fa28006fec0 2024-04-25T12:35:01.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.275+0000 7fa276a00640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa25c0744b0 0x7fa25c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.276+0000 7fa276a00640 1 -- 172.21.15.123:0/4263237541 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fa2700b5a50 con 0x7fa28006fec0 2024-04-25T12:35:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.276+0000 7fa275400640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa25c0744b0 0x7fa25c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.276+0000 7fa275400640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa25c0744b0 0x7fa25c0769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa28015b4b0 tx=0x7fa268009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:01.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.281+0000 7fa276a00640 1 -- 172.21.15.123:0/4263237541 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa27007f8a0 con 0x7fa28006fec0 2024-04-25T12:35:01.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.481+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2800ca610 con 0x7fa28006fec0 2024-04-25T12:35:01.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.482+0000 7fa276a00640 1 -- 172.21.15.123:0/4263237541 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa270083ba0 con 0x7fa28006fec0 2024-04-25T12:35:01.482 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T12:35:01.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa25c0744b0 msgr2=0x7fa25c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa25c0744b0 0x7fa25c0769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa28015b4b0 tx=0x7fa268009190 comp rx=0 tx=0).stop 2024-04-25T12:35:01.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28006fec0 msgr2=0x7fa2801f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28006fec0 0x7fa2801f2bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa270031000 tx=0x7fa2700046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 shutdown_connections 2024-04-25T12:35:01.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa25c0744b0 0x7fa25c0769c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2800b3be0 0x7fa280151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa280071350 0x7fa280158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 --2- 172.21.15.123:0/4263237541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28006fec0 0x7fa2801f2bd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.484+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 >> 172.21.15.123:0/4263237541 conn(0x7fa28006dd90 msgr2=0x7fa28015ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:01.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.485+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 shutdown_connections 2024-04-25T12:35:01.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.485+0000 7fa285000640 1 -- 172.21.15.123:0/4263237541 wait complete. 2024-04-25T12:35:01.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T12:35:01.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.790+0000 7fce94600640 1 Processor -- start 2024-04-25T12:35:01.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.791+0000 7fce94600640 1 -- start start 2024-04-25T12:35:01.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce94600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce8c161470 0x7fce8c0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce94600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c16b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce94600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce8c0ca9b0 con 0x7fce8c15aca0 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce94600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fce8c074170 con 0x7fce8c161470 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce94600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fce8c073230 con 0x7fce8c0ca5a0 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce91200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce8c161470 0x7fce8c0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce91c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fce8c15aca0 0x7fce8c0c3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45918/0 (socket says 172.21.15.123:45918) 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce91c00640 1 -- 172.21.15.123:0/3862573056 learned_addr learned my addr 172.21.15.123:0/3862573056 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:01.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.793+0000 7fce92600640 1 --2- 172.21.15.123:0/3862573056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c16b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.794+0000 7fce8be00640 1 -- 172.21.15.123:0/3862573056 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 26607541 0 0) 0x7fce8c0ca9b0 con 0x7fce8c15aca0 2024-04-25T12:35:01.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.794+0000 7fce8be00640 1 -- 172.21.15.123:0/3862573056 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce74003520 con 0x7fce8c15aca0 2024-04-25T12:35:01.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.794+0000 7fce91200640 1 -- 172.21.15.123:0/3862573056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 msgr2=0x7fce8c16b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:01.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.794+0000 7fce91200640 1 --2- 172.21.15.123:0/3862573056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c16b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.794+0000 7fce91200640 1 -- 172.21.15.123:0/3862573056 >> v1:172.21.15.123:6789/0 conn(0x7fce8c15aca0 legacy=0x7fce8c0c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.794+0000 7fce91200640 1 -- 172.21.15.123:0/3862573056 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c15b0b0 con 0x7fce8c161470 2024-04-25T12:35:01.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.794+0000 7fce92600640 1 --2- 172.21.15.123:0/3862573056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c16b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.795+0000 7fce91200640 1 --2- 172.21.15.123:0/3862573056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c161470 0x7fce8c0ca060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce80009d90 tx=0x7fce8002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a603a835dc397d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce8be00640 1 -- 172.21.15.123:0/3862573056 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80007740 con 0x7fce8c161470 2024-04-25T12:35:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce8be00640 1 -- 172.21.15.123:0/3862573056 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce800078e0 con 0x7fce8c161470 2024-04-25T12:35:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce8be00640 1 -- 172.21.15.123:0/3862573056 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80007cb0 con 0x7fce8c161470 2024-04-25T12:35:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce94600640 1 -- 172.21.15.123:0/3862573056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c161470 msgr2=0x7fce8c0ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce94600640 1 --2- 172.21.15.123:0/3862573056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c161470 0x7fce8c0ca060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce80009d90 tx=0x7fce8002a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce94600640 1 -- 172.21.15.123:0/3862573056 shutdown_connections 2024-04-25T12:35:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce94600640 1 --2- 172.21.15.123:0/3862573056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c16b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce94600640 1 --2- 172.21.15.123:0/3862573056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c161470 0x7fce8c0ca060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.796+0000 7fce94600640 1 -- 172.21.15.123:0/3862573056 >> 172.21.15.123:0/3862573056 conn(0x7fce8c06f440 msgr2=0x7fce8c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:01.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.797+0000 7fce94600640 1 -- 172.21.15.123:0/3862573056 shutdown_connections 2024-04-25T12:35:01.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.797+0000 7fce94600640 1 -- 172.21.15.123:0/3862573056 wait complete. 2024-04-25T12:35:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.799+0000 7fce94600640 1 Processor -- start 2024-04-25T12:35:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.799+0000 7fce94600640 1 -- start start 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.800+0000 7fce94600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce94600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c15aca0 0x7fce8c1fac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce94600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce8c161470 0x7fce8c156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce94600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c073760 con 0x7fce8c15aca0 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce94600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c071220 con 0x7fce8c161470 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce94600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fce8c0708f0 con 0x7fce8c0ca5a0 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c15aca0 0x7fce8c1fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c15aca0 0x7fce8c1fac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39740/0 (socket says 172.21.15.123:39740) 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 -- 172.21.15.123:0/2558720044 learned_addr learned my addr 172.21.15.123:0/2558720044 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91c00640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce92600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce8c161470 0x7fce8c156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 -- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 msgr2=0x7fce8c0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c0ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 -- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce8c161470 msgr2=0x7fce8c156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce8c161470 0x7fce8c156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.801+0000 7fce91200640 1 -- 172.21.15.123:0/2558720044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c1fb1b0 con 0x7fce8c15aca0 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.802+0000 7fce91200640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c15aca0 0x7fce8c1fac70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce80009eb0 tx=0x7fce80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.802+0000 7fce89600640 1 -- 172.21.15.123:0/2558720044 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8003f070 con 0x7fce8c15aca0 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.802+0000 7fce89600640 1 -- 172.21.15.123:0/2558720044 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce80031c40 con 0x7fce8c15aca0 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.802+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8c156090 con 0x7fce8c15aca0 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.802+0000 7fce89600640 1 -- 172.21.15.123:0/2558720044 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80009450 con 0x7fce8c15aca0 2024-04-25T12:35:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.802+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8c156350 con 0x7fce8c15aca0 2024-04-25T12:35:01.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.803+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8c1f7cd0 con 0x7fce8c15aca0 2024-04-25T12:35:01.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.804+0000 7fce89600640 1 -- 172.21.15.123:0/2558720044 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce80044460 con 0x7fce8c15aca0 2024-04-25T12:35:01.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.807+0000 7fce89600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce700744b0 0x7fce700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:01.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.807+0000 7fce89600640 1 -- 172.21.15.123:0/2558720044 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fce800b6be0 con 0x7fce8c15aca0 2024-04-25T12:35:01.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.807+0000 7fce91c00640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce700744b0 0x7fce700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:01.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.808+0000 7fce91c00640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce700744b0 0x7fce700769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005880 tx=0x7fce7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:01.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:01.811+0000 7fce89600640 1 -- 172.21.15.123:0/2558720044 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce80080ff0 con 0x7fce8c15aca0 2024-04-25T12:35:02.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.004+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fce8c076700 con 0x7fce8c15aca0 2024-04-25T12:35:02.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.005+0000 7fce89600640 1 -- 172.21.15.123:0/2558720044 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fce800808f0 con 0x7fce8c15aca0 2024-04-25T12:35:02.005 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce700744b0 msgr2=0x7fce700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce700744b0 0x7fce700769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005880 tx=0x7fce7c009040 comp rx=0 tx=0).stop 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c15aca0 msgr2=0x7fce8c1fac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c15aca0 0x7fce8c1fac70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce80009eb0 tx=0x7fce80004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 shutdown_connections 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fce700744b0 0x7fce700769c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce8c161470 0x7fce8c156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8c15aca0 0x7fce8c1fac70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 --2- 172.21.15.123:0/2558720044 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce8c0ca5a0 0x7fce8c0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.007+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 >> 172.21.15.123:0/2558720044 conn(0x7fce8c06f440 msgr2=0x7fce8c15e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:02.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.008+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 shutdown_connections 2024-04-25T12:35:02.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.008+0000 7fce94600640 1 -- 172.21.15.123:0/2558720044 wait complete. 2024-04-25T12:35:02.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:35:02.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.295+0000 7fe8d6800640 1 Processor -- start 2024-04-25T12:35:02.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.295+0000 7fe8d6800640 1 -- start start 2024-04-25T12:35:02.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.297+0000 7fe8d6800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8d0163ec0 0x7fe8d015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.297+0000 7fe8d6800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d0164390 con 0x7fe8d00c8930 2024-04-25T12:35:02.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.297+0000 7fe8d6800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d0164630 con 0x7fe8d00c8520 2024-04-25T12:35:02.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.297+0000 7fe8d6800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d0164e70 con 0x7fe8d0163ec0 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8d5400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8d0163ec0 0x7fe8d015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8d5e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe8d00c8930 0x7fe8d006c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45920/0 (socket says 172.21.15.123:45920) 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8d5400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8d0163ec0 0x7fe8d015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39756/0 (socket says 172.21.15.123:39756) 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8d5e00640 1 -- 172.21.15.123:0/3029754692 learned_addr learned my addr 172.21.15.123:0/3029754692 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8cfe00640 1 -- 172.21.15.123:0/3029754692 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430925551 0 0) 0x7fe8d0164390 con 0x7fe8d00c8930 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8cfe00640 1 -- 172.21.15.123:0/3029754692 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8b8003520 con 0x7fe8d00c8930 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8d5400640 1 -- 172.21.15.123:0/3029754692 >> v1:172.21.15.153:6789/0 conn(0x7fe8d00c8520 legacy=0x7fe8d0161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.298+0000 7fe8d5400640 1 -- 172.21.15.123:0/3029754692 >> v1:172.21.15.123:6789/0 conn(0x7fe8d00c8930 legacy=0x7fe8d006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.299+0000 7fe8d5400640 1 -- 172.21.15.123:0/3029754692 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d01648d0 con 0x7fe8d0163ec0 2024-04-25T12:35:02.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.300+0000 7fe8d5400640 1 --2- 172.21.15.123:0/3029754692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d015ac10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40097a0 tx=0x7fe8c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e80f2e22da46e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8cfe00640 1 -- 172.21.15.123:0/3029754692 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4037070 con 0x7fe8d0163ec0 2024-04-25T12:35:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8cfe00640 1 -- 172.21.15.123:0/3029754692 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4032440 con 0x7fe8d0163ec0 2024-04-25T12:35:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8cfe00640 1 -- 172.21.15.123:0/3029754692 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4032810 con 0x7fe8d0163ec0 2024-04-25T12:35:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8d6800640 1 -- 172.21.15.123:0/3029754692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 msgr2=0x7fe8d015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3029754692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d015ac10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40097a0 tx=0x7fe8c402a230 comp rx=0 tx=0).stop 2024-04-25T12:35:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8d6800640 1 -- 172.21.15.123:0/3029754692 shutdown_connections 2024-04-25T12:35:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3029754692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d015ac10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.301+0000 7fe8d6800640 1 -- 172.21.15.123:0/3029754692 >> 172.21.15.123:0/3029754692 conn(0x7fe8d00cd600 msgr2=0x7fe8d00cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.302+0000 7fe8d6800640 1 -- 172.21.15.123:0/3029754692 shutdown_connections 2024-04-25T12:35:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.302+0000 7fe8d6800640 1 -- 172.21.15.123:0/3029754692 wait complete. 2024-04-25T12:35:02.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.304+0000 7fe8d6800640 1 Processor -- start 2024-04-25T12:35:02.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.304+0000 7fe8d6800640 1 -- start start 2024-04-25T12:35:02.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.305+0000 7fe8d6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d00c8520 0x7fe8d0158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.305+0000 7fe8d6800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8d00c8930 0x7fe8d01ed690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.305+0000 7fe8d6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.305+0000 7fe8d6800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d00c9230 con 0x7fe8d0163ec0 2024-04-25T12:35:02.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.305+0000 7fe8d6800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d0160b70 con 0x7fe8d00c8520 2024-04-25T12:35:02.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.305+0000 7fe8d6800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d00c93b0 con 0x7fe8d00c8930 2024-04-25T12:35:02.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39762/0 (socket says 172.21.15.123:39762) 2024-04-25T12:35:02.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 -- 172.21.15.123:0/3358722348 learned_addr learned my addr 172.21.15.123:0/3358722348 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5400640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d00c8520 0x7fe8d0158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d4a00640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8d00c8930 0x7fe8d01ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 -- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8d00c8930 msgr2=0x7fe8d01ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8d00c8930 0x7fe8d01ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 -- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d00c8520 msgr2=0x7fe8d0158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d00c8520 0x7fe8d0158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 -- 172.21.15.123:0/3358722348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d0158c70 con 0x7fe8d0163ec0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.306+0000 7fe8d5e00640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d014d630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800cb10 tx=0x7fe8c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.307+0000 7fe8cd600640 1 -- 172.21.15.123:0/3358722348 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8017070 con 0x7fe8d0163ec0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.307+0000 7fe8cd600640 1 -- 172.21.15.123:0/3358722348 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c800ac70 con 0x7fe8d0163ec0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.307+0000 7fe8cd600640 1 -- 172.21.15.123:0/3358722348 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8012600 con 0x7fe8d0163ec0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.307+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d0158ea0 con 0x7fe8d0163ec0 2024-04-25T12:35:02.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.307+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d0156960 con 0x7fe8d0163ec0 2024-04-25T12:35:02.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.308+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d0156b60 con 0x7fe8d0163ec0 2024-04-25T12:35:02.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.309+0000 7fe8cd600640 1 -- 172.21.15.123:0/3358722348 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe8c801a070 con 0x7fe8d0163ec0 2024-04-25T12:35:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.311+0000 7fe8cd600640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8b00744e0 0x7fe8b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.311+0000 7fe8cd600640 1 -- 172.21.15.123:0/3358722348 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fe8c8095210 con 0x7fe8d0163ec0 2024-04-25T12:35:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.311+0000 7fe8d5400640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8b00744e0 0x7fe8b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.312+0000 7fe8d5400640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8b00744e0 0x7fe8b00769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4034000 tx=0x7fe8c402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:02.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.316+0000 7fe8cd600640 1 -- 172.21.15.123:0/3358722348 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8c800ae20 con 0x7fe8d0163ec0 2024-04-25T12:35:02.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.536+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8d00cc660 con 0x7fe8d0163ec0 2024-04-25T12:35:02.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.537+0000 7fe8cd600640 1 -- 172.21.15.123:0/3358722348 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6881 (secure 0 0 0) 0x7fe8d00cc660 con 0x7fe8d0163ec0 2024-04-25T12:35:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:02.537 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:55.199669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4270,"mds_1":4264,"mds_2":4258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4258":{"gid":4258,"name":"d","rank":2,"incarnation":13,"state":"up:active","state_seq":13,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4264":{"gid":4264,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":12,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4258,4264,4270]},"id":1}]} 2024-04-25T12:35:02.538 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 14 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8b00744e0 msgr2=0x7fe8b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8b00744e0 0x7fe8b00769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4034000 tx=0x7fe8c402e000 comp rx=0 tx=0).stop 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 msgr2=0x7fe8d014d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d014d630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800cb10 tx=0x7fe8c800a060 comp rx=0 tx=0).stop 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 shutdown_connections 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8b00744e0 0x7fe8b00769f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0163ec0 0x7fe8d014d630 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8d00c8930 0x7fe8d01ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 --2- 172.21.15.123:0/3358722348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d00c8520 0x7fe8d0158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 >> 172.21.15.123:0/3358722348 conn(0x7fe8d00cd600 msgr2=0x7fe8d006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:02.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 shutdown_connections 2024-04-25T12:35:02.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.539+0000 7fe8d6800640 1 -- 172.21.15.123:0/3358722348 wait complete. 2024-04-25T12:35:02.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:35:02.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.824+0000 7f3ee0c00640 1 Processor -- start 2024-04-25T12:35:02.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.825+0000 7f3ee0c00640 1 -- start start 2024-04-25T12:35:02.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.827+0000 7f3ee0c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc152690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.827+0000 7f3ee0c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc0c8c30 con 0x7f3edc0c8760 2024-04-25T12:35:02.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.827+0000 7f3ee0c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc0c7750 con 0x7f3edc074500 2024-04-25T12:35:02.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.827+0000 7f3ee0c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc0c96e0 con 0x7f3edc0c5a70 2024-04-25T12:35:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.827+0000 7f3edb400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3edc0c8760 0x7f3edc073600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40250/0 (socket says 172.21.15.123:40250) 2024-04-25T12:35:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.827+0000 7f3edb400640 1 -- 172.21.15.123:0/3625673530 learned_addr learned my addr 172.21.15.123:0/3625673530 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.827+0000 7f3edbe00640 1 --2- 172.21.15.123:0/3625673530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.828+0000 7f3edaa00640 1 -- 172.21.15.123:0/3625673530 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418560628 0 0) 0x7f3edc0c8c30 con 0x7f3edc0c8760 2024-04-25T12:35:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.828+0000 7f3edbe00640 1 -- 172.21.15.123:0/3625673530 >> v1:172.21.15.153:6789/0 conn(0x7f3edc074500 legacy=0x7f3edc0c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.828+0000 7f3edbe00640 1 -- 172.21.15.123:0/3625673530 >> v1:172.21.15.123:6789/0 conn(0x7f3edc0c8760 legacy=0x7f3edc073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.828+0000 7f3edbe00640 1 -- 172.21.15.123:0/3625673530 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc0c7a30 con 0x7f3edc0c5a70 2024-04-25T12:35:02.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.829+0000 7f3edbe00640 1 --2- 172.21.15.123:0/3625673530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc152690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009dc0 tx=0x7f3ed402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f82d1f1ad3f99f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:02.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.829+0000 7f3edaa00640 1 -- 172.21.15.123:0/3625673530 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4007740 con 0x7f3edc0c5a70 2024-04-25T12:35:02.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.829+0000 7f3edaa00640 1 -- 172.21.15.123:0/3625673530 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed40078e0 con 0x7f3edc0c5a70 2024-04-25T12:35:02.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.829+0000 7f3edaa00640 1 -- 172.21.15.123:0/3625673530 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4007cb0 con 0x7f3edc0c5a70 2024-04-25T12:35:02.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.830+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3625673530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 msgr2=0x7f3edc152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.830+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3625673530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc152690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009dc0 tx=0x7f3ed402a680 comp rx=0 tx=0).stop 2024-04-25T12:35:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.830+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3625673530 shutdown_connections 2024-04-25T12:35:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.830+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3625673530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc152690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.830+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3625673530 >> 172.21.15.123:0/3625673530 conn(0x7f3edc0ce170 msgr2=0x7f3edc0ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.830+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3625673530 shutdown_connections 2024-04-25T12:35:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.830+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3625673530 wait complete. 2024-04-25T12:35:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.832+0000 7f3ee0c00640 1 Processor -- start 2024-04-25T12:35:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.832+0000 7f3ee0c00640 1 -- start start 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3ee0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3edc074500 0x7f3edc143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3ee0c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3ee0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3edc0c8760 0x7f3edc159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3ee0c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0ca560 con 0x7f3edc074500 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3ee0c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0c8080 con 0x7f3edc0c8760 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3ee0c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc0ca6e0 con 0x7f3edc0c5a70 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3edb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3edc074500 0x7f3edc143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3edb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3edc074500 0x7f3edc143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52338/0 (socket says 172.21.15.123:52338) 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3edb400640 1 -- 172.21.15.123:0/3168770270 learned_addr learned my addr 172.21.15.123:0/3168770270 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:02.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3ed2a00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3edbe00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3edc0c8760 0x7f3edc159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.834+0000 7f3edb400640 1 -- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 msgr2=0x7f3edc158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3edb400640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3edb400640 1 -- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3edc0c8760 msgr2=0x7f3edc159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3edb400640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3edc0c8760 0x7f3edc159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3edb400640 1 -- 172.21.15.123:0/3168770270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc150450 con 0x7f3edc074500 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3ed2a00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3edb400640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3edc074500 0x7f3edc143400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00c420 tx=0x7f3ecc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:02.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3ed3e00640 1 -- 172.21.15.123:0/3168770270 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0074e0 con 0x7f3edc074500 2024-04-25T12:35:02.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3ed3e00640 1 -- 172.21.15.123:0/3168770270 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0047e0 con 0x7f3edc074500 2024-04-25T12:35:02.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3ed3e00640 1 -- 172.21.15.123:0/3168770270 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc004c10 con 0x7f3edc074500 2024-04-25T12:35:02.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc143c30 con 0x7f3edc074500 2024-04-25T12:35:02.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.835+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc073b30 con 0x7f3edc074500 2024-04-25T12:35:02.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.837+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc074a70 con 0x7f3edc074500 2024-04-25T12:35:02.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.838+0000 7f3ed3e00640 1 -- 172.21.15.123:0/3168770270 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ecc011070 con 0x7f3edc074500 2024-04-25T12:35:02.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.838+0000 7f3ed3e00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3eb80744b0 0x7f3eb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:02.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.839+0000 7f3ed3e00640 1 -- 172.21.15.123:0/3168770270 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f3ecc095150 con 0x7f3edc074500 2024-04-25T12:35:02.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.839+0000 7f3ed2a00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3eb80744b0 0x7f3eb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:02.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.839+0000 7f3ed2a00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3eb80744b0 0x7f3eb80769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3edc155240 tx=0x7f3ec4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:02.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:02.843+0000 7f3ed3e00640 1 -- 172.21.15.123:0/3168770270 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ecc05ee80 con 0x7f3edc074500 2024-04-25T12:35:03.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.044+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3edc076da0 con 0x7f3edc074500 2024-04-25T12:35:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.046+0000 7f3ed3e00640 1 -- 172.21.15.123:0/3168770270 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18673 (secure 0 0 0) 0x7f3ecc063180 con 0x7f3edc074500 2024-04-25T12:35:03.046 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:03.047 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:35:00.228853+0000","last_up_change":"2024-04-25T12:34:07.013466+0000","last_in_change":"2024-04-25T12:33:58.290536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6827","nonce":2327013656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6829","nonce":2327013656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6833","nonce":2327013656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2327013656},{"type":"v1","addr":"172.21.15.153:6831","nonce":2327013656}]},"public_addr":"172.21.15.153:6827/2327013656","cluster_addr":"172.21.15.153:6829/2327013656","heartbeat_back_addr":"172.21.15.153:6833/2327013656","heartbeat_front_addr":"172.21.15.153:6831/2327013656","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3eb80744b0 msgr2=0x7f3eb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3eb80744b0 0x7f3eb80769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3edc155240 tx=0x7f3ec4009040 comp rx=0 tx=0).stop 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3edc074500 msgr2=0x7f3edc143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3edc074500 0x7f3edc143400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00c420 tx=0x7f3ecc00c8e0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 shutdown_connections 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3eb80744b0 0x7f3eb80769c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3edc0c8760 0x7f3edc159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3edc0c5a70 0x7f3edc158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 --2- 172.21.15.123:0/3168770270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3edc074500 0x7f3edc143400 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.048+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 >> 172.21.15.123:0/3168770270 conn(0x7f3edc0ce170 msgr2=0x7f3edc0c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:03.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.049+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 shutdown_connections 2024-04-25T12:35:03.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.049+0000 7f3ee0c00640 1 -- 172.21.15.123:0/3168770270 wait complete. 2024-04-25T12:35:03.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:35:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.346+0000 7fc183000640 1 Processor -- start 2024-04-25T12:35:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.346+0000 7fc183000640 1 -- start start 2024-04-25T12:35:03.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc183000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc17c0c8250 0x7fc17c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:03.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc183000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc17c156e60 con 0x7fc17c0c9b80 2024-04-25T12:35:03.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc183000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc17c0c7130 con 0x7fc17c159820 2024-04-25T12:35:03.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc183000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc17c0749c0 con 0x7fc17c0c8250 2024-04-25T12:35:03.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc17be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc17c0c9b80 0x7fc17c0742b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40256/0 (socket says 172.21.15.123:40256) 2024-04-25T12:35:03.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc17be00640 1 -- 172.21.15.123:0/3595883443 learned_addr learned my addr 172.21.15.123:0/3595883443 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:03.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc173400640 1 --2- 172.21.15.123:0/3595883443 >> v2:172.21.15.153:3300/0 conn(0x7fc17c0c8250 0x7fc17c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:03.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.349+0000 7fc17b400640 1 -- 172.21.15.123:0/3595883443 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861384940 0 0) 0x7fc17c156e60 con 0x7fc17c0c9b80 2024-04-25T12:35:03.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.350+0000 7fc17b400640 1 -- 172.21.15.123:0/3595883443 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc160003520 con 0x7fc17c0c9b80 2024-04-25T12:35:03.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.350+0000 7fc17b400640 1 -- 172.21.15.123:0/3595883443 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3764663867 0 0) 0x7fc17c0c7130 con 0x7fc17c159820 2024-04-25T12:35:03.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.350+0000 7fc17b400640 1 -- 172.21.15.123:0/3595883443 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc17c156e60 con 0x7fc17c159820 2024-04-25T12:35:03.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.350+0000 7fc173400640 1 -- 172.21.15.123:0/3595883443 >> v1:172.21.15.153:6789/0 conn(0x7fc17c159820 legacy=0x7fc17c159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.350+0000 7fc173400640 1 -- 172.21.15.123:0/3595883443 >> v1:172.21.15.123:6789/0 conn(0x7fc17c0c9b80 legacy=0x7fc17c0742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.350+0000 7fc173400640 1 -- 172.21.15.123:0/3595883443 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c15ca70 con 0x7fc17c0c8250 2024-04-25T12:35:03.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.351+0000 7fc173400640 1 --2- 172.21.15.123:0/3595883443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c8250 0x7fc17c0c8660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc1640097a0 tx=0x7fc16402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1bd06603aea4ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:03.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.351+0000 7fc17b400640 1 -- 172.21.15.123:0/3595883443 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164037070 con 0x7fc17c0c8250 2024-04-25T12:35:03.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.351+0000 7fc17b400640 1 -- 172.21.15.123:0/3595883443 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc164032440 con 0x7fc17c0c8250 2024-04-25T12:35:03.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.351+0000 7fc17b400640 1 -- 172.21.15.123:0/3595883443 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1640327f0 con 0x7fc17c0c8250 2024-04-25T12:35:03.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.352+0000 7fc183000640 1 -- 172.21.15.123:0/3595883443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c8250 msgr2=0x7fc17c0c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.352+0000 7fc183000640 1 --2- 172.21.15.123:0/3595883443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c8250 0x7fc17c0c8660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc1640097a0 tx=0x7fc16402a230 comp rx=0 tx=0).stop 2024-04-25T12:35:03.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.352+0000 7fc183000640 1 -- 172.21.15.123:0/3595883443 shutdown_connections 2024-04-25T12:35:03.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.352+0000 7fc183000640 1 --2- 172.21.15.123:0/3595883443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c8250 0x7fc17c0c8660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.352+0000 7fc183000640 1 -- 172.21.15.123:0/3595883443 >> 172.21.15.123:0/3595883443 conn(0x7fc17c06c750 msgr2=0x7fc17c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:03.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.352+0000 7fc183000640 1 -- 172.21.15.123:0/3595883443 shutdown_connections 2024-04-25T12:35:03.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.352+0000 7fc183000640 1 -- 172.21.15.123:0/3595883443 wait complete. 2024-04-25T12:35:03.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.354+0000 7fc183000640 1 Processor -- start 2024-04-25T12:35:03.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.355+0000 7fc183000640 1 -- start start 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc183000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17c0c8250 0x7fc17c1459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc183000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c9b80 0x7fc17c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc183000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc17c159820 0x7fc17c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc183000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c15d030 con 0x7fc17c159820 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc183000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c0c8ba0 con 0x7fc17c0c9b80 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc183000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc17c15d310 con 0x7fc17c0c8250 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc181000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc17c159820 0x7fc17c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc173400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c9b80 0x7fc17c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:03.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc173400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c9b80 0x7fc17c152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:50712/0 (socket says 172.21.15.123:50712) 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc173400640 1 -- 172.21.15.123:0/2651204164 learned_addr learned my addr 172.21.15.123:0/2651204164 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc181000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc17c159820 0x7fc17c1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52344/0 (socket says 172.21.15.123:52344) 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.356+0000 7fc17be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17c0c8250 0x7fc17c1459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc173400640 1 -- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17c0c8250 msgr2=0x7fc17c1459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc173400640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17c0c8250 0x7fc17c1459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc173400640 1 -- 172.21.15.123:0/2651204164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc17c159820 msgr2=0x7fc17c1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc173400640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc17c159820 0x7fc17c1508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc173400640 1 -- 172.21.15.123:0/2651204164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c153160 con 0x7fc17c0c9b80 2024-04-25T12:35:03.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc181000640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc17c159820 0x7fc17c1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:03.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc173400640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c9b80 0x7fc17c152c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc16402a270 tx=0x7fc1640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:03.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc178c00640 1 -- 172.21.15.123:0/2651204164 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164037040 con 0x7fc17c0c9b80 2024-04-25T12:35:03.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc178c00640 1 -- 172.21.15.123:0/2651204164 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc164042070 con 0x7fc17c0c9b80 2024-04-25T12:35:03.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.357+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c1f1de0 con 0x7fc17c0c9b80 2024-04-25T12:35:03.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.358+0000 7fc178c00640 1 -- 172.21.15.123:0/2651204164 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164032830 con 0x7fc17c0c9b80 2024-04-25T12:35:03.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.358+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c1f2340 con 0x7fc17c0c9b80 2024-04-25T12:35:03.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.359+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17c157030 con 0x7fc17c0c9b80 2024-04-25T12:35:03.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.360+0000 7fc178c00640 1 -- 172.21.15.123:0/2651204164 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc1640329d0 con 0x7fc17c0c9b80 2024-04-25T12:35:03.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.361+0000 7fc178c00640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc1580744b0 0x7fc1580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:03.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.361+0000 7fc178c00640 1 -- 172.21.15.123:0/2651204164 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fc1640b5a70 con 0x7fc17c0c9b80 2024-04-25T12:35:03.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.361+0000 7fc17be00640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc1580744b0 0x7fc1580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:03.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.362+0000 7fc17be00640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc1580744b0 0x7fc1580769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000f60 tx=0x7fc16c007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:03.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.366+0000 7fc178c00640 1 -- 172.21.15.123:0/2651204164 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc17c157030 con 0x7fc17c0c9b80 2024-04-25T12:35:03.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.605+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc17c076700 con 0x7fc17c0c9b80 2024-04-25T12:35:03.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.606+0000 7fc178c00640 1 -- 172.21.15.123:0/2651204164 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6881 (secure 0 0 0) 0x7fc16404c020 con 0x7fc17c0c9b80 2024-04-25T12:35:03.607 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:03.607 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:55.199669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4270,"mds_1":4264,"mds_2":4258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4258":{"gid":4258,"name":"d","rank":2,"incarnation":13,"state":"up:active","state_seq":13,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4264":{"gid":4264,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":12,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4258,4264,4270]},"id":1}]} 2024-04-25T12:35:03.607 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 14 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc1580744b0 msgr2=0x7fc1580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc1580744b0 0x7fc1580769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000f60 tx=0x7fc16c007d30 comp rx=0 tx=0).stop 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c9b80 msgr2=0x7fc17c152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c9b80 0x7fc17c152c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc16402a270 tx=0x7fc1640046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 shutdown_connections 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc1580744b0 0x7fc1580769c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc17c159820 0x7fc17c1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc17c0c9b80 0x7fc17c152c20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 --2- 172.21.15.123:0/2651204164 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17c0c8250 0x7fc17c1459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 >> 172.21.15.123:0/2651204164 conn(0x7fc17c06c750 msgr2=0x7fc17c151870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 shutdown_connections 2024-04-25T12:35:03.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:03.609+0000 7fc183000640 1 -- 172.21.15.123:0/2651204164 wait complete. 2024-04-25T12:35:04.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T12:34:13.424937+0000', 'modified': '2024-04-25T12:34:55.199669+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4270, 'mds_1': 4264, 'mds_2': 4258}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4258': {'gid': 4258, 'name': 'd', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.153:6835/3198384231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 3198384231}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 3198384231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4264': {'gid': 4264, 'name': 'f', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.153:6839/4167745001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 4167745001}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 4167745001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4270': {'gid': 4270, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6834/1720190095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1720190095}, {'type': 'v1', 'addr': '172.21.15.123:6834', 'nonce': 1720190095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4270, 'qdb_cluster': [4258, 4264, 4270]} 2024-04-25T12:35:04.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T12:35:04.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T12:35:04.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.526+0000 7fe8cb200640 1 Processor -- start 2024-04-25T12:35:04.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.526+0000 7fe8cb200640 1 -- start start 2024-04-25T12:35:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.528+0000 7fe8cb200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8cc0c94f0 0x7fe8cc0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.528+0000 7fe8cb200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8cc1593c0 con 0x7fe8cc0c2720 2024-04-25T12:35:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.528+0000 7fe8cb200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8cc1596a0 con 0x7fe8cc072d70 2024-04-25T12:35:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.528+0000 7fe8cb200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe8cc159fd0 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8ca800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe8cc0c2720 0x7fe8cc0c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40262/0 (socket says 172.21.15.123:40262) 2024-04-25T12:35:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8ca800640 1 -- 172.21.15.123:0/19538466 learned_addr learned my addr 172.21.15.123:0/19538466 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8c9400640 1 --2- 172.21.15.123:0/19538466 >> v2:172.21.15.153:3300/0 conn(0x7fe8cc0c94f0 0x7fe8cc0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8c8a00640 1 -- 172.21.15.123:0/19538466 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095020730 0 0) 0x7fe8cc1593c0 con 0x7fe8cc0c2720 2024-04-25T12:35:04.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8c8a00640 1 -- 172.21.15.123:0/19538466 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8b0005730 con 0x7fe8cc0c2720 2024-04-25T12:35:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8c8a00640 1 -- 172.21.15.123:0/19538466 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689938083 0 0) 0x7fe8cc1596a0 con 0x7fe8cc072d70 2024-04-25T12:35:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8c9400640 1 -- 172.21.15.123:0/19538466 >> v1:172.21.15.153:6789/0 conn(0x7fe8cc072d70 legacy=0x7fe8cc1420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8c9400640 1 -- 172.21.15.123:0/19538466 >> v1:172.21.15.123:6789/0 conn(0x7fe8cc0c2720 legacy=0x7fe8cc0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.529+0000 7fe8c9400640 1 -- 172.21.15.123:0/19538466 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc159980 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.530+0000 7fe8c9400640 1 --2- 172.21.15.123:0/19538466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc0c9900 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009d90 tx=0x7fe8bc02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4177ef1bcc077a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.531+0000 7fe8c8a00640 1 -- 172.21.15.123:0/19538466 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc037070 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.531+0000 7fe8c8a00640 1 -- 172.21.15.123:0/19538466 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8bc007740 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.531+0000 7fe8c8a00640 1 -- 172.21.15.123:0/19538466 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc007b10 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.532+0000 7fe8cb200640 1 -- 172.21.15.123:0/19538466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c94f0 msgr2=0x7fe8cc0c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.532+0000 7fe8cb200640 1 --2- 172.21.15.123:0/19538466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc0c9900 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009d90 tx=0x7fe8bc02a780 comp rx=0 tx=0).stop 2024-04-25T12:35:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.532+0000 7fe8cb200640 1 -- 172.21.15.123:0/19538466 shutdown_connections 2024-04-25T12:35:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.532+0000 7fe8cb200640 1 --2- 172.21.15.123:0/19538466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc0c9900 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.532+0000 7fe8cb200640 1 -- 172.21.15.123:0/19538466 >> 172.21.15.123:0/19538466 conn(0x7fe8cc0cd600 msgr2=0x7fe8cc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.532+0000 7fe8cb200640 1 -- 172.21.15.123:0/19538466 shutdown_connections 2024-04-25T12:35:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.532+0000 7fe8cb200640 1 -- 172.21.15.123:0/19538466 wait complete. 2024-04-25T12:35:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.535+0000 7fe8cb200640 1 Processor -- start 2024-04-25T12:35:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.535+0000 7fe8cb200640 1 -- start start 2024-04-25T12:35:04.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.536+0000 7fe8cb200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8cc072d70 0x7fe8cc15be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.536+0000 7fe8cb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c2720 0x7fe8cc13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.536+0000 7fe8cb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc13de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.536+0000 7fe8cb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc0c8990 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.536+0000 7fe8cb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc0c8810 con 0x7fe8cc0c2720 2024-04-25T12:35:04.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.536+0000 7fe8cb200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe8cc1587f0 con 0x7fe8cc072d70 2024-04-25T12:35:04.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc13de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8c9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c2720 0x7fe8cc13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc13de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52348/0 (socket says 172.21.15.123:52348) 2024-04-25T12:35:04.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 -- 172.21.15.123:0/2928082731 learned_addr learned my addr 172.21.15.123:0/2928082731 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8cc072d70 0x7fe8cc15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 -- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8cc072d70 msgr2=0x7fe8cc15be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8cc072d70 0x7fe8cc15be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 -- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c2720 msgr2=0x7fe8cc13d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c2720 0x7fe8cc13d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8ca800640 1 -- 172.21.15.123:0/2928082731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc0cc6c0 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.537+0000 7fe8c9400640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c2720 0x7fe8cc13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.538+0000 7fe8ca800640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc13de80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c390 tx=0x7fe8c400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.538+0000 7fe8c2000640 1 -- 172.21.15.123:0/2928082731 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4017070 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.538+0000 7fe8c2000640 1 -- 172.21.15.123:0/2928082731 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4005ce0 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.538+0000 7fe8c2000640 1 -- 172.21.15.123:0/2928082731 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40072d0 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.538+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8cc15c370 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.538+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8cc0cc930 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.541+0000 7fe8c2000640 1 -- 172.21.15.123:0/2928082731 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe8c401a070 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.542+0000 7fe8c2000640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8a80744e0 0x7fe8a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.542+0000 7fe8c2000640 1 -- 172.21.15.123:0/2928082731 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fe8c4094740 con 0x7fe8cc0c94f0 2024-04-25T12:35:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.542+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8a80744e0 0x7fe8a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.542+0000 7fe8d7c3e740 1 Processor -- start 2024-04-25T12:35:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.542+0000 7fe8d7c3e740 1 -- start start 2024-04-25T12:35:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.543+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8a80744e0 0x7fe8a80769f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4000f60 tx=0x7fe8b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c46ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c453600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c453b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55795c392600 con 0x55795c06b590 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55795c32f100 con 0x55795c0633b0 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55795c38e5c0 con 0x55795c08c270 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c453b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8c9e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c46ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c453b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52364/0 (socket says 172.21.15.123:52364) 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8c9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c453600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 -- 172.21.15.123:0/3795304398 learned_addr learned my addr 172.21.15.123:0/3795304398 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 -- 172.21.15.123:0/3795304398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 msgr2=0x55795c46ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c46ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 -- 172.21.15.123:0/3795304398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 msgr2=0x55795c453600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c453600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8ca800640 1 -- 172.21.15.123:0/3795304398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55795c0a42d0 con 0x55795c06b590 2024-04-25T12:35:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.544+0000 7fe8c9400640 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c453600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:04.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.545+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c453b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8c405ab80 tx=0x7fe8c405dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2eda9cab6b7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.545+0000 7fe8ba600640 1 -- 172.21.15.123:0/3795304398 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c405e590 con 0x55795c06b590 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.545+0000 7fe8ba600640 1 -- 172.21.15.123:0/3795304398 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4062890 con 0x55795c06b590 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.545+0000 7fe8ba600640 1 -- 172.21.15.123:0/3795304398 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4065d80 con 0x55795c06b590 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3795304398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 msgr2=0x55795c453b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c453b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8c405ab80 tx=0x7fe8c405dee0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3795304398 shutdown_connections 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c453b40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c453600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3795304398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c46ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3795304398 >> 172.21.15.123:0/3795304398 conn(0x55795c0803c0 msgr2=0x55795c42dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3795304398 shutdown_connections 2024-04-25T12:35:04.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.546+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3795304398 wait complete. 2024-04-25T12:35:04.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.547+0000 7fe8d7c3e740 1 -- start start 2024-04-25T12:35:04.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.547+0000 7fe8d7c3e740 1 Processor -- start 2024-04-25T12:35:04.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c47fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c4739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c42f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55795c42da60 con 0x55795c06b590 2024-04-25T12:35:04.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55795c319f70 con 0x55795c0633b0 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55795c467740 con 0x55795c08c270 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8d7c3e740 10 client.0 _ll_register_callbacks cb 0x55795c388480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c4739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c42f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c47fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c47fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:50742/0 (socket says 172.21.15.123:50742) 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c42f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:59566/0 (socket says 172.21.15.123:59566) 2024-04-25T12:35:04.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 -- 172.21.15.123:0/311249393 learned_addr learned my addr 172.21.15.123:0/311249393 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 -- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 msgr2=0x55795c42f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c42f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 -- 172.21.15.123:0/311249393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 msgr2=0x55795c4739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c4739d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8c9e00640 1 -- 172.21.15.123:0/311249393 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55795c16e6b0 con 0x55795c0633b0 2024-04-25T12:35:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.549+0000 7fe8ca800640 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c42f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c47fa90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4014730 tx=0x7fe8b4001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84af89572e078bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8aea00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8aea00640 1 -- 172.21.15.123:0/311249393 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4025020 con 0x55795c0633b0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c4062890 con 0x55795c0633b0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8aea00640 1 -- 172.21.15.123:0/311249393 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fe8b4013730 con 0x55795c0633b0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8aea00640 1 -- 172.21.15.123:0/311249393 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fe8b402f050 con 0x55795c0633b0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8d7c3e740 10 client.4852 fetch_fsmap learned FSMap version 14 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.550+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55795c4678e0 con 0x55795c0633b0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8aea00640 1 -- 172.21.15.123:0/311249393 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4062890 con 0x55795c0633b0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8aea00640 1 -- 172.21.15.123:0/311249393 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 14) ==== 4752+0+0 (secure 0 0 0) 0x7fe8b4032020 con 0x55795c0633b0 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8d7c3e740 10 client.4852 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8d7c3e740 10 client.4852 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8d7c3e740 20 client.4852 populate_metadata read hostname 'smithi123' 2024-04-25T12:35:04.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c04ae40 0x55795c48daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8d7c3e740 4 client.4852 mds_command: new command op to 4258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:35:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 --> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55795c3a0210 con 0x55795c04ae40 2024-04-25T12:35:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.551+0000 7fe8c9400640 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c04ae40 0x55795c48daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.552+0000 7fe8aea00640 10 client.4852 ms_handle_connect on v2:172.21.15.153:6834/3198384231 2024-04-25T12:35:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.552+0000 7fe8c9400640 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c04ae40 0x55795c48daf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=826fe52e76a15c48 server_cookie=fd1a1a0604e06d7d in_seq=0 out_seq=0 2024-04-25T12:35:04.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8aea00640 1 -- 172.21.15.123:0/311249393 <== mds.2 v2:172.21.15.153:6834/3198384231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55795c3a0210 con 0x55795c04ae40 2024-04-25T12:35:04.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8aea00640 10 client.4852 handle_command_reply: tid=0 2024-04-25T12:35:04.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 client.4852 shutdown 2024-04-25T12:35:04.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 msgr2=0x55795c47fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c47fa90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4014730 tx=0x7fe8b4001080 comp rx=0 tx=0).stop 2024-04-25T12:35:04.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 shutdown_connections 2024-04-25T12:35:04.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c04ae40 0x55795c48daf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c08c270 0x55795c42f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c06b590 0x55795c4739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/311249393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c0633b0 0x55795c47fa90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 >> 172.21.15.123:0/311249393 conn(0x55795c0803c0 msgr2=0x55795c429700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8aea00640 0 client.4852 ms_handle_reset on v2:172.21.15.153:6834/3198384231 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 shutdown_connections 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/311249393 wait complete. 2024-04-25T12:35:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.553+0000 7fe8d7c3e740 20 client.4852 trim_cache size 0 max 16384 2024-04-25T12:35:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.573+0000 7fe8d7c3e740 1 Processor -- start 2024-04-25T12:35:04.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.573+0000 7fe8d7c3e740 1 -- start start 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c429700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c429c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c46da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc1fe7a0 con 0x55795c08c270 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55795c47ef40 con 0x55795c06b590 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55795c472b90 con 0x55795c0633b0 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c46da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c46da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52390/0 (socket says 172.21.15.123:52390) 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 -- 172.21.15.123:0/3369648753 learned_addr learned my addr 172.21.15.123:0/3369648753 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8c9400640 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c429c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c429700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 -- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 msgr2=0x55795c429700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c429700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 -- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 msgr2=0x55795c429c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c429c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.575+0000 7fe8ca800640 1 -- 172.21.15.123:0/3369648753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55795c46cb70 con 0x55795c08c270 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.576+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c46da50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe8c405e520 tx=0x7fe8c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670197d217280a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.576+0000 7fe8b9c00640 1 -- 172.21.15.123:0/3369648753 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4065d80 con 0x55795c08c270 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.576+0000 7fe8b9c00640 1 -- 172.21.15.123:0/3369648753 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4062890 con 0x55795c08c270 2024-04-25T12:35:04.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.576+0000 7fe8b9c00640 1 -- 172.21.15.123:0/3369648753 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4007670 con 0x55795c08c270 2024-04-25T12:35:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3369648753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 msgr2=0x55795c46da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c46da50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe8c405e520 tx=0x7fe8c4013000 comp rx=0 tx=0).stop 2024-04-25T12:35:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3369648753 shutdown_connections 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c46da50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c429c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3369648753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c429700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3369648753 >> 172.21.15.123:0/3369648753 conn(0x55795c0803c0 msgr2=0x55795c42f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3369648753 shutdown_connections 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.577+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3369648753 wait complete. 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.578+0000 7fe8d7c3e740 1 -- start start 2024-04-25T12:35:04.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.578+0000 7fe8d7c3e740 1 Processor -- start 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.579+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c4e3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c4ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8d7c3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c4ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55795c468700 con 0x55795c08c270 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55795c4680c0 con 0x55795c06b590 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8d7c3e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55795c482820 con 0x55795c0633b0 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8d7c3e740 10 client.0 _ll_register_callbacks cb 0x7fe8b4001de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c4ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c4ece80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52402/0 (socket says 172.21.15.123:52402) 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 -- 172.21.15.123:0/3066265150 learned_addr learned my addr 172.21.15.123:0/3066265150 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c4e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8c9400640 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c4ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 -- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 msgr2=0x55795c4e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c4e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 -- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 msgr2=0x55795c4ec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c4ec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.580+0000 7fe8ca800640 1 -- 172.21.15.123:0/3066265150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55795c458810 con 0x55795c08c270 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8ca800640 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c4ece80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe8c405ab80 tx=0x7fe8c4001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8127055a4d44b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8aea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8aea00640 1 -- 172.21.15.123:0/3066265150 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c405e590 con 0x55795c08c270 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8aea00640 1 -- 172.21.15.123:0/3066265150 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fe8c408fb70 con 0x55795c08c270 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c4062890 con 0x55795c08c270 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8aea00640 1 -- 172.21.15.123:0/3066265150 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fe8c408fd70 con 0x55795c08c270 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8d7c3e740 10 client.4773 fetch_fsmap learned FSMap version 14 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55795c4290c0 con 0x55795c08c270 2024-04-25T12:35:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8aea00640 1 -- 172.21.15.123:0/3066265150 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4062890 con 0x55795c08c270 2024-04-25T12:35:04.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8aea00640 1 -- 172.21.15.123:0/3066265150 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 14) ==== 4752+0+0 (secure 0 0 0) 0x7fe8c4092350 con 0x55795c08c270 2024-04-25T12:35:04.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8d7c3e740 10 client.4773 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T12:35:04.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8d7c3e740 10 client.4773 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T12:35:04.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.581+0000 7fe8d7c3e740 20 client.4773 populate_metadata read hostname 'smithi123' 2024-04-25T12:35:04.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.582+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c4f0340 0x55795c4f0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.582+0000 7fe8d7c3e740 4 client.4773 mds_command: new command op to 4258 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:35:04.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.582+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 --> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] -- command(tid 0: {"prefix": "status"}) -- 0x55795c39a080 con 0x55795c4f0340 2024-04-25T12:35:04.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.582+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c4f0340 0x55795c4f0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.582+0000 7fe8c9e00640 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c4f0340 0x55795c4f0750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4f0e8a3d4b29652 server_cookie=42171260e913fbc8 in_seq=0 out_seq=0 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.582+0000 7fe8aea00640 10 client.4773 ms_handle_connect on v2:172.21.15.153:6834/3198384231 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8aea00640 1 -- 172.21.15.123:0/3066265150 <== mds.2 v2:172.21.15.153:6834/3198384231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55795c39a080 con 0x55795c4f0340 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8aea00640 10 client.4773 handle_command_reply: tid=0 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 client.4773 shutdown 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 msgr2=0x55795c4ece80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c4ece80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe8c405ab80 tx=0x7fe8c4001150 comp rx=0 tx=0).stop 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 shutdown_connections 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:6834/3198384231,v1:172.21.15.153:6835/3198384231] conn(0x55795c4f0340 0x55795c4f0750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55795c08c270 0x55795c4ece80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55795c06b590 0x55795c4ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8aea00640 0 client.4773 ms_handle_reset on v2:172.21.15.153:6834/3198384231 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 --2- 172.21.15.123:0/3066265150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55795c0633b0 0x55795c4e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 >> 172.21.15.123:0/3066265150 conn(0x55795c0803c0 msgr2=0x55795c466f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 shutdown_connections 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 1 -- 172.21.15.123:0/3066265150 wait complete. 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.583+0000 7fe8d7c3e740 20 client.4773 trim_cache size 0 max 16384 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 2, 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4258, 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 14.435946965999999, 2024-04-25T12:35:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 14, 2024-04-25T12:35:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 27, 2024-04-25T12:35:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T12:35:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 57.054233318999998 2024-04-25T12:35:04.588 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8a80744e0 msgr2=0x7fe8a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8a80744e0 0x7fe8a80769f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4000f60 tx=0x7fe8b4009040 comp rx=0 tx=0).stop 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8cc0c94f0 msgr2=0x7fe8cc13de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc13de80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c390 tx=0x7fe8c400c850 comp rx=0 tx=0).stop 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 shutdown_connections 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe8a80744e0 0x7fe8a80769f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8cc0c94f0 0x7fe8cc13de80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8cc0c2720 0x7fe8cc13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 --2- 172.21.15.123:0/2928082731 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8cc072d70 0x7fe8cc15be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 >> 172.21.15.123:0/2928082731 conn(0x7fe8cc0cd600 msgr2=0x7fe8cc074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 shutdown_connections 2024-04-25T12:35:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.584+0000 7fe8cb200640 1 -- 172.21.15.123:0/2928082731 wait complete. 2024-04-25T12:35:04.609 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T12:35:04.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.882+0000 7eff2fa00640 1 Processor -- start 2024-04-25T12:35:04.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.882+0000 7eff2fa00640 1 -- start start 2024-04-25T12:35:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2fa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff280c4d50 0x7eff28154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2fa00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7eff2815afa0 0x7eff280cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2fa00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2815b3b0 con 0x7eff28074870 2024-04-25T12:35:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2fa00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7eff280c61f0 con 0x7eff280c4d50 2024-04-25T12:35:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2fa00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7eff280c87d0 con 0x7eff2815afa0 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff280c4d50 0x7eff28154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff280c4d50 0x7eff28154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52404/0 (socket says 172.21.15.123:52404) 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2d000640 1 -- 172.21.15.123:0/4049933105 learned_addr learned my addr 172.21.15.123:0/4049933105 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2da00640 1 --2- 172.21.15.123:0/4049933105 >> v2:172.21.15.153:3300/0 conn(0x7eff2815afa0 0x7eff280cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2d000640 1 -- 172.21.15.123:0/4049933105 >> v2:172.21.15.153:3300/0 conn(0x7eff2815afa0 msgr2=0x7eff280cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2d000640 1 --2- 172.21.15.123:0/4049933105 >> v2:172.21.15.153:3300/0 conn(0x7eff2815afa0 0x7eff280cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.884+0000 7eff2d000640 1 -- 172.21.15.123:0/4049933105 >> v1:172.21.15.153:6789/0 conn(0x7eff28074870 legacy=0x7eff280c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.885+0000 7eff2d000640 1 -- 172.21.15.123:0/4049933105 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff28073ce0 con 0x7eff280c4d50 2024-04-25T12:35:04.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.885+0000 7eff2d000640 1 --2- 172.21.15.123:0/4049933105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff280c4d50 0x7eff28154890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff280ce8b0 tx=0x7eff1802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eddd4174fa8acca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.886+0000 7eff27e00640 1 -- 172.21.15.123:0/4049933105 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18037070 con 0x7eff280c4d50 2024-04-25T12:35:04.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.886+0000 7eff27e00640 1 -- 172.21.15.123:0/4049933105 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff1802ae50 con 0x7eff280c4d50 2024-04-25T12:35:04.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.886+0000 7eff27e00640 1 -- 172.21.15.123:0/4049933105 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18032860 con 0x7eff280c4d50 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 -- 172.21.15.123:0/4049933105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff280c4d50 msgr2=0x7eff28154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 --2- 172.21.15.123:0/4049933105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff280c4d50 0x7eff28154890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff280ce8b0 tx=0x7eff1802a300 comp rx=0 tx=0).stop 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 -- 172.21.15.123:0/4049933105 shutdown_connections 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 --2- 172.21.15.123:0/4049933105 >> v2:172.21.15.153:3300/0 conn(0x7eff2815afa0 0x7eff280cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 --2- 172.21.15.123:0/4049933105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff280c4d50 0x7eff28154890 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 -- 172.21.15.123:0/4049933105 >> 172.21.15.123:0/4049933105 conn(0x7eff2806c750 msgr2=0x7eff2806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 -- 172.21.15.123:0/4049933105 shutdown_connections 2024-04-25T12:35:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.887+0000 7eff2fa00640 1 -- 172.21.15.123:0/4049933105 wait complete. 2024-04-25T12:35:04.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.889+0000 7eff2fa00640 1 Processor -- start 2024-04-25T12:35:04.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.889+0000 7eff2fa00640 1 -- start start 2024-04-25T12:35:04.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2fa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff28074870 0x7eff2814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2fa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff280c4d50 0x7eff281f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff2815afa0 0x7eff280c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff280c8460 con 0x7eff2815afa0 2024-04-25T12:35:04.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2fa00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff280742a0 con 0x7eff28074870 2024-04-25T12:35:04.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2fa00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7eff280c9370 con 0x7eff280c4d50 2024-04-25T12:35:04.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff2815afa0 0x7eff280c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff2815afa0 0x7eff280c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52408/0 (socket says 172.21.15.123:52408) 2024-04-25T12:35:04.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 -- 172.21.15.123:0/1298352438 learned_addr learned my addr 172.21.15.123:0/1298352438 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2d000640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff28074870 0x7eff2814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff1fe00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff280c4d50 0x7eff281f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 -- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff280c4d50 msgr2=0x7eff281f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff280c4d50 0x7eff281f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 -- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff28074870 msgr2=0x7eff2814ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff28074870 0x7eff2814ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.891+0000 7eff2da00640 1 -- 172.21.15.123:0/1298352438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff281f5540 con 0x7eff2815afa0 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.892+0000 7eff2da00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff2815afa0 0x7eff280c3aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff2000efc0 tx=0x7eff2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.892+0000 7eff25600640 1 -- 172.21.15.123:0/1298352438 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20018020 con 0x7eff2815afa0 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.892+0000 7eff25600640 1 -- 172.21.15.123:0/1298352438 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff20014480 con 0x7eff2815afa0 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.892+0000 7eff25600640 1 -- 172.21.15.123:0/1298352438 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20009e60 con 0x7eff2815afa0 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.892+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2814b1e0 con 0x7eff2815afa0 2024-04-25T12:35:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.892+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff281f4240 con 0x7eff2815afa0 2024-04-25T12:35:04.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.895+0000 7eff25600640 1 -- 172.21.15.123:0/1298352438 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7eff20005d10 con 0x7eff2815afa0 2024-04-25T12:35:04.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.896+0000 7eff25600640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7eff040744b0 0x7eff040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.896+0000 7eff25600640 1 -- 172.21.15.123:0/1298352438 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7eff200961c0 con 0x7eff2815afa0 2024-04-25T12:35:04.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.896+0000 7eff2d000640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7eff040744b0 0x7eff040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.896+0000 7eff369a3740 1 Processor -- start 2024-04-25T12:35:04.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.897+0000 7eff369a3740 1 -- start start 2024-04-25T12:35:04.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.897+0000 7eff2d000640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7eff040744b0 0x7eff040769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff18034000 tx=0x7eff1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.898+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.898+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c35430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.898+0000 7eff369a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c35970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.898+0000 7eff369a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a285b74600 con 0x55a28584d590 2024-04-25T12:35:04.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.898+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a285b11100 con 0x55a28586e270 2024-04-25T12:35:04.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.898+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55a285b705c0 con 0x55a2858453b0 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c35970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c35970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52414/0 (socket says 172.21.15.123:52414) 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 -- 172.21.15.123:0/672587151 learned_addr learned my addr 172.21.15.123:0/672587151 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff1fe00640 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c35430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2d000640 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 -- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 msgr2=0x55a285c35430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c35430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 -- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 msgr2=0x55a285c51e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c51e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 -- 172.21.15.123:0/672587151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2858862d0 con 0x55a28584d590 2024-04-25T12:35:04.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff2da00640 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c35970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7eff2005c000 tx=0x7eff2005fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eade40d79f88e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff1d600640 1 -- 172.21.15.123:0/672587151 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20064280 con 0x55a28584d590 2024-04-25T12:35:04.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.899+0000 7eff1d600640 1 -- 172.21.15.123:0/672587151 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff20068ca0 con 0x55a28584d590 2024-04-25T12:35:04.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff1d600640 1 -- 172.21.15.123:0/672587151 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2005f400 con 0x55a28584d590 2024-04-25T12:35:04.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 -- 172.21.15.123:0/672587151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 msgr2=0x55a285c35970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c35970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7eff2005c000 tx=0x7eff2005fa90 comp rx=0 tx=0).stop 2024-04-25T12:35:04.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 -- 172.21.15.123:0/672587151 shutdown_connections 2024-04-25T12:35:04.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c35970 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 --2- 172.21.15.123:0/672587151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 -- 172.21.15.123:0/672587151 >> 172.21.15.123:0/672587151 conn(0x55a2858623c0 msgr2=0x55a285c0fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.900+0000 7eff369a3740 1 -- 172.21.15.123:0/672587151 shutdown_connections 2024-04-25T12:35:04.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.901+0000 7eff369a3740 1 -- 172.21.15.123:0/672587151 wait complete. 2024-04-25T12:35:04.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.902+0000 7eff369a3740 1 -- start start 2024-04-25T12:35:04.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.902+0000 7eff369a3740 1 Processor -- start 2024-04-25T12:35:04.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.903+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.903+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28584d590 0x55a285c368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.903+0000 7eff369a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28586e270 0x55a285c32650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.903+0000 7eff369a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a285c0f890 con 0x55a28586e270 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.903+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a285afbf70 con 0x55a28584d590 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.903+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55a285c61bd0 con 0x55a2858453b0 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.903+0000 7eff369a3740 10 client.0 _ll_register_callbacks cb 0x55a285b6a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28586e270 0x55a285c32650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2d000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28586e270 0x55a285c32650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52426/0 (socket says 172.21.15.123:52426) 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 -- 172.21.15.123:0/911357315 learned_addr learned my addr 172.21.15.123:0/911357315 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff1fe00640 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28584d590 0x55a285c368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 -- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 msgr2=0x55a285c611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 -- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28584d590 msgr2=0x55a285c368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28584d590 0x55a285c368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 -- 172.21.15.123:0/911357315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2859506b0 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff2da00640 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28586e270 0x55a285c32650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff2005c570 tx=0x7eff2001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcabc3b1eecd762c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff16000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff16000640 1 -- 172.21.15.123:0/911357315 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2007d070 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff16000640 1 -- 172.21.15.123:0/911357315 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7eff200813b0 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.904+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff20068ca0 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff16000640 1 -- 172.21.15.123:0/911357315 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7eff20088050 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 10 client.4785 fetch_fsmap learned FSMap version 14 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55a285c60c70 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff16000640 1 -- 172.21.15.123:0/911357315 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff20068ca0 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff16000640 1 -- 172.21.15.123:0/911357315 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 14) ==== 4752+0+0 (secure 0 0 0) 0x7eff2008b020 con 0x55a28586e270 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 10 client.4785 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T12:35:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 10 client.4785 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T12:35:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 20 client.4785 populate_metadata read hostname 'smithi123' 2024-04-25T12:35:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a28582ce40 0x55a285c6f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 4 client.4785 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:35:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.905+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 --> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a285b82210 con 0x55a28582ce40 2024-04-25T12:35:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.906+0000 7eff2d000640 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a28582ce40 0x55a285c6f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.906+0000 7eff2d000640 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a28582ce40 0x55a285c6f8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25ae53e599535aca server_cookie=c8ef5e1e125ded45 in_seq=0 out_seq=0 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.906+0000 7eff16000640 10 client.4785 ms_handle_connect on v2:172.21.15.153:6838/4167745001 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff16000640 1 -- 172.21.15.123:0/911357315 <== mds.1 v2:172.21.15.153:6838/4167745001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a285b82210 con 0x55a28582ce40 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff16000640 10 client.4785 handle_command_reply: tid=0 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 client.4785 shutdown 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28586e270 msgr2=0x55a285c32650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28586e270 0x55a285c32650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff2005c570 tx=0x7eff2001f040 comp rx=0 tx=0).stop 2024-04-25T12:35:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 shutdown_connections 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a28582ce40 0x55a285c6f8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28586e270 0x55a285c32650 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28584d590 0x55a285c368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 --2- 172.21.15.123:0/911357315 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff16000640 0 client.4785 ms_handle_reset on v2:172.21.15.153:6838/4167745001 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.907+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 >> 172.21.15.123:0/911357315 conn(0x55a2858623c0 msgr2=0x55a285c51d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.908+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 shutdown_connections 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.908+0000 7eff369a3740 1 -- 172.21.15.123:0/911357315 wait complete. 2024-04-25T12:35:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.908+0000 7eff369a3740 20 client.4785 trim_cache size 0 max 16384 2024-04-25T12:35:04.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.931+0000 7eff369a3740 1 Processor -- start 2024-04-25T12:35:04.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.931+0000 7eff369a3740 1 -- start start 2024-04-25T12:35:04.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.932+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c0e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.932+0000 7eff369a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.932+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c50550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.932+0000 7eff369a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff281f6e30 con 0x55a28584d590 2024-04-25T12:35:04.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.932+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a285c35590 con 0x55a28586e270 2024-04-25T12:35:04.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.932+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55a285c48520 con 0x55a2858453b0 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.932+0000 7eff1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52436/0 (socket says 172.21.15.123:52436) 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 -- 172.21.15.123:0/1889291837 learned_addr learned my addr 172.21.15.123:0/1889291837 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff2d000640 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c0e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff2da00640 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c50550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 -- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 msgr2=0x55a285c0e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c0e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 -- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 msgr2=0x55a285c50550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c50550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 -- 172.21.15.123:0/1889291837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a285c0c8f0 con 0x55a28584d590 2024-04-25T12:35:04.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.933+0000 7eff1fe00640 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0ece0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7eff280b3270 tx=0x7eff10007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dc2d95fcfa4e07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.934+0000 7eff1cc00640 1 -- 172.21.15.123:0/1889291837 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10005320 con 0x55a28584d590 2024-04-25T12:35:04.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.934+0000 7eff1cc00640 1 -- 172.21.15.123:0/1889291837 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff10015910 con 0x55a28584d590 2024-04-25T12:35:04.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.934+0000 7eff1cc00640 1 -- 172.21.15.123:0/1889291837 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10015cc0 con 0x55a28584d590 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.934+0000 7eff369a3740 1 -- 172.21.15.123:0/1889291837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 msgr2=0x55a285c0ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.934+0000 7eff369a3740 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0ece0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7eff280b3270 tx=0x7eff10007e50 comp rx=0 tx=0).stop 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.935+0000 7eff369a3740 1 -- 172.21.15.123:0/1889291837 shutdown_connections 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.935+0000 7eff369a3740 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28586e270 0x55a285c50550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.935+0000 7eff369a3740 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0ece0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.935+0000 7eff369a3740 1 --2- 172.21.15.123:0/1889291837 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a2858453b0 0x55a285c0e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.935+0000 7eff369a3740 1 -- 172.21.15.123:0/1889291837 >> 172.21.15.123:0/1889291837 conn(0x55a2858623c0 msgr2=0x55a285c366c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.935+0000 7eff369a3740 1 -- 172.21.15.123:0/1889291837 shutdown_connections 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.935+0000 7eff369a3740 1 -- 172.21.15.123:0/1889291837 wait complete. 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.936+0000 7eff369a3740 1 -- start start 2024-04-25T12:35:04.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.936+0000 7eff369a3740 1 Processor -- start 2024-04-25T12:35:04.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.937+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2858453b0 0x55a285c33160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.937+0000 7eff369a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.937+0000 7eff369a3740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a28586e270 0x55a285cd1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.937+0000 7eff369a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a285c360d0 con 0x55a28584d590 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.937+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a285c36df0 con 0x55a2858453b0 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.937+0000 7eff369a3740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55a285c64510 con 0x55a28586e270 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.937+0000 7eff369a3740 10 client.0 _ll_register_callbacks cb 0x7eff200095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52452/0 (socket says 172.21.15.123:52452) 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 -- 172.21.15.123:0/1991992834 learned_addr learned my addr 172.21.15.123:0/1991992834 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff2d000640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2858453b0 0x55a285c33160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff2da00640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a28586e270 0x55a285cd1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 -- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a28586e270 msgr2=0x55a285cd1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a28586e270 0x55a285cd1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 -- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2858453b0 msgr2=0x55a285c33160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2858453b0 0x55a285c33160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 -- 172.21.15.123:0/1991992834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a285c36530 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff2d000640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2858453b0 0x55a285c33160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff1fe00640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0eba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff10001110 tx=0x7eff10000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1733d1cbb956626c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.938+0000 7eff16000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff16000640 1 -- 172.21.15.123:0/1991992834 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10015cd0 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff16000640 1 -- 172.21.15.123:0/1991992834 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7eff100067d0 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff10015910 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff16000640 1 -- 172.21.15.123:0/1991992834 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7eff10015e70 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 10 client.4791 fetch_fsmap learned FSMap version 14 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55a285c2fb30 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff16000640 1 -- 172.21.15.123:0/1991992834 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff10015910 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff16000640 1 -- 172.21.15.123:0/1991992834 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 14) ==== 4752+0+0 (secure 0 0 0) 0x7eff100069d0 con 0x55a28584d590 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 10 client.4791 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T12:35:04.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 10 client.4791 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T12:35:04.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 20 client.4791 populate_metadata read hostname 'smithi123' 2024-04-25T12:35:04.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a285c59290 0x55a285ce40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:04.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 4 client.4791 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:35:04.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.939+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 --> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] -- command(tid 0: {"prefix": "status"}) -- 0x55a285b7c080 con 0x55a285c59290 2024-04-25T12:35:04.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.940+0000 7eff2d000640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a285c59290 0x55a285ce40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:04.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.940+0000 7eff2d000640 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a285c59290 0x55a285ce40d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a6b7e0b049469b5 server_cookie=a8ff2bcfed30d4aa in_seq=0 out_seq=0 2024-04-25T12:35:04.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.940+0000 7eff16000640 10 client.4791 ms_handle_connect on v2:172.21.15.153:6838/4167745001 2024-04-25T12:35:04.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.940+0000 7eff16000640 1 -- 172.21.15.123:0/1991992834 <== mds.1 v2:172.21.15.153:6838/4167745001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55a285b7c080 con 0x55a285c59290 2024-04-25T12:35:04.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.940+0000 7eff16000640 10 client.4791 handle_command_reply: tid=0 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.940+0000 7eff369a3740 1 client.4791 shutdown 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 msgr2=0x55a285c0eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0eba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff10001110 tx=0x7eff10000bc0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 shutdown_connections 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:6838/4167745001,v1:172.21.15.153:6839/4167745001] conn(0x55a285c59290 0x55a285ce40d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a28586e270 0x55a285cd1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a28584d590 0x55a285c0eba0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 --2- 172.21.15.123:0/1991992834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2858453b0 0x55a285c33160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff16000640 0 client.4791 ms_handle_reset on v2:172.21.15.153:6838/4167745001 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 >> 172.21.15.123:0/1991992834 conn(0x55a2858623c0 msgr2=0x55a285c343a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 shutdown_connections 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 1 -- 172.21.15.123:0/1991992834 wait complete. 2024-04-25T12:35:04.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.941+0000 7eff369a3740 20 client.4791 trim_cache size 0 max 16384 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 1, 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4264, 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 19.831177457999999, 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 14, 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 27, 2024-04-25T12:35:04.945 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T12:35:04.946 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 57.409291562 2024-04-25T12:35:04.946 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:04.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7eff040744b0 msgr2=0x7eff040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7eff040744b0 0x7eff040769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff18034000 tx=0x7eff1802e000 comp rx=0 tx=0).stop 2024-04-25T12:35:04.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff2815afa0 msgr2=0x7eff280c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff2815afa0 0x7eff280c3aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff2000efc0 tx=0x7eff2000c490 comp rx=0 tx=0).stop 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 shutdown_connections 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7eff040744b0 0x7eff040769c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff2815afa0 0x7eff280c3aa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7eff280c4d50 0x7eff281f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 --2- 172.21.15.123:0/1298352438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff28074870 0x7eff2814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 >> 172.21.15.123:0/1298352438 conn(0x7eff2806c750 msgr2=0x7eff2806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 shutdown_connections 2024-04-25T12:35:04.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:04.942+0000 7eff2fa00640 1 -- 172.21.15.123:0/1298352438 wait complete. 2024-04-25T12:35:04.967 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T12:35:05.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.243+0000 7f7e95400640 1 Processor -- start 2024-04-25T12:35:05.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.243+0000 7f7e95400640 1 -- start start 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.245+0000 7f7e95400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e900c9080 0x7f7e901546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e95400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7e900c82b0 0x7f7e9015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e95400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e9015e590 0x7f7e9006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e95400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9015a330 con 0x7f7e9015e590 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e95400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7e90073b10 con 0x7f7e900c82b0 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e95400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7e90073df0 con 0x7f7e900c9080 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e94a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e9015e590 0x7f7e9006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e94a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e9015e590 0x7f7e9006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52458/0 (socket says 172.21.15.123:52458) 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e94a00640 1 -- 172.21.15.123:0/2260422912 learned_addr learned my addr 172.21.15.123:0/2260422912 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e87400640 1 --2- 172.21.15.123:0/2260422912 >> v2:172.21.15.153:3300/0 conn(0x7f7e900c82b0 0x7f7e9015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2260422912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e900c9080 0x7f7e901546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e94a00640 1 -- 172.21.15.123:0/2260422912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e900c9080 msgr2=0x7f7e901546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e94a00640 1 --2- 172.21.15.123:0/2260422912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e900c9080 0x7f7e901546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e94a00640 1 -- 172.21.15.123:0/2260422912 >> v2:172.21.15.153:3300/0 conn(0x7f7e900c82b0 msgr2=0x7f7e9015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.246+0000 7f7e94a00640 1 --2- 172.21.15.123:0/2260422912 >> v2:172.21.15.153:3300/0 conn(0x7f7e900c82b0 0x7f7e9015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.247+0000 7f7e94a00640 1 -- 172.21.15.123:0/2260422912 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e900740d0 con 0x7f7e9015e590 2024-04-25T12:35:05.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.247+0000 7f7e87400640 1 --2- 172.21.15.123:0/2260422912 >> v2:172.21.15.153:3300/0 conn(0x7f7e900c82b0 0x7f7e9015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:05.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.248+0000 7f7e94a00640 1 --2- 172.21.15.123:0/2260422912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e9015e590 0x7f7e9006c570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e900ce480 tx=0x7f7e8802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aeb10b351a8344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.248+0000 7f7e8f400640 1 -- 172.21.15.123:0/2260422912 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88036070 con 0x7f7e9015e590 2024-04-25T12:35:05.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.248+0000 7f7e8f400640 1 -- 172.21.15.123:0/2260422912 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e88031600 con 0x7f7e9015e590 2024-04-25T12:35:05.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.248+0000 7f7e8f400640 1 -- 172.21.15.123:0/2260422912 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880319d0 con 0x7f7e9015e590 2024-04-25T12:35:05.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 -- 172.21.15.123:0/2260422912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e9015e590 msgr2=0x7f7e9006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 --2- 172.21.15.123:0/2260422912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e9015e590 0x7f7e9006c570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e900ce480 tx=0x7f7e8802a6a0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 -- 172.21.15.123:0/2260422912 shutdown_connections 2024-04-25T12:35:05.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 --2- 172.21.15.123:0/2260422912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e9015e590 0x7f7e9006c570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 --2- 172.21.15.123:0/2260422912 >> v2:172.21.15.153:3300/0 conn(0x7f7e900c82b0 0x7f7e9015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 --2- 172.21.15.123:0/2260422912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e900c9080 0x7f7e901546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 -- 172.21.15.123:0/2260422912 >> 172.21.15.123:0/2260422912 conn(0x7f7e900cd600 msgr2=0x7f7e900cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:05.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 -- 172.21.15.123:0/2260422912 shutdown_connections 2024-04-25T12:35:05.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.249+0000 7f7e95400640 1 -- 172.21.15.123:0/2260422912 wait complete. 2024-04-25T12:35:05.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.251+0000 7f7e95400640 1 Processor -- start 2024-04-25T12:35:05.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.251+0000 7f7e95400640 1 -- start start 2024-04-25T12:35:05.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e95400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e900c82b0 0x7f7e9015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e95400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e900c9080 0x7f7e90153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e95400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e9015e590 0x7f7e90144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e95400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90073760 con 0x7f7e900c9080 2024-04-25T12:35:05.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e95400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9015db30 con 0x7f7e900c82b0 2024-04-25T12:35:05.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e95400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7e90152e50 con 0x7f7e9015e590 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e900c9080 0x7f7e90153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e900c9080 0x7f7e90153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52474/0 (socket says 172.21.15.123:52474) 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e87400640 1 -- 172.21.15.123:0/2545118829 learned_addr learned my addr 172.21.15.123:0/2545118829 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e900c82b0 0x7f7e9015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.253+0000 7f7e94a00640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e9015e590 0x7f7e90144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8fe00640 1 -- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e9015e590 msgr2=0x7f7e90144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e9015e590 0x7f7e90144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8fe00640 1 -- 172.21.15.123:0/2545118829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e900c9080 msgr2=0x7f7e90153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e900c9080 0x7f7e90153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8fe00640 1 -- 172.21.15.123:0/2545118829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9015cf30 con 0x7f7e900c82b0 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e87400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e900c9080 0x7f7e90153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:05.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e900c82b0 0x7f7e9015c7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000e870 tx=0x7f7e8000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8cc00640 1 -- 172.21.15.123:0/2545118829 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80017070 con 0x7f7e900c82b0 2024-04-25T12:35:05.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8cc00640 1 -- 172.21.15.123:0/2545118829 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8000ee60 con 0x7f7e900c82b0 2024-04-25T12:35:05.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9015d4b0 con 0x7f7e900c82b0 2024-04-25T12:35:05.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.254+0000 7f7e8cc00640 1 -- 172.21.15.123:0/2545118829 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000cb60 con 0x7f7e900c82b0 2024-04-25T12:35:05.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.255+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e900ccab0 con 0x7f7e900c82b0 2024-04-25T12:35:05.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.257+0000 7f7e8cc00640 1 -- 172.21.15.123:0/2545118829 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e8001a070 con 0x7f7e900c82b0 2024-04-25T12:35:05.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.258+0000 7f7e8cc00640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e6c0744b0 0x7f7e6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.258+0000 7f7e8cc00640 1 -- 172.21.15.123:0/2545118829 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f7e80095750 con 0x7f7e900c82b0 2024-04-25T12:35:05.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.258+0000 7f7e87400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e6c0744b0 0x7f7e6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.258+0000 7f7e9d5fd740 1 Processor -- start 2024-04-25T12:35:05.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.259+0000 7f7e9d5fd740 1 -- start start 2024-04-25T12:35:05.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.259+0000 7f7e87400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e6c0744b0 0x7f7e6c0769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009560 tx=0x7f7e78009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af98619fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af985fd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55af9853c600 con 0x55af98215590 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af984d9100 con 0x55af98236270 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55af985385c0 con 0x55af9820d3b0 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e8fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af98619fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e87400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af985fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e8fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af98619fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:50832/0 (socket says 172.21.15.123:50832) 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e8fe00640 1 -- 172.21.15.123:0/2344354922 learned_addr learned my addr 172.21.15.123:0/2344354922 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:05.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.260+0000 7f7e94a00640 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e8fe00640 1 -- 172.21.15.123:0/2344354922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 msgr2=0x55af985fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af985fd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e8fe00640 1 -- 172.21.15.123:0/2344354922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 msgr2=0x55af985fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e8fe00640 1 -- 172.21.15.123:0/2344354922 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9824e2d0 con 0x55af98236270 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af98619fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e80093810 tx=0x7f7e8005ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad421967681e0d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e7fe00640 1 -- 172.21.15.123:0/2344354922 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80021040 con 0x55af98236270 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e7fe00640 1 -- 172.21.15.123:0/2344354922 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80063810 con 0x55af98236270 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.261+0000 7f7e7fe00640 1 -- 172.21.15.123:0/2344354922 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80068230 con 0x55af98236270 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/2344354922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 msgr2=0x55af98619fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af98619fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e80093810 tx=0x7f7e8005ec80 comp rx=0 tx=0).stop 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/2344354922 shutdown_connections 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af985fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/2344354922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af98619fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/2344354922 >> 172.21.15.123:0/2344354922 conn(0x55af9822a3c0 msgr2=0x55af985d7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:05.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/2344354922 shutdown_connections 2024-04-25T12:35:05.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.262+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/2344354922 wait complete. 2024-04-25T12:35:05.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.263+0000 7f7e9d5fd740 1 -- start start 2024-04-25T12:35:05.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.264+0000 7f7e9d5fd740 1 Processor -- start 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af986146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af98615d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af985d9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55af985d7a60 con 0x55af98215590 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af984c3f70 con 0x55af98236270 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55af9862cbf0 con 0x55af9820d3b0 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e9d5fd740 10 client.0 _ll_register_callbacks cb 0x55af98532480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af98615d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af98615d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52492/0 (socket says 172.21.15.123:52492) 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.265+0000 7f7e87400640 1 -- 172.21.15.123:0/1334164526 learned_addr learned my addr 172.21.15.123:0/1334164526 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e94a00640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af985d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af986146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e87400640 1 -- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 msgr2=0x55af986146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e87400640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af986146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e87400640 1 -- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 msgr2=0x55af985d9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e87400640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af985d9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e87400640 1 -- 172.21.15.123:0/1334164526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af983186b0 con 0x55af98215590 2024-04-25T12:35:05.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af986146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e87400640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af98615d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e78014e40 tx=0x7f7e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ee5a9e9948ba45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.266+0000 7f7e7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e7cc00640 1 -- 172.21.15.123:0/1334164526 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78027030 con 0x55af98215590 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e80063810 con 0x55af98215590 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e7cc00640 1 -- 172.21.15.123:0/1334164526 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f7e7800f040 con 0x55af98215590 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e7cc00640 1 -- 172.21.15.123:0/1334164526 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f7e78013bf0 con 0x55af98215590 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e9d5fd740 10 client.4803 fetch_fsmap learned FSMap version 14 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55af986141c0 con 0x55af98215590 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e7cc00640 1 -- 172.21.15.123:0/1334164526 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80063810 con 0x55af98215590 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.267+0000 7f7e7cc00640 1 -- 172.21.15.123:0/1334164526 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 14) ==== 4752+0+0 (secure 0 0 0) 0x7f7e78036080 con 0x55af98215590 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.268+0000 7f7e9d5fd740 10 client.4803 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.268+0000 7f7e9d5fd740 10 client.4803 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.268+0000 7f7e9d5fd740 20 client.4803 populate_metadata read hostname 'smithi123' 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.268+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af981f4e40 0x55af98637f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.268+0000 7f7e9d5fd740 4 client.4803 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:35:05.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.268+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 --> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af9854a210 con 0x55af981f4e40 2024-04-25T12:35:05.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.268+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af981f4e40 0x55af98637f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.269+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af981f4e40 0x55af98637f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25be23c73545d689 server_cookie=5214e288e9f6f165 in_seq=0 out_seq=0 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.269+0000 7f7e7cc00640 10 client.4803 ms_handle_connect on v2:172.21.15.123:6832/1720190095 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.269+0000 7f7e7cc00640 1 -- 172.21.15.123:0/1334164526 <== mds.0 v2:172.21.15.123:6832/1720190095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af9854a210 con 0x55af981f4e40 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.269+0000 7f7e7cc00640 10 client.4803 handle_command_reply: tid=0 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.269+0000 7f7e9d5fd740 1 client.4803 shutdown 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 msgr2=0x55af98615d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af98615d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e78014e40 tx=0x7f7e780046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 shutdown_connections 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af98236270 0x55af985d9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af981f4e40 0x55af98637f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af98615d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/1334164526 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af9820d3b0 0x55af986146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e7cc00640 0 client.4803 ms_handle_reset on v2:172.21.15.123:6832/1720190095 2024-04-25T12:35:05.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 >> 172.21.15.123:0/1334164526 conn(0x55af9822a3c0 msgr2=0x55af985d8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:05.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 shutdown_connections 2024-04-25T12:35:05.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/1334164526 wait complete. 2024-04-25T12:35:05.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.270+0000 7f7e9d5fd740 20 client.4803 trim_cache size 0 max 16384 2024-04-25T12:35:05.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.291+0000 7f7e9d5fd740 1 Processor -- start 2024-04-25T12:35:05.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.291+0000 7f7e9d5fd740 1 -- start start 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9862d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98236270 0x55af985d8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901fcb10 con 0x55af98215590 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af98612b40 con 0x55af9820d3b0 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55af9861cb90 con 0x55af98236270 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985d8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52494/0 (socket says 172.21.15.123:52494) 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e87400640 1 -- 172.21.15.123:0/4285908905 learned_addr learned my addr 172.21.15.123:0/4285908905 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e94a00640 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98236270 0x55af985d8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.293+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9862d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e87400640 1 -- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98236270 msgr2=0x55af985d8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e87400640 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98236270 0x55af985d8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e87400640 1 -- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 msgr2=0x55af9862d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e87400640 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9862d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e87400640 1 -- 172.21.15.123:0/4285908905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af98616120 con 0x55af98215590 2024-04-25T12:35:05.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e87400640 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985d8730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000c00 tx=0x7f7e7801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144ab11996379456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e7f400640 1 -- 172.21.15.123:0/4285908905 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78027070 con 0x55af98215590 2024-04-25T12:35:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e7f400640 1 -- 172.21.15.123:0/4285908905 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78039020 con 0x55af98215590 2024-04-25T12:35:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.294+0000 7f7e7f400640 1 -- 172.21.15.123:0/4285908905 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78013770 con 0x55af98215590 2024-04-25T12:35:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/4285908905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 msgr2=0x55af985d8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985d8730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000c00 tx=0x7f7e7801b000 comp rx=0 tx=0).stop 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/4285908905 shutdown_connections 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98236270 0x55af985d8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98215590 0x55af985d8730 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/4285908905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9862d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/4285908905 >> 172.21.15.123:0/4285908905 conn(0x55af9822a3c0 msgr2=0x55af985ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/4285908905 shutdown_connections 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.295+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/4285908905 wait complete. 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.296+0000 7f7e9d5fd740 1 -- start start 2024-04-25T12:35:05.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.297+0000 7f7e9d5fd740 1 Processor -- start 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9860f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98215590 0x55af9869f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e9d5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98236270 0x55af986955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55af98611740 con 0x55af98236270 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af98600df0 con 0x55af9820d3b0 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e9d5fd740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55af98611e10 con 0x55af98215590 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e9d5fd740 10 client.0 _ll_register_callbacks cb 0x7f7e780093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e94a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98236270 0x55af986955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e94a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98236270 0x55af986955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52510/0 (socket says 172.21.15.123:52510) 2024-04-25T12:35:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e94a00640 1 -- 172.21.15.123:0/269630880 learned_addr learned my addr 172.21.15.123:0/269630880 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9860f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.298+0000 7f7e87400640 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98215590 0x55af9869f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e94a00640 1 -- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98215590 msgr2=0x55af9869f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e94a00640 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98215590 0x55af9869f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e94a00640 1 -- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 msgr2=0x55af9860f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e94a00640 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9860f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e94a00640 1 -- 172.21.15.123:0/269630880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af98602b10 con 0x55af98236270 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e94a00640 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98236270 0x55af986955f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009f10 tx=0x7f7e88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9162d058972fe3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.299+0000 7f7e7cc00640 1 -- 172.21.15.123:0/269630880 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88036040 con 0x55af98236270 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e78039020 con 0x55af98236270 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e7cc00640 1 -- 172.21.15.123:0/269630880 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f7e88040c90 con 0x55af98236270 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e7cc00640 1 -- 172.21.15.123:0/269630880 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f7e88050050 con 0x55af98236270 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 10 client.4809 fetch_fsmap learned FSMap version 14 2024-04-25T12:35:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55af98600750 con 0x55af98236270 2024-04-25T12:35:05.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e7cc00640 1 -- 172.21.15.123:0/269630880 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78039020 con 0x55af98236270 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e7cc00640 1 -- 172.21.15.123:0/269630880 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 14) ==== 4752+0+0 (secure 0 0 0) 0x7f7e88053020 con 0x55af98236270 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 10 client.4809 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 10 client.4809 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 20 client.4809 populate_metadata read hostname 'smithi123' 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af98618060 0x55af986a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 4 client.4809 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.300+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 --> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] -- command(tid 0: {"prefix": "status"}) -- 0x55af98544080 con 0x55af98618060 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.301+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af98618060 0x55af986a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.301+0000 7f7e8fe00640 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af98618060 0x55af986a8040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d08a438d0be183 server_cookie=ab4bde6956c103e5 in_seq=0 out_seq=0 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.301+0000 7f7e7cc00640 10 client.4809 ms_handle_connect on v2:172.21.15.123:6832/1720190095 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.301+0000 7f7e7cc00640 1 -- 172.21.15.123:0/269630880 <== mds.0 v2:172.21.15.123:6832/1720190095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55af98544080 con 0x55af98618060 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.301+0000 7f7e7cc00640 10 client.4809 handle_command_reply: tid=0 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.301+0000 7f7e9d5fd740 1 client.4809 shutdown 2024-04-25T12:35:05.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98236270 msgr2=0x55af986955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98236270 0x55af986955f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009f10 tx=0x7f7e88004ab0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 shutdown_connections 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:6832/1720190095,v1:172.21.15.123:6834/1720190095] conn(0x55af98618060 0x55af986a8040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af98236270 0x55af986955f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55af98215590 0x55af9869f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 --2- 172.21.15.123:0/269630880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af9820d3b0 0x55af9860f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 >> 172.21.15.123:0/269630880 conn(0x55af9822a3c0 msgr2=0x55af985ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e7cc00640 0 client.4809 ms_handle_reset on v2:172.21.15.123:6832/1720190095 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 shutdown_connections 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 1 -- 172.21.15.123:0/269630880 wait complete. 2024-04-25T12:35:05.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.302+0000 7f7e9d5fd740 20 client.4809 trim_cache size 0 max 16384 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4270, 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 51.363067459, 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 14, 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 27, 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 57.658549219000001 2024-04-25T12:35:05.306 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:05.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e6c0744b0 msgr2=0x7f7e6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e6c0744b0 0x7f7e6c0769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009560 tx=0x7f7e78009190 comp rx=0 tx=0).stop 2024-04-25T12:35:05.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e900c82b0 msgr2=0x7f7e9015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e900c82b0 0x7f7e9015c7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000e870 tx=0x7f7e8000ed30 comp rx=0 tx=0).stop 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 shutdown_connections 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e6c0744b0 0x7f7e6c0769c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7e9015e590 0x7f7e90144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e900c9080 0x7f7e90153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 --2- 172.21.15.123:0/2545118829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e900c82b0 0x7f7e9015c7c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 >> 172.21.15.123:0/2545118829 conn(0x7f7e900cd600 msgr2=0x7f7e9006cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 shutdown_connections 2024-04-25T12:35:05.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.303+0000 7f7e95400640 1 -- 172.21.15.123:0/2545118829 wait complete. 2024-04-25T12:35:05.327 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T12:35:05.337 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T12:35:05.337 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T12:35:05.337 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T12:35:05.337 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T12:35:05.483 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T12:35:05.483 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T12:35:05.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:05.484 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T12:35:05.542 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T12:35:05.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:05.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T12:35:05.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:05.629 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T12:35:05.735 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T12:35:05.735 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T12:35:05.735 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:05.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T12:35:05.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.984+0000 7fde92800640 1 Processor -- start 2024-04-25T12:35:05.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.985+0000 7fde92800640 1 -- start start 2024-04-25T12:35:05.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.987+0000 7fde92800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde8c154890 0x7fde8c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.987+0000 7fde92800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8c154ca0 con 0x7fde8c074870 2024-04-25T12:35:05.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.987+0000 7fde92800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8c073ce0 con 0x7fde8c0c4d50 2024-04-25T12:35:05.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.987+0000 7fde92800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fde8c0c61f0 con 0x7fde8c154890 2024-04-25T12:35:05.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.987+0000 7fde8aa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fde8c074870 0x7fde8c0cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40270/0 (socket says 172.21.15.123:40270) 2024-04-25T12:35:05.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.987+0000 7fde8aa00640 1 -- 172.21.15.123:0/2236912231 learned_addr learned my addr 172.21.15.123:0/2236912231 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:05.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.987+0000 7fde8be00640 1 --2- 172.21.15.123:0/2236912231 >> v2:172.21.15.123:3300/0 conn(0x7fde8c154890 0x7fde8c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.988+0000 7fde8a000640 1 -- 172.21.15.123:0/2236912231 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519746039 0 0) 0x7fde8c154ca0 con 0x7fde8c074870 2024-04-25T12:35:05.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.988+0000 7fde8a000640 1 -- 172.21.15.123:0/2236912231 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde70003520 con 0x7fde8c074870 2024-04-25T12:35:05.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.988+0000 7fde8be00640 1 -- 172.21.15.123:0/2236912231 >> v1:172.21.15.153:6789/0 conn(0x7fde8c0c4d50 legacy=0x7fde8c0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.988+0000 7fde8a000640 1 -- 172.21.15.123:0/2236912231 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 608258767 0 0) 0x7fde8c073ce0 con 0x7fde8c0c4d50 2024-04-25T12:35:05.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.988+0000 7fde8be00640 1 -- 172.21.15.123:0/2236912231 >> v1:172.21.15.123:6789/0 conn(0x7fde8c074870 legacy=0x7fde8c0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.988+0000 7fde8be00640 1 -- 172.21.15.123:0/2236912231 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c073fc0 con 0x7fde8c154890 2024-04-25T12:35:05.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.989+0000 7fde8be00640 1 --2- 172.21.15.123:0/2236912231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c15afa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde84009d90 tx=0x7fde8402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cce00efde08cd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.989+0000 7fde8a000640 1 -- 172.21.15.123:0/2236912231 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84037070 con 0x7fde8c154890 2024-04-25T12:35:05.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.989+0000 7fde8a000640 1 -- 172.21.15.123:0/2236912231 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84007740 con 0x7fde8c154890 2024-04-25T12:35:05.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.989+0000 7fde8a000640 1 -- 172.21.15.123:0/2236912231 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84007b10 con 0x7fde8c154890 2024-04-25T12:35:05.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.990+0000 7fde92800640 1 -- 172.21.15.123:0/2236912231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 msgr2=0x7fde8c15afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.990+0000 7fde92800640 1 --2- 172.21.15.123:0/2236912231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c15afa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde84009d90 tx=0x7fde8402a780 comp rx=0 tx=0).stop 2024-04-25T12:35:05.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.990+0000 7fde92800640 1 -- 172.21.15.123:0/2236912231 shutdown_connections 2024-04-25T12:35:05.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.990+0000 7fde92800640 1 --2- 172.21.15.123:0/2236912231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c15afa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.990+0000 7fde92800640 1 -- 172.21.15.123:0/2236912231 >> 172.21.15.123:0/2236912231 conn(0x7fde8c06c750 msgr2=0x7fde8c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:05.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.990+0000 7fde92800640 1 -- 172.21.15.123:0/2236912231 shutdown_connections 2024-04-25T12:35:05.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.990+0000 7fde92800640 1 -- 172.21.15.123:0/2236912231 wait complete. 2024-04-25T12:35:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.992+0000 7fde92800640 1 Processor -- start 2024-04-25T12:35:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.993+0000 7fde92800640 1 -- start start 2024-04-25T12:35:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde92800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde8c074870 0x7fde8c1e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde92800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c0c4d50 0x7fde8c1e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde92800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c0c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde92800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c0c8460 con 0x7fde8c154890 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde92800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c0c87d0 con 0x7fde8c0c4d50 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde92800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c0c9370 con 0x7fde8c074870 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde8be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c0c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde8be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c0c33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52522/0 (socket says 172.21.15.123:52522) 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde8be00640 1 -- 172.21.15.123:0/677938576 learned_addr learned my addr 172.21.15.123:0/677938576 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde8aa00640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c0c4d50 0x7fde8c1e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.994+0000 7fde8b400640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde8c074870 0x7fde8c1e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.995+0000 7fde8be00640 1 -- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde8c074870 msgr2=0x7fde8c1e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.995+0000 7fde8be00640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde8c074870 0x7fde8c1e6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.995+0000 7fde8be00640 1 -- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c0c4d50 msgr2=0x7fde8c1e5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.995+0000 7fde8be00640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c0c4d50 0x7fde8c1e5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.995+0000 7fde8be00640 1 -- 172.21.15.123:0/677938576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c1f2f80 con 0x7fde8c154890 2024-04-25T12:35:05.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.995+0000 7fde8be00640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c0c33d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde84035040 tx=0x7fde84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.995+0000 7fde83400640 1 -- 172.21.15.123:0/677938576 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84042030 con 0x7fde8c154890 2024-04-25T12:35:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.996+0000 7fde83400640 1 -- 172.21.15.123:0/677938576 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84007540 con 0x7fde8c154890 2024-04-25T12:35:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.996+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c1f3260 con 0x7fde8c154890 2024-04-25T12:35:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.996+0000 7fde83400640 1 -- 172.21.15.123:0/677938576 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84037040 con 0x7fde8c154890 2024-04-25T12:35:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.996+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1e61d0 con 0x7fde8c154890 2024-04-25T12:35:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.997+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8c1e6520 con 0x7fde8c154890 2024-04-25T12:35:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.998+0000 7fde83400640 1 -- 172.21.15.123:0/677938576 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde84045070 con 0x7fde8c154890 2024-04-25T12:35:05.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.999+0000 7fde83400640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde6c0744e0 0x7fde6c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:05.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.999+0000 7fde83400640 1 -- 172.21.15.123:0/677938576 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fde840b5f40 con 0x7fde8c154890 2024-04-25T12:35:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:05.999+0000 7fde8b400640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde6c0744e0 0x7fde6c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.000+0000 7fde8b400640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde6c0744e0 0x7fde6c0769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000f60 tx=0x7fde7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:06.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.004+0000 7fde83400640 1 -- 172.21.15.123:0/677938576 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde8404d020 con 0x7fde8c154890 2024-04-25T12:35:06.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.242+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fde8c15b820 con 0x7fde8c154890 2024-04-25T12:35:06.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.243+0000 7fde83400640 1 -- 172.21.15.123:0/677938576 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+88 (secure 0 0 0) 0x7fde8407fc70 con 0x7fde8c154890 2024-04-25T12:35:06.243 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T12:35:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.245+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde6c0744e0 msgr2=0x7fde6c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.245+0000 7fde92800640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde6c0744e0 0x7fde6c0769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000f60 tx=0x7fde7c009040 comp rx=0 tx=0).stop 2024-04-25T12:35:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.245+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 msgr2=0x7fde8c0c33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.245+0000 7fde92800640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c0c33d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde84035040 tx=0x7fde84004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 shutdown_connections 2024-04-25T12:35:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde6c0744e0 0x7fde6c0769f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:06.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde8c154890 0x7fde8c0c33d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:06.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c0c4d50 0x7fde8c1e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:06.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 --2- 172.21.15.123:0/677938576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde8c074870 0x7fde8c1e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:06.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 >> 172.21.15.123:0/677938576 conn(0x7fde8c06c750 msgr2=0x7fde8c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:06.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 shutdown_connections 2024-04-25T12:35:06.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06.246+0000 7fde92800640 1 -- 172.21.15.123:0/677938576 wait complete. 2024-04-25T12:35:06.270 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T12:35:06.271 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T12:35:06.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:06.271 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-25T12:35:06.320 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T12:35:06.320 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T12:35:06.320 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42657sec preferred_lft 42657sec 2024-04-25T12:35:06.321 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-25T12:35:06.322 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:35:06.322 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T12:35:06.323 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:35:06.323 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:35:06.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-25T12:35:06.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-25T12:35:06.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-25T12:35:06.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T12:35:06.323 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:35:06.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:35:06.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:06.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:06.596 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T12:35:06.656 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-25T12:35:06.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:06.660 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-25T12:35:06.699 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-25T12:35:06.699 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T12:35:06.700 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T12:35:06.700 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T12:35:06.700 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T12:35:06.700 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:35:06.700 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:35:06.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T12:35:06.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T12:35:06.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T12:35:06.700 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:35:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:35:06.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:06.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:06.941 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T12:35:06.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:06.988 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-25T12:35:07.085 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:35:07.085 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:35:07.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T12:35:07.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T12:35:07.085 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:35:07.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:35:07.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:35:07.247 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:35:07.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:07.273 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T12:35:07.273 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:35:07.273 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:35:07.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T12:35:07.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T12:35:07.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T12:35:07.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T12:35:07.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T12:35:07.274 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:35:07.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:35:07.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:07.583 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:35:07.583 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T12:35:07.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-25T12:35:07.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T12:35:07.584 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T12:35:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:35:07.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:07.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T12:35:07.783 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:07.830 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T12:35:07.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T12:35:07.831 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:07.879 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T12:35:07.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:07.879 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T12:35:08.082 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T12:35:08.082 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T12:35:08.082 INFO:teuthology.orchestra.run.smithi123.stdout:invalid new device string format 2024-04-25T12:35:08.847 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T12:35:08.847 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T12:35:08.847 INFO:teuthology.orchestra.run.smithi123.stdout:invalid new device string format 2024-04-25T12:35:08.847 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-04-25T12:35:08.847 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T12:35:08.889 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T12:35:08.890 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T12:35:08.890 INFO:teuthology.orchestra.run.smithi123.stdout:invalid new device string format 2024-04-25T12:35:08.890 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-04-25T12:35:08.890 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T12:35:08.890 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:6789,172.21.15.153:6789,172.21.15.153:6790" 2024-04-25T12:35:08.890 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.123:6789,172.21.15.153:6789,172.21.15.153:6790:/ 2024-04-25T12:35:08.890 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=2239057a-dc25-4ee3-8341-8c80e403f296" will pass to kernel 2024-04-25T12:35:08.893 INFO:teuthology.orchestra.run.smithi123.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T12:35:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: You just mounted a file system that supports labels which does not 2024-04-25T12:35:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T12:35:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T12:35:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T12:35:08.898 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T12:35:08.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:35:08.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:08.978 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T12:35:09.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T12:35:09.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:09.090 INFO:teuthology.orchestra.run.smithi123.stdout:client4742 2024-04-25T12:35:09.090 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:35:09.090 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:35:09.140 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T12:35:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T12:35:09.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:09.312 INFO:teuthology.orchestra.run.smithi123.stdout:client4742 2024-04-25T12:35:09.312 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T12:35:09.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T12:35:09.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:09.427 INFO:teuthology.orchestra.run.smithi123.stdout:client4742 2024-04-25T12:35:09.427 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2239057a-dc25-4ee3-8341-8c80e403f296.client4742/status') 2024-04-25T12:35:09.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:35:09.560 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.4742 (3)172.21.15.123:0/2511611602 2024-04-25T12:35:09.560 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2024-04-25T12:35:09.560 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2024-04-25T12:35:09.560 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2024-04-25T12:35:09.560 INFO:teuthology.orchestra.run.smithi123.stderr:71 bytes copied, 9.7875e-05 s, 725 kB/s 2024-04-25T12:35:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:35:09.573 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-25T12:35:09.623 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-25T12:35:09.623 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T12:35:09.623 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-25T12:35:09.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-25T12:35:09.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.910+0000 7f84eee00640 1 Processor -- start 2024-04-25T12:35:09.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.910+0000 7f84eee00640 1 -- start start 2024-04-25T12:35:09.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.912+0000 7f84eee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84e8152c70 0x7f84e8074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.912+0000 7f84eee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e80c8580 con 0x7f84e80c8170 2024-04-25T12:35:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.912+0000 7f84eee00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e8158bd0 con 0x7f84e80c2790 2024-04-25T12:35:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.912+0000 7f84eee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f84e806d390 con 0x7f84e8152c70 2024-04-25T12:35:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.913+0000 7f84e7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84e8152c70 0x7f84e8074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.913+0000 7f84e7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84e8152c70 0x7f84e8074820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52532/0 (socket says 172.21.15.123:52532) 2024-04-25T12:35:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.913+0000 7f84e7e00640 1 -- 172.21.15.123:0/3465129655 learned_addr learned my addr 172.21.15.123:0/3465129655 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.913+0000 7f84e6a00640 1 -- 172.21.15.123:0/3465129655 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586648521 0 0) 0x7f84e8158bd0 con 0x7f84e80c2790 2024-04-25T12:35:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.914+0000 7f84e6a00640 1 -- 172.21.15.123:0/3465129655 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84cc003520 con 0x7f84e80c2790 2024-04-25T12:35:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.914+0000 7f84e7e00640 1 -- 172.21.15.123:0/3465129655 >> v1:172.21.15.153:6789/0 conn(0x7f84e80c2790 legacy=0x7f84e806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.914+0000 7f84e7e00640 1 -- 172.21.15.123:0/3465129655 >> v1:172.21.15.123:6789/0 conn(0x7f84e80c8170 legacy=0x7f84e80c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.914+0000 7f84e7e00640 1 -- 172.21.15.123:0/3465129655 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e8158eb0 con 0x7f84e8152c70 2024-04-25T12:35:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.915+0000 7f84e7e00640 1 --2- 172.21.15.123:0/3465129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e8152c70 0x7f84e8074820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d150 tx=0x7f84e002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f04ee44fd5a906a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.915+0000 7f84e6a00640 1 -- 172.21.15.123:0/3465129655 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e000b2e0 con 0x7f84e8152c70 2024-04-25T12:35:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.915+0000 7f84e6a00640 1 -- 172.21.15.123:0/3465129655 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0037070 con 0x7f84e8152c70 2024-04-25T12:35:09.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.915+0000 7f84e6a00640 1 -- 172.21.15.123:0/3465129655 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e000b480 con 0x7f84e8152c70 2024-04-25T12:35:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.917+0000 7f84eee00640 1 -- 172.21.15.123:0/3465129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e8152c70 msgr2=0x7f84e8074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.917+0000 7f84eee00640 1 --2- 172.21.15.123:0/3465129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e8152c70 0x7f84e8074820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d150 tx=0x7f84e002c940 comp rx=0 tx=0).stop 2024-04-25T12:35:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.917+0000 7f84eee00640 1 -- 172.21.15.123:0/3465129655 shutdown_connections 2024-04-25T12:35:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.917+0000 7f84eee00640 1 --2- 172.21.15.123:0/3465129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e8152c70 0x7f84e8074820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.917+0000 7f84eee00640 1 -- 172.21.15.123:0/3465129655 >> 172.21.15.123:0/3465129655 conn(0x7f84e806f440 msgr2=0x7f84e806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.917+0000 7f84eee00640 1 -- 172.21.15.123:0/3465129655 shutdown_connections 2024-04-25T12:35:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.917+0000 7f84eee00640 1 -- 172.21.15.123:0/3465129655 wait complete. 2024-04-25T12:35:09.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.919+0000 7f84eee00640 1 Processor -- start 2024-04-25T12:35:09.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.919+0000 7f84eee00640 1 -- start start 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84eee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e80c2790 0x7f84e80ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84eee00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84e80c8170 0x7f84e81f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84eee00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84e8152c70 0x7f84e814a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84eee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8159750 con 0x7f84e80c2790 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84eee00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8159470 con 0x7f84e80c8170 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84eee00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80c46f0 con 0x7f84e8152c70 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e80c2790 0x7f84e80ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84e7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84e80c8170 0x7f84e81f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84e7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84e80c8170 0x7f84e81f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:50876/0 (socket says 172.21.15.123:50876) 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84ece00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84e8152c70 0x7f84e814a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.921+0000 7f84e7400640 1 -- 172.21.15.123:0/744265145 learned_addr learned my addr 172.21.15.123:0/744265145 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:09.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.922+0000 7f84e7400640 1 -- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84e8152c70 msgr2=0x7f84e814a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:09.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.922+0000 7f84e7400640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84e8152c70 0x7f84e814a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:09.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.922+0000 7f84e7400640 1 -- 172.21.15.123:0/744265145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e80c2790 msgr2=0x7f84e80ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:09.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.922+0000 7f84e7400640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e80c2790 0x7f84e80ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:09.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.922+0000 7f84e7400640 1 -- 172.21.15.123:0/744265145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e81f5630 con 0x7f84e80c8170 2024-04-25T12:35:09.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.922+0000 7f84e7400640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84e80c8170 0x7f84e81f50f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84d400c490 tx=0x7f84d400c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.923+0000 7f84dfe00640 1 -- 172.21.15.123:0/744265145 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d40052d0 con 0x7f84e80c8170 2024-04-25T12:35:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.923+0000 7f84dfe00640 1 -- 172.21.15.123:0/744265145 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d4005490 con 0x7f84e80c8170 2024-04-25T12:35:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.923+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e81f4050 con 0x7f84e80c8170 2024-04-25T12:35:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.923+0000 7f84dfe00640 1 -- 172.21.15.123:0/744265145 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4012870 con 0x7f84e80c8170 2024-04-25T12:35:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.923+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81f4610 con 0x7f84e80c8170 2024-04-25T12:35:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.925+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e8068eb0 con 0x7f84e80c8170 2024-04-25T12:35:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.926+0000 7f84dfe00640 1 -- 172.21.15.123:0/744265145 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f84d4012a10 con 0x7f84e80c8170 2024-04-25T12:35:09.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.927+0000 7f84dfe00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f84c80744b0 0x7f84c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:09.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.927+0000 7f84dfe00640 1 -- 172.21.15.123:0/744265145 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f84d40950d0 con 0x7f84e80c8170 2024-04-25T12:35:09.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.927+0000 7f84e7e00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f84c80744b0 0x7f84c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:09.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.928+0000 7f84e7e00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f84c80744b0 0x7f84c80769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f84e0036000 tx=0x7f84e000b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:09.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:09.931+0000 7f84dfe00640 1 -- 172.21.15.123:0/744265145 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84e8068eb0 con 0x7f84e80c8170 2024-04-25T12:35:10.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.141+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) -- 0x7f84e8076700 con 0x7f84e80c8170 2024-04-25T12:35:10.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.149+0000 7f84dfe00640 1 -- 172.21.15.123:0/744265145 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v10)=0 v10) ==== 136+0+0 (secure 0 0 0) 0x7f84d405ee90 con 0x7f84e80c8170 2024-04-25T12:35:10.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f84c80744b0 msgr2=0x7f84c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f84c80744b0 0x7f84c80769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f84e0036000 tx=0x7f84e000b000 comp rx=0 tx=0).stop 2024-04-25T12:35:10.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84e80c8170 msgr2=0x7f84e81f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84e80c8170 0x7f84e81f50f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84d400c490 tx=0x7f84d400c950 comp rx=0 tx=0).stop 2024-04-25T12:35:10.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 shutdown_connections 2024-04-25T12:35:10.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f84c80744b0 0x7f84c80769c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84e8152c70 0x7f84e814a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84e80c8170 0x7f84e81f50f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 --2- 172.21.15.123:0/744265145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84e80c2790 0x7f84e80ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 >> 172.21.15.123:0/744265145 conn(0x7f84e806f440 msgr2=0x7f84e8073d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:10.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 shutdown_connections 2024-04-25T12:35:10.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.152+0000 7f84eee00640 1 -- 172.21.15.123:0/744265145 wait complete. 2024-04-25T12:35:10.179 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-25T12:35:10.179 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T12:35:10.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:35:10.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.447+0000 7fc48ac00640 1 Processor -- start 2024-04-25T12:35:10.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.448+0000 7fc48ac00640 1 -- start start 2024-04-25T12:35:10.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc48ac00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc48406dbf0 0x7fc4841565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:10.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc48ac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc48415cf10 0x7fc48406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:10.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc48ac00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4840c8760 con 0x7fc4840c6d50 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc48ac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc4840c82c0 con 0x7fc48415cf10 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc48ac00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc48406ca80 con 0x7fc48406dbf0 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc488e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc48415cf10 0x7fc48406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc489800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc48406dbf0 0x7fc4841565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc488e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc48415cf10 0x7fc48406cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52554/0 (socket says 172.21.15.123:52554) 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.450+0000 7fc488e00640 1 -- 172.21.15.123:0/1346112288 learned_addr learned my addr 172.21.15.123:0/1346112288 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.451+0000 7fc488e00640 1 -- 172.21.15.123:0/1346112288 >> v2:172.21.15.153:3300/0 conn(0x7fc48406dbf0 msgr2=0x7fc4841565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.451+0000 7fc488e00640 1 --2- 172.21.15.123:0/1346112288 >> v2:172.21.15.153:3300/0 conn(0x7fc48406dbf0 0x7fc4841565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.451+0000 7fc488e00640 1 -- 172.21.15.123:0/1346112288 >> v1:172.21.15.153:6789/0 conn(0x7fc4840c6d50 legacy=0x7fc48415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.451+0000 7fc483e00640 1 -- 172.21.15.123:0/1346112288 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3900271216 0 0) 0x7fc4840c8760 con 0x7fc4840c6d50 2024-04-25T12:35:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.451+0000 7fc488e00640 1 -- 172.21.15.123:0/1346112288 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4840c5630 con 0x7fc48415cf10 2024-04-25T12:35:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.452+0000 7fc488e00640 1 --2- 172.21.15.123:0/1346112288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48415cf10 0x7fc48406cf10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc484071fa0 tx=0x7fc47002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a564a3744ec3736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.452+0000 7fc483e00640 1 -- 172.21.15.123:0/1346112288 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470037070 con 0x7fc48415cf10 2024-04-25T12:35:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.452+0000 7fc483e00640 1 -- 172.21.15.123:0/1346112288 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc470032440 con 0x7fc48415cf10 2024-04-25T12:35:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.452+0000 7fc483e00640 1 -- 172.21.15.123:0/1346112288 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470032810 con 0x7fc48415cf10 2024-04-25T12:35:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 -- 172.21.15.123:0/1346112288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48415cf10 msgr2=0x7fc48406cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 --2- 172.21.15.123:0/1346112288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48415cf10 0x7fc48406cf10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc484071fa0 tx=0x7fc47002a270 comp rx=0 tx=0).stop 2024-04-25T12:35:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 -- 172.21.15.123:0/1346112288 shutdown_connections 2024-04-25T12:35:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 --2- 172.21.15.123:0/1346112288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48415cf10 0x7fc48406cf10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 --2- 172.21.15.123:0/1346112288 >> v2:172.21.15.153:3300/0 conn(0x7fc48406dbf0 0x7fc4841565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 -- 172.21.15.123:0/1346112288 >> 172.21.15.123:0/1346112288 conn(0x7fc4840ce170 msgr2=0x7fc4840ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 -- 172.21.15.123:0/1346112288 shutdown_connections 2024-04-25T12:35:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.453+0000 7fc48ac00640 1 -- 172.21.15.123:0/1346112288 wait complete. 2024-04-25T12:35:10.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.455+0000 7fc48ac00640 1 Processor -- start 2024-04-25T12:35:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.456+0000 7fc48ac00640 1 -- start start 2024-04-25T12:35:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc48ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48406dbf0 0x7fc48414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc48ac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4840c6d50 0x7fc4841f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc48ac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc48415cf10 0x7fc48414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc48ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc48406d450 con 0x7fc48406dbf0 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc48ac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc48406c710 con 0x7fc48415cf10 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc48ac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc48415d320 con 0x7fc4840c6d50 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc489800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48406dbf0 0x7fc48414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc489800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48406dbf0 0x7fc48414e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52558/0 (socket says 172.21.15.123:52558) 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc489800640 1 -- 172.21.15.123:0/559415465 learned_addr learned my addr 172.21.15.123:0/559415465 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc48a200640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc48415cf10 0x7fc48414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.457+0000 7fc488e00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4840c6d50 0x7fc4841f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:10.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.458+0000 7fc489800640 1 -- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4840c6d50 msgr2=0x7fc4841f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.458+0000 7fc489800640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4840c6d50 0x7fc4841f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.458+0000 7fc489800640 1 -- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc48415cf10 msgr2=0x7fc48414fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.458+0000 7fc489800640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc48415cf10 0x7fc48414fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.458+0000 7fc489800640 1 -- 172.21.15.123:0/559415465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4841502b0 con 0x7fc48406dbf0 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.458+0000 7fc48a200640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc48415cf10 0x7fc48414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.458+0000 7fc489800640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48406dbf0 0x7fc48414e6c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e8a0 tx=0x7fc47400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.459+0000 7fc481600640 1 -- 172.21.15.123:0/559415465 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474017070 con 0x7fc48406dbf0 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.459+0000 7fc481600640 1 -- 172.21.15.123:0/559415465 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47400c940 con 0x7fc48406dbf0 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.459+0000 7fc481600640 1 -- 172.21.15.123:0/559415465 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47400cd30 con 0x7fc48406dbf0 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.459+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4841f3050 con 0x7fc48406dbf0 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.459+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4841f2d70 con 0x7fc48406dbf0 2024-04-25T12:35:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.460+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc484076da0 con 0x7fc48406dbf0 2024-04-25T12:35:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.461+0000 7fc481600640 1 -- 172.21.15.123:0/559415465 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc47401a070 con 0x7fc48406dbf0 2024-04-25T12:35:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.462+0000 7fc481600640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4640744b0 0x7fc4640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.462+0000 7fc481600640 1 -- 172.21.15.123:0/559415465 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fc4740954b0 con 0x7fc48406dbf0 2024-04-25T12:35:10.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.463+0000 7fc488e00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4640744b0 0x7fc4640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:10.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.463+0000 7fc488e00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4640744b0 0x7fc4640769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc470034000 tx=0x7fc47002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:10.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.467+0000 7fc481600640 1 -- 172.21.15.123:0/559415465 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc47405f1f0 con 0x7fc48406dbf0 2024-04-25T12:35:10.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.661+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc484074a10 con 0x7fc4640744b0 2024-04-25T12:35:10.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.671+0000 7fc481600640 1 -- 172.21.15.123:0/559415465 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332654 (secure 0 0 0) 0x7fc470093380 con 0x7fc4640744b0 2024-04-25T12:35:10.675 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:10.676 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4640744b0 msgr2=0x7fc4640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4640744b0 0x7fc4640769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc470034000 tx=0x7fc47002e000 comp rx=0 tx=0).stop 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48406dbf0 msgr2=0x7fc48414e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48406dbf0 0x7fc48414e6c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e8a0 tx=0x7fc47400ed60 comp rx=0 tx=0).stop 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 shutdown_connections 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4640744b0 0x7fc4640769c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc48415cf10 0x7fc48414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4840c6d50 0x7fc4841f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 --2- 172.21.15.123:0/559415465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc48406dbf0 0x7fc48414e6c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 >> 172.21.15.123:0/559415465 conn(0x7fc4840ce170 msgr2=0x7fc4840c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 shutdown_connections 2024-04-25T12:35:10.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:10.677+0000 7fc48ac00640 1 -- 172.21.15.123:0/559415465 wait complete. 2024-04-25T12:35:10.706 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-04-25T12:35:08.806509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":230,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218680,"kb_used_data":2496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756040,"statfs":{"total":773094113280,"available":772870184960,"internally_reserved":0,"allocated":2555904,"data_stored":2128336,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.575936"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694930+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:35:00.694930+0000","last_peered":"2024-04-25T12:35:00.694930+0000","last_clean":"2024-04-25T12:35:00.694930+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:35:00.694930+0000","last_undegraded":"2024-04-25T12:35:00.694930+0000","last_fullsized":"2024-04-25T12:35:00.694930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488862+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:35:00.488862+0000","last_peered":"2024-04-25T12:35:00.488862+0000","last_clean":"2024-04-25T12:35:00.488862+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:35:00.488862+0000","last_undegraded":"2024-04-25T12:35:00.488862+0000","last_fullsized":"2024-04-25T12:35:00.488862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238136+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:35:00.238136+0000","last_peered":"2024-04-25T12:35:00.238136+0000","last_clean":"2024-04-25T12:35:00.238136+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:35:00.238136+0000","last_undegraded":"2024-04-25T12:35:00.238136+0000","last_fullsized":"2024-04-25T12:35:00.238136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488973+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:35:00.488973+0000","last_peered":"2024-04-25T12:35:00.488973+0000","last_clean":"2024-04-25T12:35:00.488973+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:35:00.488973+0000","last_undegraded":"2024-04-25T12:35:00.488973+0000","last_fullsized":"2024-04-25T12:35:00.488973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488971+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:35:00.488971+0000","last_peered":"2024-04-25T12:35:00.488971+0000","last_clean":"2024-04-25T12:35:00.488971+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:35:00.488971+0000","last_undegraded":"2024-04-25T12:35:00.488971+0000","last_fullsized":"2024-04-25T12:35:00.488971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697778+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:35:00.697778+0000","last_peered":"2024-04-25T12:35:00.697778+0000","last_clean":"2024-04-25T12:35:00.697778+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:35:00.697778+0000","last_undegraded":"2024-04-25T12:35:00.697778+0000","last_fullsized":"2024-04-25T12:35:00.697778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244935+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:35:00.244935+0000","last_peered":"2024-04-25T12:35:00.244935+0000","last_clean":"2024-04-25T12:35:00.244935+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:35:00.244935+0000","last_undegraded":"2024-04-25T12:35:00.244935+0000","last_fullsized":"2024-04-25T12:35:00.244935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697494+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:35:00.697494+0000","last_peered":"2024-04-25T12:35:00.697494+0000","last_clean":"2024-04-25T12:35:00.697494+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:35:00.697494+0000","last_undegraded":"2024-04-25T12:35:00.697494+0000","last_fullsized":"2024-04-25T12:35:00.697494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699179+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:35:00.699179+0000","last_peered":"2024-04-25T12:35:00.699179+0000","last_clean":"2024-04-25T12:35:00.699179+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:35:00.699179+0000","last_undegraded":"2024-04-25T12:35:00.699179+0000","last_fullsized":"2024-04-25T12:35:00.699179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245331+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:35:00.245331+0000","last_peered":"2024-04-25T12:35:00.245331+0000","last_clean":"2024-04-25T12:35:00.245331+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:35:00.245331+0000","last_undegraded":"2024-04-25T12:35:00.245331+0000","last_fullsized":"2024-04-25T12:35:00.245331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237412+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:35:00.237412+0000","last_peered":"2024-04-25T12:35:00.237412+0000","last_clean":"2024-04-25T12:35:00.237412+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:35:00.237412+0000","last_undegraded":"2024-04-25T12:35:00.237412+0000","last_fullsized":"2024-04-25T12:35:00.237412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697778+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:35:00.697778+0000","last_peered":"2024-04-25T12:35:00.697778+0000","last_clean":"2024-04-25T12:35:00.697778+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:35:00.697778+0000","last_undegraded":"2024-04-25T12:35:00.697778+0000","last_fullsized":"2024-04-25T12:35:00.697778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236722+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:35:00.236722+0000","last_peered":"2024-04-25T12:35:00.236722+0000","last_clean":"2024-04-25T12:35:00.236722+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:35:00.236722+0000","last_undegraded":"2024-04-25T12:35:00.236722+0000","last_fullsized":"2024-04-25T12:35:00.236722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237528+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:35:00.237528+0000","last_peered":"2024-04-25T12:35:00.237528+0000","last_clean":"2024-04-25T12:35:00.237528+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:35:00.237528+0000","last_undegraded":"2024-04-25T12:35:00.237528+0000","last_fullsized":"2024-04-25T12:35:00.237528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.706022+0000","last_change":"2024-04-25T12:35:00.703950+0000","last_active":"2024-04-25T12:35:00.706022+0000","last_peered":"2024-04-25T12:35:00.706022+0000","last_clean":"2024-04-25T12:35:00.706022+0000","last_became_active":"2024-04-25T12:35:00.702548+0000","last_became_peered":"2024-04-25T12:35:00.702548+0000","last_unstale":"2024-04-25T12:35:00.706022+0000","last_undegraded":"2024-04-25T12:35:00.706022+0000","last_fullsized":"2024-04-25T12:35:00.706022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238506+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:35:00.238506+0000","last_peered":"2024-04-25T12:35:00.238506+0000","last_clean":"2024-04-25T12:35:00.238506+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:35:00.238506+0000","last_undegraded":"2024-04-25T12:35:00.238506+0000","last_fullsized":"2024-04-25T12:35:00.238506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235664+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:35:00.235664+0000","last_peered":"2024-04-25T12:35:00.235664+0000","last_clean":"2024-04-25T12:35:00.235664+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:35:00.235664+0000","last_undegraded":"2024-04-25T12:35:00.235664+0000","last_fullsized":"2024-04-25T12:35:00.235664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244076+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:35:00.244076+0000","last_peered":"2024-04-25T12:35:00.244076+0000","last_clean":"2024-04-25T12:35:00.244076+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:35:00.244076+0000","last_undegraded":"2024-04-25T12:35:00.244076+0000","last_fullsized":"2024-04-25T12:35:00.244076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694168+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:35:00.694168+0000","last_peered":"2024-04-25T12:35:00.694168+0000","last_clean":"2024-04-25T12:35:00.694168+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:35:00.694168+0000","last_undegraded":"2024-04-25T12:35:00.694168+0000","last_fullsized":"2024-04-25T12:35:00.694168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694486+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:35:00.694486+0000","last_peered":"2024-04-25T12:35:00.694486+0000","last_clean":"2024-04-25T12:35:00.694486+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:35:00.694486+0000","last_undegraded":"2024-04-25T12:35:00.694486+0000","last_fullsized":"2024-04-25T12:35:00.694486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238547+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:35:00.238547+0000","last_peered":"2024-04-25T12:35:00.238547+0000","last_clean":"2024-04-25T12:35:00.238547+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:35:00.238547+0000","last_undegraded":"2024-04-25T12:35:00.238547+0000","last_fullsized":"2024-04-25T12:35:00.238547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236478+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:35:00.236478+0000","last_peered":"2024-04-25T12:35:00.236478+0000","last_clean":"2024-04-25T12:35:00.236478+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:35:00.236478+0000","last_undegraded":"2024-04-25T12:35:00.236478+0000","last_fullsized":"2024-04-25T12:35:00.236478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242241+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:35:00.242241+0000","last_peered":"2024-04-25T12:35:00.242241+0000","last_clean":"2024-04-25T12:35:00.242241+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:35:00.242241+0000","last_undegraded":"2024-04-25T12:35:00.242241+0000","last_fullsized":"2024-04-25T12:35:00.242241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700917+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:35:00.700917+0000","last_peered":"2024-04-25T12:35:00.700917+0000","last_clean":"2024-04-25T12:35:00.700917+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:35:00.700917+0000","last_undegraded":"2024-04-25T12:35:00.700917+0000","last_fullsized":"2024-04-25T12:35:00.700917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694705+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:35:00.694705+0000","last_peered":"2024-04-25T12:35:00.694705+0000","last_clean":"2024-04-25T12:35:00.694705+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:35:00.694705+0000","last_undegraded":"2024-04-25T12:35:00.694705+0000","last_fullsized":"2024-04-25T12:35:00.694705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238249+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:35:00.238249+0000","last_peered":"2024-04-25T12:35:00.238249+0000","last_clean":"2024-04-25T12:35:00.238249+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:35:00.238249+0000","last_undegraded":"2024-04-25T12:35:00.238249+0000","last_fullsized":"2024-04-25T12:35:00.238249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699021+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:35:00.699021+0000","last_peered":"2024-04-25T12:35:00.699021+0000","last_clean":"2024-04-25T12:35:00.699021+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:35:00.699021+0000","last_undegraded":"2024-04-25T12:35:00.699021+0000","last_fullsized":"2024-04-25T12:35:00.699021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694870+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:35:00.694870+0000","last_peered":"2024-04-25T12:35:00.694870+0000","last_clean":"2024-04-25T12:35:00.694870+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:35:00.694870+0000","last_undegraded":"2024-04-25T12:35:00.694870+0000","last_fullsized":"2024-04-25T12:35:00.694870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694849+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:35:00.694849+0000","last_peered":"2024-04-25T12:35:00.694849+0000","last_clean":"2024-04-25T12:35:00.694849+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:35:00.694849+0000","last_undegraded":"2024-04-25T12:35:00.694849+0000","last_fullsized":"2024-04-25T12:35:00.694849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694157+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:35:00.694157+0000","last_peered":"2024-04-25T12:35:00.694157+0000","last_clean":"2024-04-25T12:35:00.694157+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:35:00.694157+0000","last_undegraded":"2024-04-25T12:35:00.694157+0000","last_fullsized":"2024-04-25T12:35:00.694157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236113+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:35:00.236113+0000","last_peered":"2024-04-25T12:35:00.236113+0000","last_clean":"2024-04-25T12:35:00.236113+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:35:00.236113+0000","last_undegraded":"2024-04-25T12:35:00.236113+0000","last_fullsized":"2024-04-25T12:35:00.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237837+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:35:00.237837+0000","last_peered":"2024-04-25T12:35:00.237837+0000","last_clean":"2024-04-25T12:35:00.237837+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:35:00.237837+0000","last_undegraded":"2024-04-25T12:35:00.237837+0000","last_fullsized":"2024-04-25T12:35:00.237837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235604+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:35:00.235604+0000","last_peered":"2024-04-25T12:35:00.235604+0000","last_clean":"2024-04-25T12:35:00.235604+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:35:00.235604+0000","last_undegraded":"2024-04-25T12:35:00.235604+0000","last_fullsized":"2024-04-25T12:35:00.235604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699987+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:35:00.699987+0000","last_peered":"2024-04-25T12:35:00.699987+0000","last_clean":"2024-04-25T12:35:00.699987+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:35:00.699987+0000","last_undegraded":"2024-04-25T12:35:00.699987+0000","last_fullsized":"2024-04-25T12:35:00.699987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488578+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:35:00.488578+0000","last_peered":"2024-04-25T12:35:00.488578+0000","last_clean":"2024-04-25T12:35:00.488578+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:35:00.488578+0000","last_undegraded":"2024-04-25T12:35:00.488578+0000","last_fullsized":"2024-04-25T12:35:00.488578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235386+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:35:00.235386+0000","last_peered":"2024-04-25T12:35:00.235386+0000","last_clean":"2024-04-25T12:35:00.235386+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:35:00.235386+0000","last_undegraded":"2024-04-25T12:35:00.235386+0000","last_fullsized":"2024-04-25T12:35:00.235386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.243856+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:35:00.243856+0000","last_peered":"2024-04-25T12:35:00.243856+0000","last_clean":"2024-04-25T12:35:00.243856+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:35:00.243856+0000","last_undegraded":"2024-04-25T12:35:00.243856+0000","last_fullsized":"2024-04-25T12:35:00.243856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237018+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:35:00.237018+0000","last_peered":"2024-04-25T12:35:00.237018+0000","last_clean":"2024-04-25T12:35:00.237018+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:35:00.237018+0000","last_undegraded":"2024-04-25T12:35:00.237018+0000","last_fullsized":"2024-04-25T12:35:00.237018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235850+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:35:00.235850+0000","last_peered":"2024-04-25T12:35:00.235850+0000","last_clean":"2024-04-25T12:35:00.235850+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:35:00.235850+0000","last_undegraded":"2024-04-25T12:35:00.235850+0000","last_fullsized":"2024-04-25T12:35:00.235850+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488045+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:35:00.488045+0000","last_peered":"2024-04-25T12:35:00.488045+0000","last_clean":"2024-04-25T12:35:00.488045+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:35:00.488045+0000","last_undegraded":"2024-04-25T12:35:00.488045+0000","last_fullsized":"2024-04-25T12:35:00.488045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238322+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:35:00.238322+0000","last_peered":"2024-04-25T12:35:00.238322+0000","last_clean":"2024-04-25T12:35:00.238322+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:35:00.238322+0000","last_undegraded":"2024-04-25T12:35:00.238322+0000","last_fullsized":"2024-04-25T12:35:00.238322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236008+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:35:00.236008+0000","last_peered":"2024-04-25T12:35:00.236008+0000","last_clean":"2024-04-25T12:35:00.236008+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:35:00.236008+0000","last_undegraded":"2024-04-25T12:35:00.236008+0000","last_fullsized":"2024-04-25T12:35:00.236008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235351+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:35:00.235351+0000","last_peered":"2024-04-25T12:35:00.235351+0000","last_clean":"2024-04-25T12:35:00.235351+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:35:00.235351+0000","last_undegraded":"2024-04-25T12:35:00.235351+0000","last_fullsized":"2024-04-25T12:35:00.235351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236908+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:35:00.236908+0000","last_peered":"2024-04-25T12:35:00.236908+0000","last_clean":"2024-04-25T12:35:00.236908+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:35:00.236908+0000","last_undegraded":"2024-04-25T12:35:00.236908+0000","last_fullsized":"2024-04-25T12:35:00.236908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488596+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:35:00.488596+0000","last_peered":"2024-04-25T12:35:00.488596+0000","last_clean":"2024-04-25T12:35:00.488596+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:35:00.488596+0000","last_undegraded":"2024-04-25T12:35:00.488596+0000","last_fullsized":"2024-04-25T12:35:00.488596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694405+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:35:00.694405+0000","last_peered":"2024-04-25T12:35:00.694405+0000","last_clean":"2024-04-25T12:35:00.694405+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:35:00.694405+0000","last_undegraded":"2024-04-25T12:35:00.694405+0000","last_fullsized":"2024-04-25T12:35:00.694405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237670+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:35:00.237670+0000","last_peered":"2024-04-25T12:35:00.237670+0000","last_clean":"2024-04-25T12:35:00.237670+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:35:00.237670+0000","last_undegraded":"2024-04-25T12:35:00.237670+0000","last_fullsized":"2024-04-25T12:35:00.237670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697341+0000","last_change":"2024-04-25T12:34:08.560989+0000","last_active":"2024-04-25T12:35:00.697341+0000","last_peered":"2024-04-25T12:35:00.697341+0000","last_clean":"2024-04-25T12:35:00.697341+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:35:00.697341+0000","last_undegraded":"2024-04-25T12:35:00.697341+0000","last_fullsized":"2024-04-25T12:35:00.697341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:49.130809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237678+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:35:00.237678+0000","last_peered":"2024-04-25T12:35:00.237678+0000","last_clean":"2024-04-25T12:35:00.237678+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:35:00.237678+0000","last_undegraded":"2024-04-25T12:35:00.237678+0000","last_fullsized":"2024-04-25T12:35:00.237678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237549+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:35:00.237549+0000","last_peered":"2024-04-25T12:35:00.237549+0000","last_clean":"2024-04-25T12:35:00.237549+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:35:00.237549+0000","last_undegraded":"2024-04-25T12:35:00.237549+0000","last_fullsized":"2024-04-25T12:35:00.237549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236215+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:35:00.236215+0000","last_peered":"2024-04-25T12:35:00.236215+0000","last_clean":"2024-04-25T12:35:00.236215+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:35:00.236215+0000","last_undegraded":"2024-04-25T12:35:00.236215+0000","last_fullsized":"2024-04-25T12:35:00.236215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698145+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:35:00.698145+0000","last_peered":"2024-04-25T12:35:00.698145+0000","last_clean":"2024-04-25T12:35:00.698145+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:35:00.698145+0000","last_undegraded":"2024-04-25T12:35:00.698145+0000","last_fullsized":"2024-04-25T12:35:00.698145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245030+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:35:00.245030+0000","last_peered":"2024-04-25T12:35:00.245030+0000","last_clean":"2024-04-25T12:35:00.245030+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:35:00.245030+0000","last_undegraded":"2024-04-25T12:35:00.245030+0000","last_fullsized":"2024-04-25T12:35:00.245030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488705+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:35:00.488705+0000","last_peered":"2024-04-25T12:35:00.488705+0000","last_clean":"2024-04-25T12:35:00.488705+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:35:00.488705+0000","last_undegraded":"2024-04-25T12:35:00.488705+0000","last_fullsized":"2024-04-25T12:35:00.488705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488389+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:35:00.488389+0000","last_peered":"2024-04-25T12:35:00.488389+0000","last_clean":"2024-04-25T12:35:00.488389+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:35:00.488389+0000","last_undegraded":"2024-04-25T12:35:00.488389+0000","last_fullsized":"2024-04-25T12:35:00.488389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245496+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:35:00.245496+0000","last_peered":"2024-04-25T12:35:00.245496+0000","last_clean":"2024-04-25T12:35:00.245496+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:35:00.245496+0000","last_undegraded":"2024-04-25T12:35:00.245496+0000","last_fullsized":"2024-04-25T12:35:00.245496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694766+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:35:00.694766+0000","last_peered":"2024-04-25T12:35:00.694766+0000","last_clean":"2024-04-25T12:35:00.694766+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:35:00.694766+0000","last_undegraded":"2024-04-25T12:35:00.694766+0000","last_fullsized":"2024-04-25T12:35:00.694766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236118+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:35:00.236118+0000","last_peered":"2024-04-25T12:35:00.236118+0000","last_clean":"2024-04-25T12:35:00.236118+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:35:00.236118+0000","last_undegraded":"2024-04-25T12:35:00.236118+0000","last_fullsized":"2024-04-25T12:35:00.236118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.701276+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:35:00.701276+0000","last_peered":"2024-04-25T12:35:00.701276+0000","last_clean":"2024-04-25T12:35:00.701276+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:35:00.701276+0000","last_undegraded":"2024-04-25T12:35:00.701276+0000","last_fullsized":"2024-04-25T12:35:00.701276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245657+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:35:00.245657+0000","last_peered":"2024-04-25T12:35:00.245657+0000","last_clean":"2024-04-25T12:35:00.245657+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:35:00.245657+0000","last_undegraded":"2024-04-25T12:35:00.245657+0000","last_fullsized":"2024-04-25T12:35:00.245657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697896+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:35:00.697896+0000","last_peered":"2024-04-25T12:35:00.697896+0000","last_clean":"2024-04-25T12:35:00.697896+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:35:00.697896+0000","last_undegraded":"2024-04-25T12:35:00.697896+0000","last_fullsized":"2024-04-25T12:35:00.697896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236512+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:35:00.236512+0000","last_peered":"2024-04-25T12:35:00.236512+0000","last_clean":"2024-04-25T12:35:00.236512+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:35:00.236512+0000","last_undegraded":"2024-04-25T12:35:00.236512+0000","last_fullsized":"2024-04-25T12:35:00.236512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694315+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:35:00.694315+0000","last_peered":"2024-04-25T12:35:00.694315+0000","last_clean":"2024-04-25T12:35:00.694315+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:35:00.694315+0000","last_undegraded":"2024-04-25T12:35:00.694315+0000","last_fullsized":"2024-04-25T12:35:00.694315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488781+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:35:00.488781+0000","last_peered":"2024-04-25T12:35:00.488781+0000","last_clean":"2024-04-25T12:35:00.488781+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:35:00.488781+0000","last_undegraded":"2024-04-25T12:35:00.488781+0000","last_fullsized":"2024-04-25T12:35:00.488781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.696800+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:35:00.696800+0000","last_peered":"2024-04-25T12:35:00.696800+0000","last_clean":"2024-04-25T12:35:00.696800+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:35:00.696800+0000","last_undegraded":"2024-04-25T12:35:00.696800+0000","last_fullsized":"2024-04-25T12:35:00.696800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242577+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:35:00.242577+0000","last_peered":"2024-04-25T12:35:00.242577+0000","last_clean":"2024-04-25T12:35:00.242577+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:35:00.242577+0000","last_undegraded":"2024-04-25T12:35:00.242577+0000","last_fullsized":"2024-04-25T12:35:00.242577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488857+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:35:00.488857+0000","last_peered":"2024-04-25T12:35:00.488857+0000","last_clean":"2024-04-25T12:35:00.488857+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:35:00.488857+0000","last_undegraded":"2024-04-25T12:35:00.488857+0000","last_fullsized":"2024-04-25T12:35:00.488857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237299+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:35:00.237299+0000","last_peered":"2024-04-25T12:35:00.237299+0000","last_clean":"2024-04-25T12:35:00.237299+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:35:00.237299+0000","last_undegraded":"2024-04-25T12:35:00.237299+0000","last_fullsized":"2024-04-25T12:35:00.237299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236954+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:35:00.236954+0000","last_peered":"2024-04-25T12:35:00.236954+0000","last_clean":"2024-04-25T12:35:00.236954+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:35:00.236954+0000","last_undegraded":"2024-04-25T12:35:00.236954+0000","last_fullsized":"2024-04-25T12:35:00.236954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235770+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:35:00.235770+0000","last_peered":"2024-04-25T12:35:00.235770+0000","last_clean":"2024-04-25T12:35:00.235770+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:35:00.235770+0000","last_undegraded":"2024-04-25T12:35:00.235770+0000","last_fullsized":"2024-04-25T12:35:00.235770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235564+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:35:00.235564+0000","last_peered":"2024-04-25T12:35:00.235564+0000","last_clean":"2024-04-25T12:35:00.235564+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:35:00.235564+0000","last_undegraded":"2024-04-25T12:35:00.235564+0000","last_fullsized":"2024-04-25T12:35:00.235564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238662+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:35:00.238662+0000","last_peered":"2024-04-25T12:35:00.238662+0000","last_clean":"2024-04-25T12:35:00.238662+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:35:00.238662+0000","last_undegraded":"2024-04-25T12:35:00.238662+0000","last_fullsized":"2024-04-25T12:35:00.238662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237434+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:35:00.237434+0000","last_peered":"2024-04-25T12:35:00.237434+0000","last_clean":"2024-04-25T12:35:00.237434+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:35:00.237434+0000","last_undegraded":"2024-04-25T12:35:00.237434+0000","last_fullsized":"2024-04-25T12:35:00.237434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.240432+0000","last_change":"2024-04-25T12:35:00.239744+0000","last_active":"2024-04-25T12:35:00.240432+0000","last_peered":"2024-04-25T12:35:00.240432+0000","last_clean":"2024-04-25T12:35:00.240432+0000","last_became_active":"2024-04-25T12:35:00.239414+0000","last_became_peered":"2024-04-25T12:35:00.239414+0000","last_unstale":"2024-04-25T12:35:00.240432+0000","last_undegraded":"2024-04-25T12:35:00.240432+0000","last_fullsized":"2024-04-25T12:35:00.240432+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:17.015606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'12","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698405+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:35:00.698405+0000","last_peered":"2024-04-25T12:35:00.698405+0000","last_clean":"2024-04-25T12:35:00.698405+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:35:00.698405+0000","last_undegraded":"2024-04-25T12:35:00.698405+0000","last_fullsized":"2024-04-25T12:35:00.698405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694248+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:35:00.694248+0000","last_peered":"2024-04-25T12:35:00.694248+0000","last_clean":"2024-04-25T12:35:00.694248+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:35:00.694248+0000","last_undegraded":"2024-04-25T12:35:00.694248+0000","last_fullsized":"2024-04-25T12:35:00.694248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236992+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:35:00.236992+0000","last_peered":"2024-04-25T12:35:00.236992+0000","last_clean":"2024-04-25T12:35:00.236992+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:35:00.236992+0000","last_undegraded":"2024-04-25T12:35:00.236992+0000","last_fullsized":"2024-04-25T12:35:00.236992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238965+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:35:00.238965+0000","last_peered":"2024-04-25T12:35:00.238965+0000","last_clean":"2024-04-25T12:35:00.238965+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:35:00.238965+0000","last_undegraded":"2024-04-25T12:35:00.238965+0000","last_fullsized":"2024-04-25T12:35:00.238965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699488+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:35:00.699488+0000","last_peered":"2024-04-25T12:35:00.699488+0000","last_clean":"2024-04-25T12:35:00.699488+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:35:00.699488+0000","last_undegraded":"2024-04-25T12:35:00.699488+0000","last_fullsized":"2024-04-25T12:35:00.699488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.489002+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:35:00.489002+0000","last_peered":"2024-04-25T12:35:00.489002+0000","last_clean":"2024-04-25T12:35:00.489002+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:35:00.489002+0000","last_undegraded":"2024-04-25T12:35:00.489002+0000","last_fullsized":"2024-04-25T12:35:00.489002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245001+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:35:00.245001+0000","last_peered":"2024-04-25T12:35:00.245001+0000","last_clean":"2024-04-25T12:35:00.245001+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:35:00.245001+0000","last_undegraded":"2024-04-25T12:35:00.245001+0000","last_fullsized":"2024-04-25T12:35:00.245001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238024+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:35:00.238024+0000","last_peered":"2024-04-25T12:35:00.238024+0000","last_clean":"2024-04-25T12:35:00.238024+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:35:00.238024+0000","last_undegraded":"2024-04-25T12:35:00.238024+0000","last_fullsized":"2024-04-25T12:35:00.238024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.701303+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:35:00.701303+0000","last_peered":"2024-04-25T12:35:00.701303+0000","last_clean":"2024-04-25T12:35:00.701303+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:35:00.701303+0000","last_undegraded":"2024-04-25T12:35:00.701303+0000","last_fullsized":"2024-04-25T12:35:00.701303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700940+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:35:00.700940+0000","last_peered":"2024-04-25T12:35:00.700940+0000","last_clean":"2024-04-25T12:35:00.700940+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:35:00.700940+0000","last_undegraded":"2024-04-25T12:35:00.700940+0000","last_fullsized":"2024-04-25T12:35:00.700940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694320+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:35:00.694320+0000","last_peered":"2024-04-25T12:35:00.694320+0000","last_clean":"2024-04-25T12:35:00.694320+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:35:00.694320+0000","last_undegraded":"2024-04-25T12:35:00.694320+0000","last_fullsized":"2024-04-25T12:35:00.694320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694611+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:35:00.694611+0000","last_peered":"2024-04-25T12:35:00.694611+0000","last_clean":"2024-04-25T12:35:00.694611+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:35:00.694611+0000","last_undegraded":"2024-04-25T12:35:00.694611+0000","last_fullsized":"2024-04-25T12:35:00.694611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694483+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:35:00.694483+0000","last_peered":"2024-04-25T12:35:00.694483+0000","last_clean":"2024-04-25T12:35:00.694483+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:35:00.694483+0000","last_undegraded":"2024-04-25T12:35:00.694483+0000","last_fullsized":"2024-04-25T12:35:00.694483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236757+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:35:00.236757+0000","last_peered":"2024-04-25T12:35:00.236757+0000","last_clean":"2024-04-25T12:35:00.236757+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:35:00.236757+0000","last_undegraded":"2024-04-25T12:35:00.236757+0000","last_fullsized":"2024-04-25T12:35:00.236757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237774+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:35:00.237774+0000","last_peered":"2024-04-25T12:35:00.237774+0000","last_clean":"2024-04-25T12:35:00.237774+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:35:00.237774+0000","last_undegraded":"2024-04-25T12:35:00.237774+0000","last_fullsized":"2024-04-25T12:35:00.237774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488252+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:35:00.488252+0000","last_peered":"2024-04-25T12:35:00.488252+0000","last_clean":"2024-04-25T12:35:00.488252+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:35:00.488252+0000","last_undegraded":"2024-04-25T12:35:00.488252+0000","last_fullsized":"2024-04-25T12:35:00.488252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235386+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:35:00.235386+0000","last_peered":"2024-04-25T12:35:00.235386+0000","last_clean":"2024-04-25T12:35:00.235386+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:35:00.235386+0000","last_undegraded":"2024-04-25T12:35:00.235386+0000","last_fullsized":"2024-04-25T12:35:00.235386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236858+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:35:00.236858+0000","last_peered":"2024-04-25T12:35:00.236858+0000","last_clean":"2024-04-25T12:35:00.236858+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:35:00.236858+0000","last_undegraded":"2024-04-25T12:35:00.236858+0000","last_fullsized":"2024-04-25T12:35:00.236858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694382+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:35:00.694382+0000","last_peered":"2024-04-25T12:35:00.694382+0000","last_clean":"2024-04-25T12:35:00.694382+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:35:00.694382+0000","last_undegraded":"2024-04-25T12:35:00.694382+0000","last_fullsized":"2024-04-25T12:35:00.694382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694541+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:35:00.694541+0000","last_peered":"2024-04-25T12:35:00.694541+0000","last_clean":"2024-04-25T12:35:00.694541+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:35:00.694541+0000","last_undegraded":"2024-04-25T12:35:00.694541+0000","last_fullsized":"2024-04-25T12:35:00.694541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488638+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:35:00.488638+0000","last_peered":"2024-04-25T12:35:00.488638+0000","last_clean":"2024-04-25T12:35:00.488638+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:35:00.488638+0000","last_undegraded":"2024-04-25T12:35:00.488638+0000","last_fullsized":"2024-04-25T12:35:00.488638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699483+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:35:00.699483+0000","last_peered":"2024-04-25T12:35:00.699483+0000","last_clean":"2024-04-25T12:35:00.699483+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:35:00.699483+0000","last_undegraded":"2024-04-25T12:35:00.699483+0000","last_fullsized":"2024-04-25T12:35:00.699483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238812+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:35:00.238812+0000","last_peered":"2024-04-25T12:35:00.238812+0000","last_clean":"2024-04-25T12:35:00.238812+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:35:00.238812+0000","last_undegraded":"2024-04-25T12:35:00.238812+0000","last_fullsized":"2024-04-25T12:35:00.238812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699291+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:35:00.699291+0000","last_peered":"2024-04-25T12:35:00.699291+0000","last_clean":"2024-04-25T12:35:00.699291+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:35:00.699291+0000","last_undegraded":"2024-04-25T12:35:00.699291+0000","last_fullsized":"2024-04-25T12:35:00.699291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238474+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:35:00.238474+0000","last_peered":"2024-04-25T12:35:00.238474+0000","last_clean":"2024-04-25T12:35:00.238474+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:35:00.238474+0000","last_undegraded":"2024-04-25T12:35:00.238474+0000","last_fullsized":"2024-04-25T12:35:00.238474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236497+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:35:00.236497+0000","last_peered":"2024-04-25T12:35:00.236497+0000","last_clean":"2024-04-25T12:35:00.236497+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:35:00.236497+0000","last_undegraded":"2024-04-25T12:35:00.236497+0000","last_fullsized":"2024-04-25T12:35:00.236497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488552+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:35:00.488552+0000","last_peered":"2024-04-25T12:35:00.488552+0000","last_clean":"2024-04-25T12:35:00.488552+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:35:00.488552+0000","last_undegraded":"2024-04-25T12:35:00.488552+0000","last_fullsized":"2024-04-25T12:35:00.488552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235986+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:35:00.235986+0000","last_peered":"2024-04-25T12:35:00.235986+0000","last_clean":"2024-04-25T12:35:00.235986+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:35:00.235986+0000","last_undegraded":"2024-04-25T12:35:00.235986+0000","last_fullsized":"2024-04-25T12:35:00.235986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236085+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:35:00.236085+0000","last_peered":"2024-04-25T12:35:00.236085+0000","last_clean":"2024-04-25T12:35:00.236085+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:35:00.236085+0000","last_undegraded":"2024-04-25T12:35:00.236085+0000","last_fullsized":"2024-04-25T12:35:00.236085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700980+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:35:00.700980+0000","last_peered":"2024-04-25T12:35:00.700980+0000","last_clean":"2024-04-25T12:35:00.700980+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:35:00.700980+0000","last_undegraded":"2024-04-25T12:35:00.700980+0000","last_fullsized":"2024-04-25T12:35:00.700980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488827+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:35:00.488827+0000","last_peered":"2024-04-25T12:35:00.488827+0000","last_clean":"2024-04-25T12:35:00.488827+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:35:00.488827+0000","last_undegraded":"2024-04-25T12:35:00.488827+0000","last_fullsized":"2024-04-25T12:35:00.488827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236512+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:35:00.236512+0000","last_peered":"2024-04-25T12:35:00.236512+0000","last_clean":"2024-04-25T12:35:00.236512+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:35:00.236512+0000","last_undegraded":"2024-04-25T12:35:00.236512+0000","last_fullsized":"2024-04-25T12:35:00.236512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488137+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:35:00.488137+0000","last_peered":"2024-04-25T12:35:00.488137+0000","last_clean":"2024-04-25T12:35:00.488137+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:35:00.488137+0000","last_undegraded":"2024-04-25T12:35:00.488137+0000","last_fullsized":"2024-04-25T12:35:00.488137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699318+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:35:00.699318+0000","last_peered":"2024-04-25T12:35:00.699318+0000","last_clean":"2024-04-25T12:35:00.699318+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:35:00.699318+0000","last_undegraded":"2024-04-25T12:35:00.699318+0000","last_fullsized":"2024-04-25T12:35:00.699318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694330+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:35:00.694330+0000","last_peered":"2024-04-25T12:35:00.694330+0000","last_clean":"2024-04-25T12:35:00.694330+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:35:00.694330+0000","last_undegraded":"2024-04-25T12:35:00.694330+0000","last_fullsized":"2024-04-25T12:35:00.694330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244838+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:35:00.244838+0000","last_peered":"2024-04-25T12:35:00.244838+0000","last_clean":"2024-04-25T12:35:00.244838+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:35:00.244838+0000","last_undegraded":"2024-04-25T12:35:00.244838+0000","last_fullsized":"2024-04-25T12:35:00.244838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237777+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:35:00.237777+0000","last_peered":"2024-04-25T12:35:00.237777+0000","last_clean":"2024-04-25T12:35:00.237777+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:35:00.237777+0000","last_undegraded":"2024-04-25T12:35:00.237777+0000","last_fullsized":"2024-04-25T12:35:00.237777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238887+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:35:00.238887+0000","last_peered":"2024-04-25T12:35:00.238887+0000","last_clean":"2024-04-25T12:35:00.238887+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:35:00.238887+0000","last_undegraded":"2024-04-25T12:35:00.238887+0000","last_fullsized":"2024-04-25T12:35:00.238887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238159+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:35:00.238159+0000","last_peered":"2024-04-25T12:35:00.238159+0000","last_clean":"2024-04-25T12:35:00.238159+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:35:00.238159+0000","last_undegraded":"2024-04-25T12:35:00.238159+0000","last_fullsized":"2024-04-25T12:35:00.238159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236425+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:35:00.236425+0000","last_peered":"2024-04-25T12:35:00.236425+0000","last_clean":"2024-04-25T12:35:00.236425+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:35:00.236425+0000","last_undegraded":"2024-04-25T12:35:00.236425+0000","last_fullsized":"2024-04-25T12:35:00.236425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488070+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:35:00.488070+0000","last_peered":"2024-04-25T12:35:00.488070+0000","last_clean":"2024-04-25T12:35:00.488070+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:35:00.488070+0000","last_undegraded":"2024-04-25T12:35:00.488070+0000","last_fullsized":"2024-04-25T12:35:00.488070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235433+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:35:00.235433+0000","last_peered":"2024-04-25T12:35:00.235433+0000","last_clean":"2024-04-25T12:35:00.235433+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:35:00.235433+0000","last_undegraded":"2024-04-25T12:35:00.235433+0000","last_fullsized":"2024-04-25T12:35:00.235433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235696+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:35:00.235696+0000","last_peered":"2024-04-25T12:35:00.235696+0000","last_clean":"2024-04-25T12:35:00.235696+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:35:00.235696+0000","last_undegraded":"2024-04-25T12:35:00.235696+0000","last_fullsized":"2024-04-25T12:35:00.235696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244713+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:35:00.244713+0000","last_peered":"2024-04-25T12:35:00.244713+0000","last_clean":"2024-04-25T12:35:00.244713+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:35:00.244713+0000","last_undegraded":"2024-04-25T12:35:00.244713+0000","last_fullsized":"2024-04-25T12:35:00.244713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244441+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:35:00.244441+0000","last_peered":"2024-04-25T12:35:00.244441+0000","last_clean":"2024-04-25T12:35:00.244441+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:35:00.244441+0000","last_undegraded":"2024-04-25T12:35:00.244441+0000","last_fullsized":"2024-04-25T12:35:00.244441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.243194+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:35:00.243194+0000","last_peered":"2024-04-25T12:35:00.243194+0000","last_clean":"2024-04-25T12:35:00.243194+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:35:00.243194+0000","last_undegraded":"2024-04-25T12:35:00.243194+0000","last_fullsized":"2024-04-25T12:35:00.243194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700840+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:35:00.700840+0000","last_peered":"2024-04-25T12:35:00.700840+0000","last_clean":"2024-04-25T12:35:00.700840+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:35:00.700840+0000","last_undegraded":"2024-04-25T12:35:00.700840+0000","last_fullsized":"2024-04-25T12:35:00.700840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488044+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:35:00.488044+0000","last_peered":"2024-04-25T12:35:00.488044+0000","last_clean":"2024-04-25T12:35:00.488044+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:35:00.488044+0000","last_undegraded":"2024-04-25T12:35:00.488044+0000","last_fullsized":"2024-04-25T12:35:00.488044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242559+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:35:00.242559+0000","last_peered":"2024-04-25T12:35:00.242559+0000","last_clean":"2024-04-25T12:35:00.242559+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:35:00.242559+0000","last_undegraded":"2024-04-25T12:35:00.242559+0000","last_fullsized":"2024-04-25T12:35:00.242559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698632+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:35:00.698632+0000","last_peered":"2024-04-25T12:35:00.698632+0000","last_clean":"2024-04-25T12:35:00.698632+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:35:00.698632+0000","last_undegraded":"2024-04-25T12:35:00.698632+0000","last_fullsized":"2024-04-25T12:35:00.698632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244141+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:35:00.244141+0000","last_peered":"2024-04-25T12:35:00.244141+0000","last_clean":"2024-04-25T12:35:00.244141+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:35:00.244141+0000","last_undegraded":"2024-04-25T12:35:00.244141+0000","last_fullsized":"2024-04-25T12:35:00.244141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698101+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:35:00.698101+0000","last_peered":"2024-04-25T12:35:00.698101+0000","last_clean":"2024-04-25T12:35:00.698101+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:35:00.698101+0000","last_undegraded":"2024-04-25T12:35:00.698101+0000","last_fullsized":"2024-04-25T12:35:00.698101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488547+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:35:00.488547+0000","last_peered":"2024-04-25T12:35:00.488547+0000","last_clean":"2024-04-25T12:35:00.488547+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:35:00.488547+0000","last_undegraded":"2024-04-25T12:35:00.488547+0000","last_fullsized":"2024-04-25T12:35:00.488547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488685+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:35:00.488685+0000","last_peered":"2024-04-25T12:35:00.488685+0000","last_clean":"2024-04-25T12:35:00.488685+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:35:00.488685+0000","last_undegraded":"2024-04-25T12:35:00.488685+0000","last_fullsized":"2024-04-25T12:35:00.488685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245426+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:35:00.245426+0000","last_peered":"2024-04-25T12:35:00.245426+0000","last_clean":"2024-04-25T12:35:00.245426+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:35:00.245426+0000","last_undegraded":"2024-04-25T12:35:00.245426+0000","last_fullsized":"2024-04-25T12:35:00.245426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":14904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":117903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":307200,"data_stored":117495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":707759,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":121997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":120463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":707115,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:35:10.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:35:11.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.024+0000 7fedee400640 1 Processor -- start 2024-04-25T12:35:11.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.024+0000 7fedee400640 1 -- start start 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.026+0000 7fedee400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fede80c9080 0x7fede81546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.026+0000 7fedee400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede806c570 0x7fede806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.026+0000 7fedee400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede815a330 con 0x7fede80c82b0 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.026+0000 7fedee400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fede8073b10 con 0x7fede80c9080 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.026+0000 7fedee400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fede8073df0 con 0x7fede806c570 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7fede7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fede80c82b0 0x7fede815e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40290/0 (socket says 172.21.15.123:40290) 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7fede7e00640 1 -- 172.21.15.123:0/3586633958 learned_addr learned my addr 172.21.15.123:0/3586633958 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7fededa00640 1 --2- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede806c570 0x7fede806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7feded000640 1 --2- 172.21.15.123:0/3586633958 >> v2:172.21.15.153:3300/0 conn(0x7fede80c9080 0x7fede81546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7fede7400640 1 -- 172.21.15.123:0/3586633958 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570762682 0 0) 0x7fede815a330 con 0x7fede80c82b0 2024-04-25T12:35:11.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7fede7400640 1 -- 172.21.15.123:0/3586633958 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0003520 con 0x7fede80c82b0 2024-04-25T12:35:11.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7feded000640 1 -- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede806c570 msgr2=0x7fede806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:11.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7feded000640 1 --2- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede806c570 0x7fede806c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7feded000640 1 -- 172.21.15.123:0/3586633958 >> v1:172.21.15.123:6789/0 conn(0x7fede80c82b0 legacy=0x7fede815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7feded000640 1 -- 172.21.15.123:0/3586633958 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede80740d0 con 0x7fede80c9080 2024-04-25T12:35:11.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.027+0000 7fededa00640 1 --2- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede806c570 0x7fede806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:11.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.029+0000 7feded000640 1 --2- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede81546d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feddc0097a0 tx=0x7feddc02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf36350203058e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:11.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.029+0000 7fede7400640 1 -- 172.21.15.123:0/3586633958 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc037070 con 0x7fede80c9080 2024-04-25T12:35:11.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.029+0000 7fede7400640 1 -- 172.21.15.123:0/3586633958 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc02ae40 con 0x7fede80c9080 2024-04-25T12:35:11.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.029+0000 7fede7400640 1 -- 172.21.15.123:0/3586633958 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc032640 con 0x7fede80c9080 2024-04-25T12:35:11.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 -- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 msgr2=0x7fede81546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 --2- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede81546d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feddc0097a0 tx=0x7feddc02a0f0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 -- 172.21.15.123:0/3586633958 shutdown_connections 2024-04-25T12:35:11.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 --2- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede806c570 0x7fede806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 --2- 172.21.15.123:0/3586633958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede81546d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 -- 172.21.15.123:0/3586633958 >> 172.21.15.123:0/3586633958 conn(0x7fede80cd600 msgr2=0x7fede80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:11.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 -- 172.21.15.123:0/3586633958 shutdown_connections 2024-04-25T12:35:11.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.030+0000 7fedee400640 1 -- 172.21.15.123:0/3586633958 wait complete. 2024-04-25T12:35:11.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.033+0000 7fedee400640 1 Processor -- start 2024-04-25T12:35:11.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.033+0000 7fedee400640 1 -- start start 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fedee400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede806c570 0x7fede815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fedee400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede80c82b0 0x7fede8153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fedee400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede8153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fedee400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fede815e760 con 0x7fede806c570 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fedee400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fede815db30 con 0x7fede80c9080 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fedee400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fede8073760 con 0x7fede80c82b0 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7feded000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede806c570 0x7fede815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7feded000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede806c570 0x7fede815d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52564/0 (socket says 172.21.15.123:52564) 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7feded000640 1 -- 172.21.15.123:0/2975112454 learned_addr learned my addr 172.21.15.123:0/2975112454 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fede7e00640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede80c82b0 0x7fede8153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.035+0000 7fededa00640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede8153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7feded000640 1 -- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede80c82b0 msgr2=0x7fede8153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7feded000640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede80c82b0 0x7fede8153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7feded000640 1 -- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 msgr2=0x7fede8153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7feded000640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede8153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7feded000640 1 -- 172.21.15.123:0/2975112454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede80c2a10 con 0x7fede806c570 2024-04-25T12:35:11.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7fede7e00640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede80c82b0 0x7fede8153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:11.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7fededa00640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede8153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:11.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7feded000640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede806c570 0x7fede815d480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feddc02a130 tx=0x7feddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:11.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7fede4c00640 1 -- 172.21.15.123:0/2975112454 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc041070 con 0x7fede806c570 2024-04-25T12:35:11.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7fede4c00640 1 -- 172.21.15.123:0/2975112454 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc03c460 con 0x7fede806c570 2024-04-25T12:35:11.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7fede4c00640 1 -- 172.21.15.123:0/2975112454 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc037040 con 0x7fede806c570 2024-04-25T12:35:11.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.036+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede815c200 con 0x7fede806c570 2024-04-25T12:35:11.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.037+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81f7360 con 0x7fede806c570 2024-04-25T12:35:11.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.038+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede80c7cf0 con 0x7fede806c570 2024-04-25T12:35:11.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.039+0000 7fede4c00640 1 -- 172.21.15.123:0/2975112454 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feddc044070 con 0x7fede806c570 2024-04-25T12:35:11.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.040+0000 7fede4c00640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fedcc0744b0 0x7fedcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.040+0000 7fede4c00640 1 -- 172.21.15.123:0/2975112454 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7feddc0b5730 con 0x7fede806c570 2024-04-25T12:35:11.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.040+0000 7fede7e00640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fedcc0744b0 0x7fedcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.041+0000 7fede7e00640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fedcc0744b0 0x7fedcc0769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fede8154480 tx=0x7fedd8000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:11.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.047+0000 7fede4c00640 1 -- 172.21.15.123:0/2975112454 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fede80c7cf0 con 0x7fede806c570 2024-04-25T12:35:11.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.243+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fede80c7ab0 con 0x7fedcc0744b0 2024-04-25T12:35:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.254+0000 7fede4c00640 1 -- 172.21.15.123:0/2975112454 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332655 (secure 0 0 0) 0x7fede80c7ab0 con 0x7fedcc0744b0 2024-04-25T12:35:11.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:11.260 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:35:11.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.259+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fedcc0744b0 msgr2=0x7fedcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.259+0000 7fedee400640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fedcc0744b0 0x7fedcc0769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fede8154480 tx=0x7fedd8000bc0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede806c570 msgr2=0x7fede815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede806c570 0x7fede815d480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feddc02a130 tx=0x7feddc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 shutdown_connections 2024-04-25T12:35:11.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fedcc0744b0 0x7fedcc0769c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede80c9080 0x7fede8153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fede80c82b0 0x7fede8153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 --2- 172.21.15.123:0/2975112454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede806c570 0x7fede815d480 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 >> 172.21.15.123:0/2975112454 conn(0x7fede80cd600 msgr2=0x7fede815a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:11.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 shutdown_connections 2024-04-25T12:35:11.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.260+0000 7fedee400640 1 -- 172.21.15.123:0/2975112454 wait complete. 2024-04-25T12:35:11.285 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-04-25T12:35:10.807313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":230,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218680,"kb_used_data":2496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756040,"statfs":{"total":773094113280,"available":772870184960,"internally_reserved":0,"allocated":2555904,"data_stored":2128336,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.573736"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694930+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:35:00.694930+0000","last_peered":"2024-04-25T12:35:00.694930+0000","last_clean":"2024-04-25T12:35:00.694930+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:35:00.694930+0000","last_undegraded":"2024-04-25T12:35:00.694930+0000","last_fullsized":"2024-04-25T12:35:00.694930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488862+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:35:00.488862+0000","last_peered":"2024-04-25T12:35:00.488862+0000","last_clean":"2024-04-25T12:35:00.488862+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:35:00.488862+0000","last_undegraded":"2024-04-25T12:35:00.488862+0000","last_fullsized":"2024-04-25T12:35:00.488862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238136+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:35:00.238136+0000","last_peered":"2024-04-25T12:35:00.238136+0000","last_clean":"2024-04-25T12:35:00.238136+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:35:00.238136+0000","last_undegraded":"2024-04-25T12:35:00.238136+0000","last_fullsized":"2024-04-25T12:35:00.238136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488973+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:35:00.488973+0000","last_peered":"2024-04-25T12:35:00.488973+0000","last_clean":"2024-04-25T12:35:00.488973+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:35:00.488973+0000","last_undegraded":"2024-04-25T12:35:00.488973+0000","last_fullsized":"2024-04-25T12:35:00.488973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488971+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:35:00.488971+0000","last_peered":"2024-04-25T12:35:00.488971+0000","last_clean":"2024-04-25T12:35:00.488971+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:35:00.488971+0000","last_undegraded":"2024-04-25T12:35:00.488971+0000","last_fullsized":"2024-04-25T12:35:00.488971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697778+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:35:00.697778+0000","last_peered":"2024-04-25T12:35:00.697778+0000","last_clean":"2024-04-25T12:35:00.697778+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:35:00.697778+0000","last_undegraded":"2024-04-25T12:35:00.697778+0000","last_fullsized":"2024-04-25T12:35:00.697778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244935+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:35:00.244935+0000","last_peered":"2024-04-25T12:35:00.244935+0000","last_clean":"2024-04-25T12:35:00.244935+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:35:00.244935+0000","last_undegraded":"2024-04-25T12:35:00.244935+0000","last_fullsized":"2024-04-25T12:35:00.244935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697494+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:35:00.697494+0000","last_peered":"2024-04-25T12:35:00.697494+0000","last_clean":"2024-04-25T12:35:00.697494+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:35:00.697494+0000","last_undegraded":"2024-04-25T12:35:00.697494+0000","last_fullsized":"2024-04-25T12:35:00.697494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699179+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:35:00.699179+0000","last_peered":"2024-04-25T12:35:00.699179+0000","last_clean":"2024-04-25T12:35:00.699179+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:35:00.699179+0000","last_undegraded":"2024-04-25T12:35:00.699179+0000","last_fullsized":"2024-04-25T12:35:00.699179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245331+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:35:00.245331+0000","last_peered":"2024-04-25T12:35:00.245331+0000","last_clean":"2024-04-25T12:35:00.245331+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:35:00.245331+0000","last_undegraded":"2024-04-25T12:35:00.245331+0000","last_fullsized":"2024-04-25T12:35:00.245331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237412+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:35:00.237412+0000","last_peered":"2024-04-25T12:35:00.237412+0000","last_clean":"2024-04-25T12:35:00.237412+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:35:00.237412+0000","last_undegraded":"2024-04-25T12:35:00.237412+0000","last_fullsized":"2024-04-25T12:35:00.237412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697778+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:35:00.697778+0000","last_peered":"2024-04-25T12:35:00.697778+0000","last_clean":"2024-04-25T12:35:00.697778+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:35:00.697778+0000","last_undegraded":"2024-04-25T12:35:00.697778+0000","last_fullsized":"2024-04-25T12:35:00.697778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236722+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:35:00.236722+0000","last_peered":"2024-04-25T12:35:00.236722+0000","last_clean":"2024-04-25T12:35:00.236722+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:35:00.236722+0000","last_undegraded":"2024-04-25T12:35:00.236722+0000","last_fullsized":"2024-04-25T12:35:00.236722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237528+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:35:00.237528+0000","last_peered":"2024-04-25T12:35:00.237528+0000","last_clean":"2024-04-25T12:35:00.237528+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:35:00.237528+0000","last_undegraded":"2024-04-25T12:35:00.237528+0000","last_fullsized":"2024-04-25T12:35:00.237528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.706022+0000","last_change":"2024-04-25T12:35:00.703950+0000","last_active":"2024-04-25T12:35:00.706022+0000","last_peered":"2024-04-25T12:35:00.706022+0000","last_clean":"2024-04-25T12:35:00.706022+0000","last_became_active":"2024-04-25T12:35:00.702548+0000","last_became_peered":"2024-04-25T12:35:00.702548+0000","last_unstale":"2024-04-25T12:35:00.706022+0000","last_undegraded":"2024-04-25T12:35:00.706022+0000","last_fullsized":"2024-04-25T12:35:00.706022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238506+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:35:00.238506+0000","last_peered":"2024-04-25T12:35:00.238506+0000","last_clean":"2024-04-25T12:35:00.238506+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:35:00.238506+0000","last_undegraded":"2024-04-25T12:35:00.238506+0000","last_fullsized":"2024-04-25T12:35:00.238506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235664+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:35:00.235664+0000","last_peered":"2024-04-25T12:35:00.235664+0000","last_clean":"2024-04-25T12:35:00.235664+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:35:00.235664+0000","last_undegraded":"2024-04-25T12:35:00.235664+0000","last_fullsized":"2024-04-25T12:35:00.235664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244076+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:35:00.244076+0000","last_peered":"2024-04-25T12:35:00.244076+0000","last_clean":"2024-04-25T12:35:00.244076+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:35:00.244076+0000","last_undegraded":"2024-04-25T12:35:00.244076+0000","last_fullsized":"2024-04-25T12:35:00.244076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694168+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:35:00.694168+0000","last_peered":"2024-04-25T12:35:00.694168+0000","last_clean":"2024-04-25T12:35:00.694168+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:35:00.694168+0000","last_undegraded":"2024-04-25T12:35:00.694168+0000","last_fullsized":"2024-04-25T12:35:00.694168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694486+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:35:00.694486+0000","last_peered":"2024-04-25T12:35:00.694486+0000","last_clean":"2024-04-25T12:35:00.694486+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:35:00.694486+0000","last_undegraded":"2024-04-25T12:35:00.694486+0000","last_fullsized":"2024-04-25T12:35:00.694486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238547+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:35:00.238547+0000","last_peered":"2024-04-25T12:35:00.238547+0000","last_clean":"2024-04-25T12:35:00.238547+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:35:00.238547+0000","last_undegraded":"2024-04-25T12:35:00.238547+0000","last_fullsized":"2024-04-25T12:35:00.238547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236478+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:35:00.236478+0000","last_peered":"2024-04-25T12:35:00.236478+0000","last_clean":"2024-04-25T12:35:00.236478+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:35:00.236478+0000","last_undegraded":"2024-04-25T12:35:00.236478+0000","last_fullsized":"2024-04-25T12:35:00.236478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242241+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:35:00.242241+0000","last_peered":"2024-04-25T12:35:00.242241+0000","last_clean":"2024-04-25T12:35:00.242241+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:35:00.242241+0000","last_undegraded":"2024-04-25T12:35:00.242241+0000","last_fullsized":"2024-04-25T12:35:00.242241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700917+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:35:00.700917+0000","last_peered":"2024-04-25T12:35:00.700917+0000","last_clean":"2024-04-25T12:35:00.700917+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:35:00.700917+0000","last_undegraded":"2024-04-25T12:35:00.700917+0000","last_fullsized":"2024-04-25T12:35:00.700917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694705+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:35:00.694705+0000","last_peered":"2024-04-25T12:35:00.694705+0000","last_clean":"2024-04-25T12:35:00.694705+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:35:00.694705+0000","last_undegraded":"2024-04-25T12:35:00.694705+0000","last_fullsized":"2024-04-25T12:35:00.694705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238249+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:35:00.238249+0000","last_peered":"2024-04-25T12:35:00.238249+0000","last_clean":"2024-04-25T12:35:00.238249+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:35:00.238249+0000","last_undegraded":"2024-04-25T12:35:00.238249+0000","last_fullsized":"2024-04-25T12:35:00.238249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699021+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:35:00.699021+0000","last_peered":"2024-04-25T12:35:00.699021+0000","last_clean":"2024-04-25T12:35:00.699021+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:35:00.699021+0000","last_undegraded":"2024-04-25T12:35:00.699021+0000","last_fullsized":"2024-04-25T12:35:00.699021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694870+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:35:00.694870+0000","last_peered":"2024-04-25T12:35:00.694870+0000","last_clean":"2024-04-25T12:35:00.694870+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:35:00.694870+0000","last_undegraded":"2024-04-25T12:35:00.694870+0000","last_fullsized":"2024-04-25T12:35:00.694870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694849+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:35:00.694849+0000","last_peered":"2024-04-25T12:35:00.694849+0000","last_clean":"2024-04-25T12:35:00.694849+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:35:00.694849+0000","last_undegraded":"2024-04-25T12:35:00.694849+0000","last_fullsized":"2024-04-25T12:35:00.694849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694157+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:35:00.694157+0000","last_peered":"2024-04-25T12:35:00.694157+0000","last_clean":"2024-04-25T12:35:00.694157+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:35:00.694157+0000","last_undegraded":"2024-04-25T12:35:00.694157+0000","last_fullsized":"2024-04-25T12:35:00.694157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236113+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:35:00.236113+0000","last_peered":"2024-04-25T12:35:00.236113+0000","last_clean":"2024-04-25T12:35:00.236113+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:35:00.236113+0000","last_undegraded":"2024-04-25T12:35:00.236113+0000","last_fullsized":"2024-04-25T12:35:00.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237837+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:35:00.237837+0000","last_peered":"2024-04-25T12:35:00.237837+0000","last_clean":"2024-04-25T12:35:00.237837+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:35:00.237837+0000","last_undegraded":"2024-04-25T12:35:00.237837+0000","last_fullsized":"2024-04-25T12:35:00.237837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235604+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:35:00.235604+0000","last_peered":"2024-04-25T12:35:00.235604+0000","last_clean":"2024-04-25T12:35:00.235604+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:35:00.235604+0000","last_undegraded":"2024-04-25T12:35:00.235604+0000","last_fullsized":"2024-04-25T12:35:00.235604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699987+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:35:00.699987+0000","last_peered":"2024-04-25T12:35:00.699987+0000","last_clean":"2024-04-25T12:35:00.699987+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:35:00.699987+0000","last_undegraded":"2024-04-25T12:35:00.699987+0000","last_fullsized":"2024-04-25T12:35:00.699987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488578+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:35:00.488578+0000","last_peered":"2024-04-25T12:35:00.488578+0000","last_clean":"2024-04-25T12:35:00.488578+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:35:00.488578+0000","last_undegraded":"2024-04-25T12:35:00.488578+0000","last_fullsized":"2024-04-25T12:35:00.488578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235386+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:35:00.235386+0000","last_peered":"2024-04-25T12:35:00.235386+0000","last_clean":"2024-04-25T12:35:00.235386+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:35:00.235386+0000","last_undegraded":"2024-04-25T12:35:00.235386+0000","last_fullsized":"2024-04-25T12:35:00.235386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.243856+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:35:00.243856+0000","last_peered":"2024-04-25T12:35:00.243856+0000","last_clean":"2024-04-25T12:35:00.243856+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:35:00.243856+0000","last_undegraded":"2024-04-25T12:35:00.243856+0000","last_fullsized":"2024-04-25T12:35:00.243856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237018+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:35:00.237018+0000","last_peered":"2024-04-25T12:35:00.237018+0000","last_clean":"2024-04-25T12:35:00.237018+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:35:00.237018+0000","last_undegraded":"2024-04-25T12:35:00.237018+0000","last_fullsized":"2024-04-25T12:35:00.237018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235850+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:35:00.235850+0000","last_peered":"2024-04-25T12:35:00.235850+0000","last_clean":"2024-04-25T12:35:00.235850+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:35:00.235850+0000","last_undegraded":"2024-04-25T12:35:00.235850+0000","last_fullsized":"2024-04-25T12:35:00.235850+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488045+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:35:00.488045+0000","last_peered":"2024-04-25T12:35:00.488045+0000","last_clean":"2024-04-25T12:35:00.488045+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:35:00.488045+0000","last_undegraded":"2024-04-25T12:35:00.488045+0000","last_fullsized":"2024-04-25T12:35:00.488045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238322+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:35:00.238322+0000","last_peered":"2024-04-25T12:35:00.238322+0000","last_clean":"2024-04-25T12:35:00.238322+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:35:00.238322+0000","last_undegraded":"2024-04-25T12:35:00.238322+0000","last_fullsized":"2024-04-25T12:35:00.238322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236008+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:35:00.236008+0000","last_peered":"2024-04-25T12:35:00.236008+0000","last_clean":"2024-04-25T12:35:00.236008+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:35:00.236008+0000","last_undegraded":"2024-04-25T12:35:00.236008+0000","last_fullsized":"2024-04-25T12:35:00.236008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235351+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:35:00.235351+0000","last_peered":"2024-04-25T12:35:00.235351+0000","last_clean":"2024-04-25T12:35:00.235351+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:35:00.235351+0000","last_undegraded":"2024-04-25T12:35:00.235351+0000","last_fullsized":"2024-04-25T12:35:00.235351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236908+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:35:00.236908+0000","last_peered":"2024-04-25T12:35:00.236908+0000","last_clean":"2024-04-25T12:35:00.236908+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:35:00.236908+0000","last_undegraded":"2024-04-25T12:35:00.236908+0000","last_fullsized":"2024-04-25T12:35:00.236908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488596+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:35:00.488596+0000","last_peered":"2024-04-25T12:35:00.488596+0000","last_clean":"2024-04-25T12:35:00.488596+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:35:00.488596+0000","last_undegraded":"2024-04-25T12:35:00.488596+0000","last_fullsized":"2024-04-25T12:35:00.488596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694405+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:35:00.694405+0000","last_peered":"2024-04-25T12:35:00.694405+0000","last_clean":"2024-04-25T12:35:00.694405+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:35:00.694405+0000","last_undegraded":"2024-04-25T12:35:00.694405+0000","last_fullsized":"2024-04-25T12:35:00.694405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237670+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:35:00.237670+0000","last_peered":"2024-04-25T12:35:00.237670+0000","last_clean":"2024-04-25T12:35:00.237670+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:35:00.237670+0000","last_undegraded":"2024-04-25T12:35:00.237670+0000","last_fullsized":"2024-04-25T12:35:00.237670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697341+0000","last_change":"2024-04-25T12:34:08.560989+0000","last_active":"2024-04-25T12:35:00.697341+0000","last_peered":"2024-04-25T12:35:00.697341+0000","last_clean":"2024-04-25T12:35:00.697341+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:35:00.697341+0000","last_undegraded":"2024-04-25T12:35:00.697341+0000","last_fullsized":"2024-04-25T12:35:00.697341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:49.130809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237678+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:35:00.237678+0000","last_peered":"2024-04-25T12:35:00.237678+0000","last_clean":"2024-04-25T12:35:00.237678+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:35:00.237678+0000","last_undegraded":"2024-04-25T12:35:00.237678+0000","last_fullsized":"2024-04-25T12:35:00.237678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237549+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:35:00.237549+0000","last_peered":"2024-04-25T12:35:00.237549+0000","last_clean":"2024-04-25T12:35:00.237549+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:35:00.237549+0000","last_undegraded":"2024-04-25T12:35:00.237549+0000","last_fullsized":"2024-04-25T12:35:00.237549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236215+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:35:00.236215+0000","last_peered":"2024-04-25T12:35:00.236215+0000","last_clean":"2024-04-25T12:35:00.236215+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:35:00.236215+0000","last_undegraded":"2024-04-25T12:35:00.236215+0000","last_fullsized":"2024-04-25T12:35:00.236215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698145+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:35:00.698145+0000","last_peered":"2024-04-25T12:35:00.698145+0000","last_clean":"2024-04-25T12:35:00.698145+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:35:00.698145+0000","last_undegraded":"2024-04-25T12:35:00.698145+0000","last_fullsized":"2024-04-25T12:35:00.698145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245030+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:35:00.245030+0000","last_peered":"2024-04-25T12:35:00.245030+0000","last_clean":"2024-04-25T12:35:00.245030+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:35:00.245030+0000","last_undegraded":"2024-04-25T12:35:00.245030+0000","last_fullsized":"2024-04-25T12:35:00.245030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488705+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:35:00.488705+0000","last_peered":"2024-04-25T12:35:00.488705+0000","last_clean":"2024-04-25T12:35:00.488705+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:35:00.488705+0000","last_undegraded":"2024-04-25T12:35:00.488705+0000","last_fullsized":"2024-04-25T12:35:00.488705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488389+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:35:00.488389+0000","last_peered":"2024-04-25T12:35:00.488389+0000","last_clean":"2024-04-25T12:35:00.488389+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:35:00.488389+0000","last_undegraded":"2024-04-25T12:35:00.488389+0000","last_fullsized":"2024-04-25T12:35:00.488389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245496+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:35:00.245496+0000","last_peered":"2024-04-25T12:35:00.245496+0000","last_clean":"2024-04-25T12:35:00.245496+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:35:00.245496+0000","last_undegraded":"2024-04-25T12:35:00.245496+0000","last_fullsized":"2024-04-25T12:35:00.245496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694766+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:35:00.694766+0000","last_peered":"2024-04-25T12:35:00.694766+0000","last_clean":"2024-04-25T12:35:00.694766+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:35:00.694766+0000","last_undegraded":"2024-04-25T12:35:00.694766+0000","last_fullsized":"2024-04-25T12:35:00.694766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236118+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:35:00.236118+0000","last_peered":"2024-04-25T12:35:00.236118+0000","last_clean":"2024-04-25T12:35:00.236118+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:35:00.236118+0000","last_undegraded":"2024-04-25T12:35:00.236118+0000","last_fullsized":"2024-04-25T12:35:00.236118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.701276+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:35:00.701276+0000","last_peered":"2024-04-25T12:35:00.701276+0000","last_clean":"2024-04-25T12:35:00.701276+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:35:00.701276+0000","last_undegraded":"2024-04-25T12:35:00.701276+0000","last_fullsized":"2024-04-25T12:35:00.701276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245657+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:35:00.245657+0000","last_peered":"2024-04-25T12:35:00.245657+0000","last_clean":"2024-04-25T12:35:00.245657+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:35:00.245657+0000","last_undegraded":"2024-04-25T12:35:00.245657+0000","last_fullsized":"2024-04-25T12:35:00.245657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697896+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:35:00.697896+0000","last_peered":"2024-04-25T12:35:00.697896+0000","last_clean":"2024-04-25T12:35:00.697896+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:35:00.697896+0000","last_undegraded":"2024-04-25T12:35:00.697896+0000","last_fullsized":"2024-04-25T12:35:00.697896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236512+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:35:00.236512+0000","last_peered":"2024-04-25T12:35:00.236512+0000","last_clean":"2024-04-25T12:35:00.236512+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:35:00.236512+0000","last_undegraded":"2024-04-25T12:35:00.236512+0000","last_fullsized":"2024-04-25T12:35:00.236512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694315+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:35:00.694315+0000","last_peered":"2024-04-25T12:35:00.694315+0000","last_clean":"2024-04-25T12:35:00.694315+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:35:00.694315+0000","last_undegraded":"2024-04-25T12:35:00.694315+0000","last_fullsized":"2024-04-25T12:35:00.694315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488781+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:35:00.488781+0000","last_peered":"2024-04-25T12:35:00.488781+0000","last_clean":"2024-04-25T12:35:00.488781+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:35:00.488781+0000","last_undegraded":"2024-04-25T12:35:00.488781+0000","last_fullsized":"2024-04-25T12:35:00.488781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.696800+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:35:00.696800+0000","last_peered":"2024-04-25T12:35:00.696800+0000","last_clean":"2024-04-25T12:35:00.696800+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:35:00.696800+0000","last_undegraded":"2024-04-25T12:35:00.696800+0000","last_fullsized":"2024-04-25T12:35:00.696800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242577+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:35:00.242577+0000","last_peered":"2024-04-25T12:35:00.242577+0000","last_clean":"2024-04-25T12:35:00.242577+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:35:00.242577+0000","last_undegraded":"2024-04-25T12:35:00.242577+0000","last_fullsized":"2024-04-25T12:35:00.242577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488857+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:35:00.488857+0000","last_peered":"2024-04-25T12:35:00.488857+0000","last_clean":"2024-04-25T12:35:00.488857+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:35:00.488857+0000","last_undegraded":"2024-04-25T12:35:00.488857+0000","last_fullsized":"2024-04-25T12:35:00.488857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237299+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:35:00.237299+0000","last_peered":"2024-04-25T12:35:00.237299+0000","last_clean":"2024-04-25T12:35:00.237299+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:35:00.237299+0000","last_undegraded":"2024-04-25T12:35:00.237299+0000","last_fullsized":"2024-04-25T12:35:00.237299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236954+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:35:00.236954+0000","last_peered":"2024-04-25T12:35:00.236954+0000","last_clean":"2024-04-25T12:35:00.236954+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:35:00.236954+0000","last_undegraded":"2024-04-25T12:35:00.236954+0000","last_fullsized":"2024-04-25T12:35:00.236954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235770+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:35:00.235770+0000","last_peered":"2024-04-25T12:35:00.235770+0000","last_clean":"2024-04-25T12:35:00.235770+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:35:00.235770+0000","last_undegraded":"2024-04-25T12:35:00.235770+0000","last_fullsized":"2024-04-25T12:35:00.235770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235564+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:35:00.235564+0000","last_peered":"2024-04-25T12:35:00.235564+0000","last_clean":"2024-04-25T12:35:00.235564+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:35:00.235564+0000","last_undegraded":"2024-04-25T12:35:00.235564+0000","last_fullsized":"2024-04-25T12:35:00.235564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238662+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:35:00.238662+0000","last_peered":"2024-04-25T12:35:00.238662+0000","last_clean":"2024-04-25T12:35:00.238662+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:35:00.238662+0000","last_undegraded":"2024-04-25T12:35:00.238662+0000","last_fullsized":"2024-04-25T12:35:00.238662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237434+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:35:00.237434+0000","last_peered":"2024-04-25T12:35:00.237434+0000","last_clean":"2024-04-25T12:35:00.237434+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:35:00.237434+0000","last_undegraded":"2024-04-25T12:35:00.237434+0000","last_fullsized":"2024-04-25T12:35:00.237434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.240432+0000","last_change":"2024-04-25T12:35:00.239744+0000","last_active":"2024-04-25T12:35:00.240432+0000","last_peered":"2024-04-25T12:35:00.240432+0000","last_clean":"2024-04-25T12:35:00.240432+0000","last_became_active":"2024-04-25T12:35:00.239414+0000","last_became_peered":"2024-04-25T12:35:00.239414+0000","last_unstale":"2024-04-25T12:35:00.240432+0000","last_undegraded":"2024-04-25T12:35:00.240432+0000","last_fullsized":"2024-04-25T12:35:00.240432+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:17.015606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'12","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698405+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:35:00.698405+0000","last_peered":"2024-04-25T12:35:00.698405+0000","last_clean":"2024-04-25T12:35:00.698405+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:35:00.698405+0000","last_undegraded":"2024-04-25T12:35:00.698405+0000","last_fullsized":"2024-04-25T12:35:00.698405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694248+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:35:00.694248+0000","last_peered":"2024-04-25T12:35:00.694248+0000","last_clean":"2024-04-25T12:35:00.694248+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:35:00.694248+0000","last_undegraded":"2024-04-25T12:35:00.694248+0000","last_fullsized":"2024-04-25T12:35:00.694248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236992+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:35:00.236992+0000","last_peered":"2024-04-25T12:35:00.236992+0000","last_clean":"2024-04-25T12:35:00.236992+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:35:00.236992+0000","last_undegraded":"2024-04-25T12:35:00.236992+0000","last_fullsized":"2024-04-25T12:35:00.236992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238965+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:35:00.238965+0000","last_peered":"2024-04-25T12:35:00.238965+0000","last_clean":"2024-04-25T12:35:00.238965+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:35:00.238965+0000","last_undegraded":"2024-04-25T12:35:00.238965+0000","last_fullsized":"2024-04-25T12:35:00.238965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699488+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:35:00.699488+0000","last_peered":"2024-04-25T12:35:00.699488+0000","last_clean":"2024-04-25T12:35:00.699488+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:35:00.699488+0000","last_undegraded":"2024-04-25T12:35:00.699488+0000","last_fullsized":"2024-04-25T12:35:00.699488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.489002+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:35:00.489002+0000","last_peered":"2024-04-25T12:35:00.489002+0000","last_clean":"2024-04-25T12:35:00.489002+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:35:00.489002+0000","last_undegraded":"2024-04-25T12:35:00.489002+0000","last_fullsized":"2024-04-25T12:35:00.489002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245001+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:35:00.245001+0000","last_peered":"2024-04-25T12:35:00.245001+0000","last_clean":"2024-04-25T12:35:00.245001+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:35:00.245001+0000","last_undegraded":"2024-04-25T12:35:00.245001+0000","last_fullsized":"2024-04-25T12:35:00.245001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238024+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:35:00.238024+0000","last_peered":"2024-04-25T12:35:00.238024+0000","last_clean":"2024-04-25T12:35:00.238024+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:35:00.238024+0000","last_undegraded":"2024-04-25T12:35:00.238024+0000","last_fullsized":"2024-04-25T12:35:00.238024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.701303+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:35:00.701303+0000","last_peered":"2024-04-25T12:35:00.701303+0000","last_clean":"2024-04-25T12:35:00.701303+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:35:00.701303+0000","last_undegraded":"2024-04-25T12:35:00.701303+0000","last_fullsized":"2024-04-25T12:35:00.701303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700940+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:35:00.700940+0000","last_peered":"2024-04-25T12:35:00.700940+0000","last_clean":"2024-04-25T12:35:00.700940+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:35:00.700940+0000","last_undegraded":"2024-04-25T12:35:00.700940+0000","last_fullsized":"2024-04-25T12:35:00.700940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694320+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:35:00.694320+0000","last_peered":"2024-04-25T12:35:00.694320+0000","last_clean":"2024-04-25T12:35:00.694320+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:35:00.694320+0000","last_undegraded":"2024-04-25T12:35:00.694320+0000","last_fullsized":"2024-04-25T12:35:00.694320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694611+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:35:00.694611+0000","last_peered":"2024-04-25T12:35:00.694611+0000","last_clean":"2024-04-25T12:35:00.694611+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:35:00.694611+0000","last_undegraded":"2024-04-25T12:35:00.694611+0000","last_fullsized":"2024-04-25T12:35:00.694611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694483+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:35:00.694483+0000","last_peered":"2024-04-25T12:35:00.694483+0000","last_clean":"2024-04-25T12:35:00.694483+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:35:00.694483+0000","last_undegraded":"2024-04-25T12:35:00.694483+0000","last_fullsized":"2024-04-25T12:35:00.694483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236757+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:35:00.236757+0000","last_peered":"2024-04-25T12:35:00.236757+0000","last_clean":"2024-04-25T12:35:00.236757+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:35:00.236757+0000","last_undegraded":"2024-04-25T12:35:00.236757+0000","last_fullsized":"2024-04-25T12:35:00.236757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237774+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:35:00.237774+0000","last_peered":"2024-04-25T12:35:00.237774+0000","last_clean":"2024-04-25T12:35:00.237774+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:35:00.237774+0000","last_undegraded":"2024-04-25T12:35:00.237774+0000","last_fullsized":"2024-04-25T12:35:00.237774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488252+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:35:00.488252+0000","last_peered":"2024-04-25T12:35:00.488252+0000","last_clean":"2024-04-25T12:35:00.488252+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:35:00.488252+0000","last_undegraded":"2024-04-25T12:35:00.488252+0000","last_fullsized":"2024-04-25T12:35:00.488252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235386+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:35:00.235386+0000","last_peered":"2024-04-25T12:35:00.235386+0000","last_clean":"2024-04-25T12:35:00.235386+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:35:00.235386+0000","last_undegraded":"2024-04-25T12:35:00.235386+0000","last_fullsized":"2024-04-25T12:35:00.235386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236858+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:35:00.236858+0000","last_peered":"2024-04-25T12:35:00.236858+0000","last_clean":"2024-04-25T12:35:00.236858+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:35:00.236858+0000","last_undegraded":"2024-04-25T12:35:00.236858+0000","last_fullsized":"2024-04-25T12:35:00.236858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694382+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:35:00.694382+0000","last_peered":"2024-04-25T12:35:00.694382+0000","last_clean":"2024-04-25T12:35:00.694382+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:35:00.694382+0000","last_undegraded":"2024-04-25T12:35:00.694382+0000","last_fullsized":"2024-04-25T12:35:00.694382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694541+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:35:00.694541+0000","last_peered":"2024-04-25T12:35:00.694541+0000","last_clean":"2024-04-25T12:35:00.694541+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:35:00.694541+0000","last_undegraded":"2024-04-25T12:35:00.694541+0000","last_fullsized":"2024-04-25T12:35:00.694541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488638+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:35:00.488638+0000","last_peered":"2024-04-25T12:35:00.488638+0000","last_clean":"2024-04-25T12:35:00.488638+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:35:00.488638+0000","last_undegraded":"2024-04-25T12:35:00.488638+0000","last_fullsized":"2024-04-25T12:35:00.488638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699483+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:35:00.699483+0000","last_peered":"2024-04-25T12:35:00.699483+0000","last_clean":"2024-04-25T12:35:00.699483+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:35:00.699483+0000","last_undegraded":"2024-04-25T12:35:00.699483+0000","last_fullsized":"2024-04-25T12:35:00.699483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238812+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:35:00.238812+0000","last_peered":"2024-04-25T12:35:00.238812+0000","last_clean":"2024-04-25T12:35:00.238812+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:35:00.238812+0000","last_undegraded":"2024-04-25T12:35:00.238812+0000","last_fullsized":"2024-04-25T12:35:00.238812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699291+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:35:00.699291+0000","last_peered":"2024-04-25T12:35:00.699291+0000","last_clean":"2024-04-25T12:35:00.699291+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:35:00.699291+0000","last_undegraded":"2024-04-25T12:35:00.699291+0000","last_fullsized":"2024-04-25T12:35:00.699291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238474+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:35:00.238474+0000","last_peered":"2024-04-25T12:35:00.238474+0000","last_clean":"2024-04-25T12:35:00.238474+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:35:00.238474+0000","last_undegraded":"2024-04-25T12:35:00.238474+0000","last_fullsized":"2024-04-25T12:35:00.238474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236497+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:35:00.236497+0000","last_peered":"2024-04-25T12:35:00.236497+0000","last_clean":"2024-04-25T12:35:00.236497+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:35:00.236497+0000","last_undegraded":"2024-04-25T12:35:00.236497+0000","last_fullsized":"2024-04-25T12:35:00.236497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488552+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:35:00.488552+0000","last_peered":"2024-04-25T12:35:00.488552+0000","last_clean":"2024-04-25T12:35:00.488552+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:35:00.488552+0000","last_undegraded":"2024-04-25T12:35:00.488552+0000","last_fullsized":"2024-04-25T12:35:00.488552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235986+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:35:00.235986+0000","last_peered":"2024-04-25T12:35:00.235986+0000","last_clean":"2024-04-25T12:35:00.235986+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:35:00.235986+0000","last_undegraded":"2024-04-25T12:35:00.235986+0000","last_fullsized":"2024-04-25T12:35:00.235986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236085+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:35:00.236085+0000","last_peered":"2024-04-25T12:35:00.236085+0000","last_clean":"2024-04-25T12:35:00.236085+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:35:00.236085+0000","last_undegraded":"2024-04-25T12:35:00.236085+0000","last_fullsized":"2024-04-25T12:35:00.236085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700980+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:35:00.700980+0000","last_peered":"2024-04-25T12:35:00.700980+0000","last_clean":"2024-04-25T12:35:00.700980+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:35:00.700980+0000","last_undegraded":"2024-04-25T12:35:00.700980+0000","last_fullsized":"2024-04-25T12:35:00.700980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488827+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:35:00.488827+0000","last_peered":"2024-04-25T12:35:00.488827+0000","last_clean":"2024-04-25T12:35:00.488827+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:35:00.488827+0000","last_undegraded":"2024-04-25T12:35:00.488827+0000","last_fullsized":"2024-04-25T12:35:00.488827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236512+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:35:00.236512+0000","last_peered":"2024-04-25T12:35:00.236512+0000","last_clean":"2024-04-25T12:35:00.236512+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:35:00.236512+0000","last_undegraded":"2024-04-25T12:35:00.236512+0000","last_fullsized":"2024-04-25T12:35:00.236512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488137+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:35:00.488137+0000","last_peered":"2024-04-25T12:35:00.488137+0000","last_clean":"2024-04-25T12:35:00.488137+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:35:00.488137+0000","last_undegraded":"2024-04-25T12:35:00.488137+0000","last_fullsized":"2024-04-25T12:35:00.488137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699318+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:35:00.699318+0000","last_peered":"2024-04-25T12:35:00.699318+0000","last_clean":"2024-04-25T12:35:00.699318+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:35:00.699318+0000","last_undegraded":"2024-04-25T12:35:00.699318+0000","last_fullsized":"2024-04-25T12:35:00.699318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694330+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:35:00.694330+0000","last_peered":"2024-04-25T12:35:00.694330+0000","last_clean":"2024-04-25T12:35:00.694330+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:35:00.694330+0000","last_undegraded":"2024-04-25T12:35:00.694330+0000","last_fullsized":"2024-04-25T12:35:00.694330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244838+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:35:00.244838+0000","last_peered":"2024-04-25T12:35:00.244838+0000","last_clean":"2024-04-25T12:35:00.244838+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:35:00.244838+0000","last_undegraded":"2024-04-25T12:35:00.244838+0000","last_fullsized":"2024-04-25T12:35:00.244838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237777+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:35:00.237777+0000","last_peered":"2024-04-25T12:35:00.237777+0000","last_clean":"2024-04-25T12:35:00.237777+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:35:00.237777+0000","last_undegraded":"2024-04-25T12:35:00.237777+0000","last_fullsized":"2024-04-25T12:35:00.237777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238887+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:35:00.238887+0000","last_peered":"2024-04-25T12:35:00.238887+0000","last_clean":"2024-04-25T12:35:00.238887+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:35:00.238887+0000","last_undegraded":"2024-04-25T12:35:00.238887+0000","last_fullsized":"2024-04-25T12:35:00.238887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238159+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:35:00.238159+0000","last_peered":"2024-04-25T12:35:00.238159+0000","last_clean":"2024-04-25T12:35:00.238159+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:35:00.238159+0000","last_undegraded":"2024-04-25T12:35:00.238159+0000","last_fullsized":"2024-04-25T12:35:00.238159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236425+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:35:00.236425+0000","last_peered":"2024-04-25T12:35:00.236425+0000","last_clean":"2024-04-25T12:35:00.236425+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:35:00.236425+0000","last_undegraded":"2024-04-25T12:35:00.236425+0000","last_fullsized":"2024-04-25T12:35:00.236425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488070+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:35:00.488070+0000","last_peered":"2024-04-25T12:35:00.488070+0000","last_clean":"2024-04-25T12:35:00.488070+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:35:00.488070+0000","last_undegraded":"2024-04-25T12:35:00.488070+0000","last_fullsized":"2024-04-25T12:35:00.488070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235433+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:35:00.235433+0000","last_peered":"2024-04-25T12:35:00.235433+0000","last_clean":"2024-04-25T12:35:00.235433+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:35:00.235433+0000","last_undegraded":"2024-04-25T12:35:00.235433+0000","last_fullsized":"2024-04-25T12:35:00.235433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235696+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:35:00.235696+0000","last_peered":"2024-04-25T12:35:00.235696+0000","last_clean":"2024-04-25T12:35:00.235696+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:35:00.235696+0000","last_undegraded":"2024-04-25T12:35:00.235696+0000","last_fullsized":"2024-04-25T12:35:00.235696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244713+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:35:00.244713+0000","last_peered":"2024-04-25T12:35:00.244713+0000","last_clean":"2024-04-25T12:35:00.244713+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:35:00.244713+0000","last_undegraded":"2024-04-25T12:35:00.244713+0000","last_fullsized":"2024-04-25T12:35:00.244713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244441+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:35:00.244441+0000","last_peered":"2024-04-25T12:35:00.244441+0000","last_clean":"2024-04-25T12:35:00.244441+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:35:00.244441+0000","last_undegraded":"2024-04-25T12:35:00.244441+0000","last_fullsized":"2024-04-25T12:35:00.244441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.243194+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:35:00.243194+0000","last_peered":"2024-04-25T12:35:00.243194+0000","last_clean":"2024-04-25T12:35:00.243194+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:35:00.243194+0000","last_undegraded":"2024-04-25T12:35:00.243194+0000","last_fullsized":"2024-04-25T12:35:00.243194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700840+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:35:00.700840+0000","last_peered":"2024-04-25T12:35:00.700840+0000","last_clean":"2024-04-25T12:35:00.700840+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:35:00.700840+0000","last_undegraded":"2024-04-25T12:35:00.700840+0000","last_fullsized":"2024-04-25T12:35:00.700840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488044+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:35:00.488044+0000","last_peered":"2024-04-25T12:35:00.488044+0000","last_clean":"2024-04-25T12:35:00.488044+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:35:00.488044+0000","last_undegraded":"2024-04-25T12:35:00.488044+0000","last_fullsized":"2024-04-25T12:35:00.488044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242559+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:35:00.242559+0000","last_peered":"2024-04-25T12:35:00.242559+0000","last_clean":"2024-04-25T12:35:00.242559+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:35:00.242559+0000","last_undegraded":"2024-04-25T12:35:00.242559+0000","last_fullsized":"2024-04-25T12:35:00.242559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698632+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:35:00.698632+0000","last_peered":"2024-04-25T12:35:00.698632+0000","last_clean":"2024-04-25T12:35:00.698632+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:35:00.698632+0000","last_undegraded":"2024-04-25T12:35:00.698632+0000","last_fullsized":"2024-04-25T12:35:00.698632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244141+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:35:00.244141+0000","last_peered":"2024-04-25T12:35:00.244141+0000","last_clean":"2024-04-25T12:35:00.244141+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:35:00.244141+0000","last_undegraded":"2024-04-25T12:35:00.244141+0000","last_fullsized":"2024-04-25T12:35:00.244141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698101+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:35:00.698101+0000","last_peered":"2024-04-25T12:35:00.698101+0000","last_clean":"2024-04-25T12:35:00.698101+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:35:00.698101+0000","last_undegraded":"2024-04-25T12:35:00.698101+0000","last_fullsized":"2024-04-25T12:35:00.698101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488547+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:35:00.488547+0000","last_peered":"2024-04-25T12:35:00.488547+0000","last_clean":"2024-04-25T12:35:00.488547+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:35:00.488547+0000","last_undegraded":"2024-04-25T12:35:00.488547+0000","last_fullsized":"2024-04-25T12:35:00.488547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488685+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:35:00.488685+0000","last_peered":"2024-04-25T12:35:00.488685+0000","last_clean":"2024-04-25T12:35:00.488685+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:35:00.488685+0000","last_undegraded":"2024-04-25T12:35:00.488685+0000","last_fullsized":"2024-04-25T12:35:00.488685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245426+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:35:00.245426+0000","last_peered":"2024-04-25T12:35:00.245426+0000","last_clean":"2024-04-25T12:35:00.245426+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:35:00.245426+0000","last_undegraded":"2024-04-25T12:35:00.245426+0000","last_fullsized":"2024-04-25T12:35:00.245426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":14904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":117903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":307200,"data_stored":117495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":707759,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":121997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":120463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":707115,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:35:11.290 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T12:35:11.290 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-25T12:35:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.611+0000 7f03ab600640 1 Processor -- start 2024-04-25T12:35:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.611+0000 7f03ab600640 1 -- start start 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.613+0000 7f03ab600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f03a40c4d50 0x7f03a4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.613+0000 7f03ab600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03a415afa0 0x7f03a40cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.613+0000 7f03ab600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a415b3b0 con 0x7f03a4074870 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.613+0000 7f03ab600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f03a40c61f0 con 0x7f03a415afa0 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.613+0000 7f03ab600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f03a40c87d0 con 0x7f03a40c4d50 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a9600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03a415afa0 0x7f03a40cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a9600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03a415afa0 0x7f03a40cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52566/0 (socket says 172.21.15.123:52566) 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a9600640 1 -- 172.21.15.123:0/2661543470 learned_addr learned my addr 172.21.15.123:0/2661543470 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a8c00640 1 --2- 172.21.15.123:0/2661543470 >> v2:172.21.15.153:3300/0 conn(0x7f03a40c4d50 0x7f03a4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a9600640 1 -- 172.21.15.123:0/2661543470 >> v2:172.21.15.153:3300/0 conn(0x7f03a40c4d50 msgr2=0x7f03a4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a9600640 1 --2- 172.21.15.123:0/2661543470 >> v2:172.21.15.153:3300/0 conn(0x7f03a40c4d50 0x7f03a4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a9600640 1 -- 172.21.15.123:0/2661543470 >> v1:172.21.15.153:6789/0 conn(0x7f03a4074870 legacy=0x7f03a40c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.614+0000 7f03a9600640 1 -- 172.21.15.123:0/2661543470 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a4073ce0 con 0x7f03a415afa0 2024-04-25T12:35:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.615+0000 7f03a9600640 1 --2- 172.21.15.123:0/2661543470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a415afa0 0x7f03a40cda90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f03a40ce6d0 tx=0x7f039c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6cd9929ecd8ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.615+0000 7f03a3e00640 1 -- 172.21.15.123:0/2661543470 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c036070 con 0x7f03a415afa0 2024-04-25T12:35:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03a3e00640 1 -- 172.21.15.123:0/2661543470 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039c031600 con 0x7f03a415afa0 2024-04-25T12:35:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03a3e00640 1 -- 172.21.15.123:0/2661543470 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c0319f0 con 0x7f03a415afa0 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 -- 172.21.15.123:0/2661543470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a415afa0 msgr2=0x7f03a40cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 --2- 172.21.15.123:0/2661543470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a415afa0 0x7f03a40cda90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f03a40ce6d0 tx=0x7f039c02a7c0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 -- 172.21.15.123:0/2661543470 shutdown_connections 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 --2- 172.21.15.123:0/2661543470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a415afa0 0x7f03a40cda90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 --2- 172.21.15.123:0/2661543470 >> v2:172.21.15.153:3300/0 conn(0x7f03a40c4d50 0x7f03a4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 -- 172.21.15.123:0/2661543470 >> 172.21.15.123:0/2661543470 conn(0x7f03a406c750 msgr2=0x7f03a406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 -- 172.21.15.123:0/2661543470 shutdown_connections 2024-04-25T12:35:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.616+0000 7f03ab600640 1 -- 172.21.15.123:0/2661543470 wait complete. 2024-04-25T12:35:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.619+0000 7f03ab600640 1 Processor -- start 2024-04-25T12:35:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.619+0000 7f03ab600640 1 -- start start 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.620+0000 7f03ab600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03a4074870 0x7f03a41f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.620+0000 7f03ab600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a40c4d50 0x7f03a41e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.620+0000 7f03ab600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a415afa0 0x7f03a4149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.620+0000 7f03ab600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03a40c8460 con 0x7f03a40c4d50 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03ab600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f03a40742a0 con 0x7f03a415afa0 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03ab600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f03a40c9370 con 0x7f03a4074870 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a415afa0 0x7f03a4149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a415afa0 0x7f03a4149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:50936/0 (socket says 172.21.15.123:50936) 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a8c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03a4074870 0x7f03a41f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 -- 172.21.15.123:0/1351590842 learned_addr learned my addr 172.21.15.123:0/1351590842 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f039be00640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a40c4d50 0x7f03a41e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 -- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03a4074870 msgr2=0x7f03a41f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03a4074870 0x7f03a41f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 -- 172.21.15.123:0/1351590842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a40c4d50 msgr2=0x7f03a41e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a40c4d50 0x7f03a41e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f03a9600640 1 -- 172.21.15.123:0/1351590842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a41f4ce0 con 0x7f03a415afa0 2024-04-25T12:35:11.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.621+0000 7f039be00640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a40c4d50 0x7f03a41e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:35:11.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.622+0000 7f03a9600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a415afa0 0x7f03a4149f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f039c02adb0 tx=0x7f039c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.622+0000 7f03a1600640 1 -- 172.21.15.123:0/1351590842 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c036050 con 0x7f03a415afa0 2024-04-25T12:35:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.622+0000 7f03a1600640 1 -- 172.21.15.123:0/1351590842 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039c042070 con 0x7f03a415afa0 2024-04-25T12:35:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.622+0000 7f03a1600640 1 -- 172.21.15.123:0/1351590842 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c009510 con 0x7f03a415afa0 2024-04-25T12:35:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.622+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a4152390 con 0x7f03a415afa0 2024-04-25T12:35:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.622+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41f4ef0 con 0x7f03a415afa0 2024-04-25T12:35:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.623+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a4068e90 con 0x7f03a415afa0 2024-04-25T12:35:11.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.625+0000 7f03a1600640 1 -- 172.21.15.123:0/1351590842 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f039c004470 con 0x7f03a415afa0 2024-04-25T12:35:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.626+0000 7f03a1600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f03800744b0 0x7f03800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.626+0000 7f03a1600640 1 -- 172.21.15.123:0/1351590842 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f039c0b67f0 con 0x7f03a415afa0 2024-04-25T12:35:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.626+0000 7f03a8c00640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f03800744b0 0x7f03800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.627+0000 7f03a8c00640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f03800744b0 0x7f03800769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f038c009aa0 tx=0x7f038c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:11.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.629+0000 7f03a1600640 1 -- 172.21.15.123:0/1351590842 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f039c080c80 con 0x7f03a415afa0 2024-04-25T12:35:11.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.830+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f03a415a6b0 con 0x7f03a415afa0 2024-04-25T12:35:11.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.831+0000 7f03a1600640 1 -- 172.21.15.123:0/1351590842 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v27) ==== 56+0+3588 (secure 0 0 0) 0x7f039c080580 con 0x7f03a415afa0 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 27 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:fsid 2239057a-dc25-4ee3-8341-8c80e403f296 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-04-25T12:33:54.601604+0000 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:modified 2024-04-25T12:35:00.228853+0000 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:crush_version 6 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:full_ratio 0.95 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:backfillfull_ratio 0.9 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:nearfull_ratio 0.85 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:require_min_compat_client luminous 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:min_compat_client luminous 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:require_osd_release squid 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:stretch_mode_enabled false 2024-04-25T12:35:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:pool 2 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:pool 3 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 25 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:max_osd 8 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0 up in weight 1 up_from 15 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] [v2:172.21.15.123:6818/1405705428,v1:172.21.15.123:6819/1405705428] exists,up 2d9ee216-abfa-45db-a0ac-ef834981be6f 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] [v2:172.21.15.123:6802/1234958824,v1:172.21.15.123:6803/1234958824] exists,up a60a3a6f-376a-4fb2-baf1-e13fc615f6e5 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] [v2:172.21.15.123:6810/2222173849,v1:172.21.15.123:6811/2222173849] exists,up cfb7cbbd-82cd-46fe-b89b-15775df1fe93 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] [v2:172.21.15.123:6826/1491071423,v1:172.21.15.123:6827/1491071423] exists,up 9e8ccd97-ab68-4f03-bbb3-5820eccf4568 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] [v2:172.21.15.153:6804/3135982807,v1:172.21.15.153:6805/3135982807] exists,up 7d5bfd6e-173a-413d-abdb-df6d82c941d3 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] [v2:172.21.15.153:6828/2327013656,v1:172.21.15.153:6829/2327013656] exists,up 9141fa91-c547-4711-910a-9ef123017fb5 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] [v2:172.21.15.153:6812/3626282289,v1:172.21.15.153:6813/3626282289] exists,up 66fbc935-086e-4b03-aa03-96e0bc8b1d27 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7 up in weight 1 up_from 14 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] [v2:172.21.15.153:6820/3086604284,v1:172.21.15.153:6821/3086604284] exists,up de59a619-b84a-4623-ac89-447178815d65 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.15 [0,7] 2024-04-25T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 4.d [3,0] 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f03800744b0 msgr2=0x7f03800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f03800744b0 0x7f03800769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f038c009aa0 tx=0x7f038c009190 comp rx=0 tx=0).stop 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a415afa0 msgr2=0x7f03a4149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a415afa0 0x7f03a4149f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f039c02adb0 tx=0x7f039c004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 shutdown_connections 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f03800744b0 0x7f03800769c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a415afa0 0x7f03a4149f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03a40c4d50 0x7f03a41e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 --2- 172.21.15.123:0/1351590842 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03a4074870 0x7f03a41f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:11.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 >> 172.21.15.123:0/1351590842 conn(0x7f03a406c750 msgr2=0x7f03a40cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:11.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.834+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 shutdown_connections 2024-04-25T12:35:11.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:11.835+0000 7f03ab600640 1 -- 172.21.15.123:0/1351590842 wait complete. 2024-04-25T12:35:11.862 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] [v2:172.21.15.123:6818/1405705428,v1:172.21.15.123:6819/1405705428] exists,up 2d9ee216-abfa-45db-a0ac-ef834981be6f', 'osd.1 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] [v2:172.21.15.123:6802/1234958824,v1:172.21.15.123:6803/1234958824] exists,up a60a3a6f-376a-4fb2-baf1-e13fc615f6e5', 'osd.2 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] [v2:172.21.15.123:6810/2222173849,v1:172.21.15.123:6811/2222173849] exists,up cfb7cbbd-82cd-46fe-b89b-15775df1fe93', 'osd.3 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] [v2:172.21.15.123:6826/1491071423,v1:172.21.15.123:6827/1491071423] exists,up 9e8ccd97-ab68-4f03-bbb3-5820eccf4568', 'osd.4 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] [v2:172.21.15.153:6804/3135982807,v1:172.21.15.153:6805/3135982807] exists,up 7d5bfd6e-173a-413d-abdb-df6d82c941d3', 'osd.5 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] [v2:172.21.15.153:6828/2327013656,v1:172.21.15.153:6829/2327013656] exists,up 9141fa91-c547-4711-910a-9ef123017fb5', 'osd.6 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] [v2:172.21.15.153:6812/3626282289,v1:172.21.15.153:6813/3626282289] exists,up 66fbc935-086e-4b03-aa03-96e0bc8b1d27', 'osd.7 up in weight 1 up_from 14 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] [v2:172.21.15.153:6820/3086604284,v1:172.21.15.153:6821/3086604284] exists,up de59a619-b84a-4623-ac89-447178815d65'] 2024-04-25T12:35:11.863 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-25T12:35:12.138 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:12.138 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mon.a", 2024-04-25T12:35:12.138 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster": "ceph", 2024-04-25T12:35:12.138 INFO:teuthology.orchestra.run.smithi123.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-25T12:35:12.138 INFO:teuthology.orchestra.run.smithi123.stdout: "admin_socket_mode": "", 2024-04-25T12:35:12.138 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-25T12:35:12.138 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_client_required": "cephx, none", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_cluster_required": "cephx", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_debug": "true", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_service_required": "cephx", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_supported": "cephx", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_aio": "true", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_aio_poll_ms": "250", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_aio_reap_max": "16", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_async_discard_threads": "0", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_block_size": "4096", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_debug_aio": "false", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-25T12:35:12.139 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_debug_inflight_ios": "false", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_enable_discard": "false", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_flock_retry": "3", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_inject_crash": "0", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_ioring": "false", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_ioring_hipri": "false", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bdev_type": "", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_alloc_size": "1048576", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_allocator": "hybrid", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_buffered_io": "true", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_check_for_zeros": "false", 2024-04-25T12:35:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_compact_log_sync": "false", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_max_log_runway": "4194304", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_max_prefetch": "1048576", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_min_flush_size": "524288", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_min_log_runway": "1048576", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_replay_recovery": "false", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_sync_write": "false", 2024-04-25T12:35:12.141 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_allocation_from_file": "true", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_allocator": "hybrid", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_create": "true", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_db_create": "false", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_db_path": "", 2024-04-25T12:35:12.142 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_db_size": "0", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_path": "", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_preallocate_file": "false", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_size": "107374182400", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_wal_create": "false", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_wal_path": "", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_block_wal_size": "100663296", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_bluefs": "true", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-25T12:35:12.143 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_autotune": "true", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_size": "0", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_type": "2q", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_clone_cow": "true", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-25T12:35:12.144 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_mode": "none", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_csum_type": "crc32c", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_freelist": "false", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-25T12:35:12.145 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_small_allocations": "0", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_default_buffered_read": "true", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_default_buffered_write": "false", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-25T12:35:12.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_elastic_shared_blobs": "true", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fail_eio": "false", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_on_mount": "false", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-25T12:35:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_on_umount": "false", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_ignore_data_csum": "false", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_log_op_age": "5.000000", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_max_alloc_size": "0", 2024-04-25T12:35:12.148 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_max_blob_size": "0", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_max_deferred_txc": "32", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_min_alloc_size": "0", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_nid_prealloc": "1024", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_retry_disk_reads": "3", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_rocksdb_cf": "true", 2024-04-25T12:35:12.149 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_spdk_mem": "512", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-25T12:35:12.150 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_tracing": "false", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_zero_block_detection": "false", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-25T12:35:12.151 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephx_cluster_require_signatures": "false", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephx_cluster_require_version": "2", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephx_require_signatures": "false", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephx_require_version": "2", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephx_service_require_signatures": "false", 2024-04-25T12:35:12.152 INFO:teuthology.orchestra.run.smithi123.stdout: "cephx_service_require_version": "2", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "cephx_sign_messages": "true", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "chdir": "", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_acl_type": "", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_asio_thread_count": "2", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_cache_mid": "0.750000", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_cache_size": "16384", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_caps_release_delay": "5", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_check_pool_perm": "true", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_debug_force_sync_read": "false", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_debug_getattr_caps": "false", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_debug_inject_tick_delay": "0", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_die_on_failed_remount": "false", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_dirsize_rbytes": "true", 2024-04-25T12:35:12.153 INFO:teuthology.orchestra.run.smithi123.stdout: "client_force_lazyio": "false", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_fs": "", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_inject_release_failure": "false", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_max_inline_size": "4096", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_mds_namespace": "", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_metadata": "", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_mount_gid": "-1", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_mount_timeout": "300", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_mount_uid": "-1", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_mountpoint": "/", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_notify_timeout": "10", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_oc": "true", 2024-04-25T12:35:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: "client_oc_max_dirty": "104857600", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_oc_max_objects": "1000", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_oc_size": "209715200", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_oc_target_dirty": "8388608", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_permissions": "true", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_quota": "true", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_quota_df": "true", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_readahead_max_bytes": "0", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_readahead_max_periods": "4", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_readahead_min": "131072", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_reconnect_stale": "false", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_shutdown_timeout": "30", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_snapdir": ".snap", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_tick_interval": "1", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_trace": "", 2024-04-25T12:35:12.155 INFO:teuthology.orchestra.run.smithi123.stdout: "client_try_dentry_invalidate": "false", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "client_use_faked_inos": "false", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "client_use_random_mds": "false", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "clog_to_graylog": "false", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "clog_to_graylog_port": "12201", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "clog_to_monitors": "default=true", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "clog_to_syslog": "false", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "clog_to_syslog_level": "info", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "-", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_network": "", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_network_interface": "", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "compressor_zlib_isal": "false", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "compressor_zlib_level": "5", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "compressor_zlib_winsize": "-15", 2024-04-25T12:35:12.156 INFO:teuthology.orchestra.run.smithi123.stdout: "compressor_zstd_level": "1", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crimson_alien_op_num_threads": "6", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crimson_osd_obc_lru_size": "512", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crimson_osd_stat_interval": "0", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crimson_seastar_cpu_cores": "", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_location": "", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_location_hook": "", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_location_hook_timeout": "10", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "daemonize": "false", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "daos_pool": "tank", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-25T12:35:12.157 INFO:teuthology.orchestra.run.smithi123.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_alienstore": "0/5", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_allow_any_pool_priority": "false", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_asok": "1/5", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_asok_assert_abort": "false", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_asserts_on_shutdown": "true", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_auth": "20/20", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_bdev": "1/3", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_bluefs": "1/5", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_bluestore": "1/5", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_buffer": "0/1", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_ceph_dedup": "0/5", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_ceph_exporter": "1/5", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_cephfs_mirror": "0/5", 2024-04-25T12:35:12.158 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_cephsqlite": "0/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_client": "0/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_compressor": "1/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_context": "0/1", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_crimson_interrupt": "0/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_crush": "1/1", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_crypto": "1/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_cyanstore": "0/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_deliberately_leak_memory": "false", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_disable_randomized_ping": "false", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_dpdk": "1/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_eventtrace": "1/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_filer": "0/1", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_filestore": "1/3", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_finisher": "1/1", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_fuse": "1/5", 2024-04-25T12:35:12.159 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_heartbeat_testing_span": "0", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_heartbeatmap": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_javaclient": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_journal": "1/3", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_journaler": "0/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_kstore": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_lockdep": "0/1", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mclock": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mds": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mds_balancer": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mds_locker": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mds_log": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mds_log_expire": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mds_migrator": "1/5", 2024-04-25T12:35:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mds_quiesce": "3/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_memstore": "1/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mgr": "2/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mgrc": "1/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_mon": "20/20", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_monc": "0/10", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_ms": "1/1", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_none": "0/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_objclass": "0/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_objectcacher": "0/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_objecter": "0/1", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_optracker": "0/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_osd": "1/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_paxos": "20/20", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_perfcounter": "1/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_prioritycache": "1/5", 2024-04-25T12:35:12.161 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rados": "0/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rbd": "0/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rbd_mirror": "0/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rbd_pwl": "0/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rbd_replay": "0/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_refs": "0/0", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_reserver": "1/1", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw_access": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw_datacache": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw_dbstore": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw_flight": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw_lifecycle": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw_notification": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rgw_sync": "1/5", 2024-04-25T12:35:12.162 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_rocksdb": "4/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_backref": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_cache": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_cleaner": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_device": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_epm": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_journal": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_lba": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_odata": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_omap": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_onode": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_t": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_seastore_tm": "0/5", 2024-04-25T12:35:12.163 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_striper": "0/1", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_test": "0/5", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_throttle": "1/1", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_timer": "0/1", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_tp": "0/5", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "debug_trace": "1/5", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "device_failure_prediction_mode": "none", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "err_to_graylog": "false", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "err_to_journald": "false", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "err_to_stderr": "true", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "err_to_syslog": "false", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event_tracing": "false", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "exporter_addr": "0.0.0.0", 2024-04-25T12:35:12.164 INFO:teuthology.orchestra.run.smithi123.stdout: "exporter_http_port": "9926", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "exporter_prio_limit": "5", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "exporter_sort_metrics": "true", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "exporter_stats_period": "5", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "fake_statfs_for_testing": "0", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "fatal_signal_handlers": "true", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filer_max_purge_ops": "10", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filer_max_truncate_ops": "128", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_apply_finisher_threads": "1", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_blackhole": "false", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_btrfs_clone_range": "true", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_btrfs_snap": "true", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_caller_concurrency": "10", 2024-04-25T12:35:12.165 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_collect_device_partition_information": "true", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_commit_timeout": "600.000000", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_debug_inject_read_err": "false", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_debug_omap_check": "false", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_debug_verify_split": "false", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_dump_file": "", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_fadvise": "true", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_fail_eio": "true", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_fd_cache_shards": "16", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_fd_cache_size": "128", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_fiemap": "false", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_fiemap_threshold": "4096", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-25T12:35:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_inject_stall": "0", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_journal_parallel": "false", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_journal_trailing": "false", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_journal_writeahead": "false", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_kill_at": "0", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattrs": "0", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_xattr_value_size": "0", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-25T12:35:12.167 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_merge_threshold": "-10", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_odsync_write": "false", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_omap_backend": "rocksdb", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_omap_backend_path": "", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_op_thread_timeout": "60", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_op_threads": "2", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_punch_hole": "false", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-25T12:35:12.168 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_queue_max_ops": "50", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_seek_data_hole": "false", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_sloppy_crc": "false", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_splice": "false", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_split_multiple": "2", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_split_rand_factor": "20", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_update_to": "1000", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-25T12:35:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_enable": "true", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_xfs_extsize": "false", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "filestore_zfs_snap": "false", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "fio_dir": "/tmp/fio", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_allow_other": "true", 2024-04-25T12:35:12.170 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_atomic_o_trunc": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_big_writes": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_debug": "false", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_default_permissions": "false", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_disable_pagecache": "false", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_max_write": "0", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_multithreaded": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_require_active_mds": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_set_user_groups": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_splice_move": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_splice_read": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_splice_write": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "fuse_use_invalidate_cb": "true", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "gss_target_name": "ceph", 2024-04-25T12:35:12.171 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_file": "", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_inject_failure": "0", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_interval": "5", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "host": "smithi123", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-25T12:35:12.172 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "inject_early_sigterm": "false", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "jaeger_agent_port": "6799", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "jaeger_tracing_enable": "false", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_aio": "true", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_align_min_size": "65536", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_block_align": "true", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_block_size": "4096", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_dio": "true", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_discard": "false", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_force_aio": "false", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_ignore_corruption": "false", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_max_write_bytes": "10485760", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_max_write_entries": "100", 2024-04-25T12:35:12.173 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_replay_from": "0", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_write_header_frequency": "0", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journal_zero_on_create": "false", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journaler_prefetch_periods": "10", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journaler_prezero_periods": "5", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "journaler_write_head_interval": "15", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "keyfile": "", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_backend": "rocksdb", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_default_stripe_size": "65536", 2024-04-25T12:35:12.174 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_fsck_on_mount": "false", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_max_bytes": "67108864", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_max_ops": "512", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_nid_prealloc": "1024", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_onode_map_size": "1024", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_sync_submit_transaction": "false", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "kstore_sync_transaction": "false", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "librados_thread_count": "2", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "lockdep": "false", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "lockdep_force_backtrace": "false", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "log_coarse_timestamps": "true", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "log_flush_on_exit": "false", 2024-04-25T12:35:12.175 INFO:teuthology.orchestra.run.smithi123.stdout: "log_graylog_host": "127.0.0.1", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_graylog_port": "12201", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_max_new": "1000", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_max_recent": "10000", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_stderr_prefix": "", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_stop_at_utilization": "0.970000", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": "true", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_graylog": "false", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_journald": "false", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_stderr": "false", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_syslog": "false", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "max_rotating_auth_attempts": "10", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_action_on_write_error": "1", 2024-04-25T12:35:12.176 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_alternate_name_max": "8192", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_asio_thread_count": "2", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_export_pin": "true", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_fragment_dirs": "true", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_fragment_interval": "5", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_interval": "10", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_max": "-1", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_max_until": "-1", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_merge_size": "50", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_midchunk": "0.300000", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_min_start": "0.200000", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_minchunk": "0.001000", 2024-04-25T12:35:12.177 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_mode": "0", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_need_max": "1.200000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_need_min": "0.800000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_overload_epochs": "2", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_split_bits": "3", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_split_size": "10000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_target_decay": "10.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_beacon_grace": "15.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_beacon_interval": "4.000000", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-25T12:35:12.178 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_mid": "0.700000", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_quiesce_delay": "0", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_quiesce_sleep": "200", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_quiesce_splitauth": "true", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_quiesce_threshold": "524288", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_release_free_interval": "10", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_reservation": "0.050000", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_trim_interval": "1", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache_trim_threshold": "262144", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-25T12:35:12.179 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_client_prealloc_inos": "1000", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_connect_bootstrapping": "false", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_damage_table_max_entries": "10000", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_debug_auth_pins": "false", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_debug_frag": "false", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_debug_scatterstat": "false", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_debug_subtrees": "false", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_decay_halflife": "5.000000", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_default_dir_hash": "2", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_defer_session_stale": "true", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_deny_all_reconnect": "false", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dir_keys_per_op": "16384", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dir_max_commit_size": "10", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dir_max_entries": "0", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dir_prefetch": "true", 2024-04-25T12:35:12.180 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dump_cache_on_map": "false", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_early_reply": "true", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_enable_op_tracker": "true", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_enforce_unique_name": "true", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_export_ephemeral_random": "true", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-25T12:35:12.181 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_health_summarize_threshold": "10", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_inject_health_dummy": "false", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_inject_migrator_session_race": "false", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_join_fs": "", 2024-04-25T12:35:12.182 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_journal_format": "1", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_create_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_export_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_import_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_journal_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_journal_expire_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_journal_replay_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_link_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_mdstable_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_openc_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_rename_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_shutdown_at": "0", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_event_large_threshold": "524288", 2024-04-25T12:35:12.183 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_events_per_segment": "1024", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_major_segment_event_ratio": "12", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_max_events": "-1", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_max_segments": "128", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_pause": "false", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_segment_size": "0", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_skip_unbounded_events": "false", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_trim_threshold": "128", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log_warn_factor": "2.000000", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_caps_per_client": "1048576", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_completed_flushes": "100000", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_completed_requests": "100000", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_export_size": "20971520", 2024-04-25T12:35:12.184 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_file_recover": "32", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_purge_files": "64", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_purge_ops": "8192", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_max_snaps_per_dir": "100", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_metrics_update_interval": "2", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_min_caps_per_client": "100", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_min_caps_working_set": "10000", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_numa_node": "-1", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_op_complaint_time": "30.000000", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_op_history_duration": "600", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_op_history_size": "20", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_op_history_slow_op_size": "20", 2024-04-25T12:35:12.185 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_op_log_threshold": "5", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_ping_grace": "15", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_ping_interval": "5", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_recall_max_caps": "30000", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_recall_warning_threshold": "262144", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_replay_interval": "1.000000", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-25T12:35:12.186 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_root_ino_gid": "0", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_root_ino_uid": "0", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_shutdown_check": "0", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_skip_ino": "0", 2024-04-25T12:35:12.187 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_snap_max_uid": "4294967294", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_snap_min_uid": "0", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_snap_rstat": "false", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_standby_replay_damaged": "false", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_symlink_recovery": "true", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_thrash_exports": "0", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_thrash_fragments": "0", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_tick_interval": "5.000000", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_valgrind_exit": "false", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_verify_backtrace": "1", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_verify_scatter": "false", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_wipe_sessions": "false", 2024-04-25T12:35:12.188 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool_debug": "false", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "memstore_device_bytes": "1073741824", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "memstore_page_set": "false", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "memstore_page_size": "65536", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_client_bytes": "134217728", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_client_messages": "512", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_disabled_modules": "", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_enable_op_tracker": "true", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-25T12:35:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_max_pg_creating": "1024", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_max_pg_num_change": "128", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_mds_bytes": "134217728", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_mds_messages": "128", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_mon_bytes": "134217728", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_mon_messages": "128", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_num_op_tracker_shard": "32", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_op_complaint_time": "30.000000", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_op_history_duration": "600", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_op_history_size": "20", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_op_history_slow_op_size": "20", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-04-25T12:35:12.190 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_op_log_threshold": "5", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_osd_bytes": "536870912", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_osd_messages": "8192", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_pool": "true", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_standby_modules": "true", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_stats_period": "5", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_stats_threshold": "5", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_tick_period": "2", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_allow_pool_delete": "true", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_allow_pool_size_one": "true", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_auth_validate_all_caps": "true", 2024-04-25T12:35:12.191 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_bytes": "104857600", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_directed_command_retry": "2", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_hunt_parallel": "3", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_log_interval": "1.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_ping_interval": "10.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_client_target_rank": "-1", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-25T12:35:12.192 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_level": "debug", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_file": "true", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_journald": "false", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_compact_on_bootstrap": "false", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_compact_on_start": "false", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_compact_on_trim": "true", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-25T12:35:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_cpu_threads": "4", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_crush_min_required_version": "hammer", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_daemon_bytes": "419430400", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_data_avail_crit": "5", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_data_avail_warn": "5", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_data_size_warn": "16106127360", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_dump_json": "false", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_dump_transactions": "false", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_extra_checks": "false", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-25T12:35:12.194 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_no_require_reef": "false", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_no_require_squid": "false", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_down_mkfs_grace": "300", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_down_uptime_grace": "60", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_election_default_strategy": "1", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_election_timeout": "5.000000", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_elector_ping_divisor": "2", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_enable_op_tracker": "true", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_fake_pool_delete": "false", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_force_quorum_join": "false", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_globalid_prealloc": "10000", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_health_detail_to_clog": "false", 2024-04-25T12:35:12.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_health_log_update_period": "5", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_health_max_detail": "50", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_health_to_clog": "true", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_health_to_clog_interval": "600", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_host": "172.21.15.123,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_host_override": "", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_initial_members": "", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_lease": "5.000000", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-25T12:35:12.196 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_log_full_interval": "50", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_log_max": "10000", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_log_max_summary": "50", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_log_epochs": "500", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_osd": "10000", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_pg_per_osd": "10000", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_pool_pg_num": "65536", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mds_force_trim_to": "0", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mds_skip_sanity": "false", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_memory_autotune": "true", 2024-04-25T12:35:12.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_memory_target": "2147483648", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mgr_beacon_grace": "30", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mgr_digest_period": "5", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mgr_inactive_grace": "60", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_min_osdmap_epochs": "500", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_op_complaint_time": "120", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_op_history_duration": "600", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_op_history_size": "20", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_op_history_slow_op_size": "20", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_op_log_threshold": "5", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-25T12:35:12.198 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_auto_mark_in": "false", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_cache_size": "500", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_down_out_interval": "600", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_force_trim_to": "0", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-25T12:35:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_min_down_reporters": "2", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_report_timeout": "900", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-25T12:35:12.200 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pg_stuck_threshold": "60", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_probe_timeout": "2.000000", 2024-04-25T12:35:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_reweight_max_change": "0.050000", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_reweight_max_osds": "4", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_scrub_interval": "86400", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_scrub_max_keys": "100", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_scrub_timeout": "300", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_session_timeout": "300", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_smart_report_timeout": "5", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_stat_smooth_intervals": "6", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-25T12:35:12.202 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_stretch_pool_min_size": "2", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_stretch_pool_size": "4", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_sync_debug": "false", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_sync_provider_kill_at": "0", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_sync_requester_kill_at": "0", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_sync_timeout": "60.000000", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_target_pg_per_osd": "100", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_tick_interval": "5", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_timecheck_interval": "300.000000", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_use_min_delay_socket": "false", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_older_version_delay": "604800", 2024-04-25T12:35:12.203 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_misplaced": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_older_version": "true", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_pool_no_app": "true", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-25T12:35:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "monmap": "", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "motr_tracing_enabled": "false", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_op_threads": "3", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_cm": "false", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_device_name": "", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_dscp": "96", 2024-04-25T12:35:12.205 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_local_gid": "", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_port_num": "1", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_sl": "3", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_support_srq": "true", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_rdma_type": "ib", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_async_reap_threshold": "5", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_before_connect": "false", 2024-04-25T12:35:12.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_ipv4": "true", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_ipv6": "false", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_msgr1": "true", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_msgr2": "true", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_port_max": "7568", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_port_min": "6800", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_retry_count": "3", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_bind_retry_delay": "5", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_blackhole_client": "false", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_blackhole_mds": "false", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_blackhole_mgr": "false", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_blackhole_mon": "false", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_blackhole_osd": "false", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_client_mode": "crc secure", 2024-04-25T12:35:12.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_cluster_mode": "crc secure", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_cluster_type": "", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_compress_secure": "false", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_connection_idle_timeout": "900", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_connection_ready_timeout": "10", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_crc_data": "true", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_crc_header": "true", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_die_on_bad_msg": "false", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_die_on_bug": "true", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_die_on_old_message": "true", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_die_on_skipped_message": "false", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_coremask": "0xF", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-25T12:35:12.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_enable_tso": "true", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_hugepages": "", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_lro": "true", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_memory_channel": "4", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_pmd": "", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_port_id": "0", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_dump_on_send": "false", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_initial_backoff": "0.200000", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_inject_delay_max": "1.000000", 2024-04-25T12:35:12.209 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_inject_delay_type": "", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_inject_network_congestion": "0", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_inject_socket_failures": "0", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_learn_addr_from_peer": "true", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_max_accept_failures": "4", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_max_backoff": "15.000000", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_mon_client_mode": "secure crc", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_mon_service_mode": "secure crc", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_osd_compress_min_size": "1024", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_osd_compress_mode": "none", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-25T12:35:12.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_pq_min_cost": "65536", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_public_type": "", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_service_mode": "crc secure", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_tcp_listen_backlog": "512", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_tcp_nodelay": "true", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_tcp_rcvbuf": "0", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ms_type": "async+posix", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "no_config_file": "false", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_completion_locks_per_session": "32", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_inflight_ops": "1024", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-25T12:35:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_tick_interval": "5.000000", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter_timeout": "10.000000", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "objectstore_blackhole": "false", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "openssl_engine_opts": "", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_agent_delay_time": "5.000000", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_agent_hist_halflife": "1000", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_agent_max_low_ops": "2", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_agent_max_ops": "4", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_agent_slop": "0.020000", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-25T12:35:12.212 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_asio_thread_count": "2", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_async_recovery_min_cost": "100", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_backfill_scan_max": "512", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_backfill_scan_min": "64", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_backoff_on_degraded": "false", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_backoff_on_peering": "false", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_backoff_on_unfound": "true", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_beacon_report_interval": "300", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bench_duration": "30", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bench_max_block_size": "67108864", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-25T12:35:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_blkin_trace_all": "false", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_check_for_log_corruption": "false", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_class_update_on_start": "true", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_client_message_cap": "256", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_client_message_size_cap": "524288000", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_client_op_priority": "63", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_client_watch_timeout": "30", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_command_max_records": "256", 2024-04-25T12:35:12.214 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_compact_on_start": "false", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_crush_update_on_start": "true", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_crush_update_weight_set": "true", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_feed_pullee": "-1", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_misdirected_ops": "false", 2024-04-25T12:35:12.215 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_no_acting_change": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_no_purge_strays": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_op_order": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_shutdown": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_trim_objects": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_verify_snaps": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_deep_scrub_keys": "1024", 2024-04-25T12:35:12.216 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_deep_scrub_stride": "524288", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_default_notify_timeout": "30", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_delete_sleep": "0.000000", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_discard_disconnected_ops": "true", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_enable_op_tracker": "true", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-25T12:35:12.217 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_fast_info": "true", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_fast_shutdown": "true", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_function_tracing": "false", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_heartbeat_grace": "20", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_heartbeat_interval": "6", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_heartbeat_min_peers": "10", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_heartbeat_min_size": "2000", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_heartbeat_stale": "600", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-25T12:35:12.218 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_hit_set_max_size": "100000", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_hit_set_min_size": "1000", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_journal_size": "5120", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_kill_backfill_at": "0", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_map_cache_size": "50", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_map_dedup": "true", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_map_message_max": "40", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-25T12:35:12.219 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_map_share_max_epochs": "40", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_attr_name_len": "100", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_attr_size": "0", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_backfills": "1", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_markdown_count": "5", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_markdown_period": "600", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_object_name_len": "2048", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_object_namespace_len": "256", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_object_size": "134217728", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_pg_blocked_by": "16", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_pg_log_entries": "10000", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_pgls": "1024", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_push_cost": "8388608", 2024-04-25T12:35:12.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_push_objects": "10", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_scrubs": "3", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_trimming_pgs": "2", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_write_op_reply_len": "64", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_max_write_size": "90", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_profile": "balanced", 2024-04-25T12:35:12.221 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_memory_base": "805306368", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_memory_cache_min": "134217728", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-25T12:35:12.222 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_memory_target": "4294967296", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_memory_target_autotune": "false", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_min_pg_log_entries": "250", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_min_recovery_priority": "0", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mon_report_interval": "5", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_num_cache_shards": "32", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_num_op_tracker_shard": "32", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_numa_auto_affinity": "true", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_numa_node": "-1", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_numa_prefer_iface": "true", 2024-04-25T12:35:12.223 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_objecter_finishers": "1", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_objectstore": "bluestore", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_objectstore_fuse": "false", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_objectstore_tracing": "false", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_complaint_time": "30.000000", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_history_duration": "600", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_history_size": "20", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_history_slow_op_size": "20", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_log_threshold": "5", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_num_shards": "0", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_num_shards_hdd": "5", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_num_shards_ssd": "8", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-25T12:35:12.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_pq_min_cost": "65536", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_queue_cut_off": "high", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_op_thread_timeout": "15", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_open_classes_on_start": "true", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_os_flags": "0", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_peering_op_priority": "255", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_delete_cost": "1048576", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_delete_priority": "5", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_log_trim_max": "10000", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_log_trim_min": "100", 2024-04-25T12:35:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_crimson": "false", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-25T12:35:12.226 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_flags": "0", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_min_size": "0", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_pg_num": "32", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_pgp_num": "0", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_read_ratio": "70", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_size": "2", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_default_type": "replicated", 2024-04-25T12:35:12.227 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_push_per_object_cost": "1000", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recover_clone_overlap": "true", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_cost": "20971520", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_max_active": "0", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_max_single_start": "1", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_op_priority": "3", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-25T12:35:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_priority": "5", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_sleep": "0.000000", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_repair_during_recovery": "false", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_requested_scrub_priority": "120", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_auto_repair": "false", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_begin_hour": "0", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_begin_week_day": "0", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_chunk_max": "25", 2024-04-25T12:35:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_chunk_min": "5", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_cost": "52428800", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_during_recovery": "false", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_end_hour": "0", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_end_week_day": "0", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_event_cost": "4096", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_invalid_stats": "true", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_max_preemptions": "5", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_priority": "5", 2024-04-25T12:35:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_reservation_timeout": "300000", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_scrub_sleep": "0.000000", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_skip_data_digest": "false", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_smart_report_timeout": "5", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_snap_trim_cost": "1048576", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_snap_trim_priority": "5", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-25T12:35:12.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_target_transaction_size": "30", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_tracing": "false", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_use_stale_snap": "false", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "osdc_blkin_trace_all": "false", 2024-04-25T12:35:12.232 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_kill_at": "0", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_max_join_drift": "10", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_min": "500", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_min_wait": "0.050000", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_propose_interval": "1.000000", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_service_trim_max": "500", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_service_trim_min": "250", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_stash_full_interval": "25", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_trim_max": "500", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "paxos_trim_min": "250", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "perf": "true", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-25T12:35:12.233 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "-", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrv": "", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "public_bind_addr": "-", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "public_network": "", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "public_network_interface": "", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "qat_compressor_enabled": "false", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "qat_compressor_session_max_number": "256", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rados_mon_op_timeout": "0", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rados_osd_op_timeout": "0", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rados_tracing": "false", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_atime_update_interval": "60", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_balance_parent_reads": "false", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_balance_snap_reads": "false", 2024-04-25T12:35:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_blkin_trace_all": "false", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache": "true", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_policy": "writearound", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_size": "33554432", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_clone_copy_on_read": "false", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_compression_hint": "none", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_concurrent_management_ops": "10", 2024-04-25T12:35:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_clone_format": "auto", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_data_pool": "", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_features": "61", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_format": "2", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_map_options": "", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_order": "22", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_pool": "rbd", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_stripe_count": "0", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_default_stripe_unit": "0", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_enable_alloc_hint": "true", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-25T12:35:12.236 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_io_scheduler": "simple", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_order": "24", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_pool": "", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_journal_splay_width": "4", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_localize_parent_reads": "false", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_localize_snap_reads": "false", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-25T12:35:12.237 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-25T12:35:12.238 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-25T12:35:12.239 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_mtime_update_interval": "60", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_non_blocking_aio": "true", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_op_thread_timeout": "60", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_op_threads": "1", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_parent_cache_enabled": "false", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_plugins": "", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_bps_burst": "0", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_bps_limit": "0", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_exclude_ops": "0", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_iops_burst": "0", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_iops_limit": "0", 2024-04-25T12:35:12.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_read_from_replica_policy": "default", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-25T12:35:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_skip_partial_discard": "true", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_tracing": "false", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_validate_names": "true", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_validate_pool": "true", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "restapi_base_url": "", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "restapi_log_level": "", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_account_default_quota_max_size": "-1", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_acl_grants_max_num": "100", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_admin_entry": "admin", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-25T12:35:12.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_backend_store": "rados", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_barbican_url": "", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_beast_enable_async": "true", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_counters_cache": "false", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_cache_enabled": "true", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_cache_expiry_interval": "900", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_cache_lru_size": "10000", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_config_store": "rados", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_content_length_compat": "false", 2024-04-25T12:35:12.243 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_copy_obj_progress": "true", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_cors_rules_max_num": "100", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_cross_domain_policy": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_addr": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_password": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_kmip_username": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_require_ssl": "true", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-25T12:35:12.244 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_addr": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_auth": "token", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_namespace": "", 2024-04-25T12:35:12.245 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_prefix": "", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_token_file": "", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_curl_buffersize": "524288", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_curl_low_speed_time": "300", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-25T12:35:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_host": "127.0.0.1", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_l1_fadvise": "4", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_d4n_port": "6379", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data_log_changes_size": "1000", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data_log_num_shards": "128", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-25T12:35:12.247 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data_log_window": "30", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-25T12:35:12.248 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dns_name": "", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dns_s3website_name": "", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_dynamic_resharding": "true", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enable_gc_threads": "true", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enable_lc_threads": "true", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enable_ops_log": "false", 2024-04-25T12:35:12.249 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enable_quota_threads": "true", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enable_static_website": "false", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enable_usage_log": "false", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_enforce_swift_acls": "true", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_exit_timeout_secs": "120", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_expose_bucket": "false", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_extended_http_attrs": "", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_filter": "none", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_frontends": "beast port=7480", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_max_deferred": "50", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_max_objs": "32", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-25T12:35:12.250 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_gc_processor_period": "3600", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_init_timeout": "300", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_kafka_connection_idle": "300", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_kafka_message_timeout": "5000", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_kafka_sleep_timeout": "10", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-04-25T12:35:12.251 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_domain": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_password": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_password_path": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_project": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_tenant": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_token": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_token_path": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_admin_user": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_api_version": "2", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_barbican_domain": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_barbican_password": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_barbican_project": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-25T12:35:12.252 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_barbican_user": "", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_url": "", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lc_debug_interval": "-1", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lc_lock_max_time": "90", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lc_max_objs": "32", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lc_max_rules": "1000", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lc_max_worker": "3", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lc_thread_delay": "0", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-25T12:35:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ldap_dnattr": "uid", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ldap_searchfilter": "", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ldap_uri": "", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lfuda_sync_frequency": "60", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_local_cache_address": "127.0.0.1:6379", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_log_http_headers": "", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_log_object_name_utc": "false", 2024-04-25T12:35:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_attr_name_len": "0", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_attr_size": "0", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_chunk_size": "4194304", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_listing_results": "1000", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_notify_retries": "10", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_put_param_size": "1048576", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_put_size": "5368709120", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_max_slo_entries": "1000", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_md_log_max_shards": "64", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-25T12:35:12.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_mp_lock_max_time": "600", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_max_gc": "300", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-25T12:35:12.256 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_num_async_rados_threads": "32", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_num_control_oids": "8", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_numa_node": "-1", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_objexp_chunk_size": "100", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_objexp_gc_interval": "600", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_op_thread_timeout": "600", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_op_tracing": "false", 2024-04-25T12:35:12.257 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_opa_token": "", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_opa_url": "", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_opa_verify_ssl": "true", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ops_log_rados": "false", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_ops_log_socket_path": "", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_period_push_interval": "2.000000", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-04-25T12:35:12.258 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_posix_cache_lanes": "3", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_posix_cache_max_buckets": "100", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_posix_cache_partitions": "3", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_print_continue": "true", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_rados_tracing": "false", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_realm": "", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_realm_id": "", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-25T12:35:12.259 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_region": "", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_request_uri": "", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_reshard_batch_size": "64", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_reshard_max_aio": "128", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_reshard_num_logs": "16", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_reshard_thread_interval": "600", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_resolve_cname": "false", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_run_sync_thread": "true", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-25T12:35:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_scheduler_type": "throttler", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_script_uri": "", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_service_provider_name": "", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sts_client_id": "", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sts_client_secret": "", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sts_entry": "sts", 2024-04-25T12:35:12.261 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sts_key": "sts", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sts_min_session_duration": "900", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sts_token_introspection_url": "", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_account_in_url": "false", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_auth_entry": "auth", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_auth_url": "", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_custom_header": "", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_need_stats": "true", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_tenant_name": "", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_token_expiration": "86400", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_url": "", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_url_prefix": "swift", 2024-04-25T12:35:12.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_lease_period": "120", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_thread_pool_size": "512", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_topic_persistency_max_retries": "0", 2024-04-25T12:35:12.263 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_topic_require_publish_policy": "false", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_comment": "", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_createby": "", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_encoding": "", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_flag": "false", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_max_size": "5368709120", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_origin": "", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_torrent_tracker": "", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_trust_forwarded_https": "false", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_usage_max_shards": "32", 2024-04-25T12:35:12.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_usage_max_user_shards": "1", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_use_opa_authz": "false", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_counters_cache": "false", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_counters_cache_size": "10000", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_max_buckets": "1000", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_policies_max_num": "100", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_user_unique_email": "true", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_verify_ssl": "true", 2024-04-25T12:35:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_zone": "", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_zone_id": "", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_zonegroup": "", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_zonegroup_id": "", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgwlc_auto_session_clear": "true", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_block_size": "4096", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-25T12:35:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cache_size": "536870912", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_index_type": "binary_search", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_partition_filters": "false", 2024-04-25T12:35:12.267 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_perf": "false", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "run_dir": "/var/run/ceph", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_block_create": "true", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_cache_lru_size": "67108864", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_data_delta_based_overwrite": "0", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_default_max_object_size": "16777216", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_device_size": "53687091200", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_journal_batch_capacity": "16", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-25T12:35:12.268 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_main_device_type": "SSD", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "seastore_segment_size": "67108864", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "set_keepcaps": "false", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "setgroup": "", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "setuser": "", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "setuser_match_path": "", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "thp": "false", 2024-04-25T12:35:12.269 INFO:teuthology.orchestra.run.smithi123.stdout: "threadpool_default_timeout": "60", 2024-04-25T12:35:12.270 INFO:teuthology.orchestra.run.smithi123.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-25T12:35:12.270 INFO:teuthology.orchestra.run.smithi123.stdout: "throttler_perf_counter": "true" 2024-04-25T12:35:12.270 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:12.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-25T12:35:12.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.551+0000 7fec25c00640 1 Processor -- start 2024-04-25T12:35:12.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.551+0000 7fec25c00640 1 -- start start 2024-04-25T12:35:12.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec25c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec200c9f00 0x7fec200c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec25c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200730e0 0x7fec200734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec25c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec20074b20 con 0x7fec20154650 2024-04-25T12:35:12.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec25c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fec200ca780 con 0x7fec200c9f00 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec25c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fec200caa60 con 0x7fec200730e0 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec1ea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fec20154650 0x7fec2015ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40302/0 (socket says 172.21.15.123:40302) 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec1ea00640 1 -- 172.21.15.123:0/2472828246 learned_addr learned my addr 172.21.15.123:0/2472828246 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.553+0000 7fec16000640 1 --2- 172.21.15.123:0/2472828246 >> v2:172.21.15.123:3300/0 conn(0x7fec200c9f00 0x7fec200c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.554+0000 7fec1f400640 1 --2- 172.21.15.123:0/2472828246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200730e0 0x7fec200734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.554+0000 7fec1e000640 1 -- 172.21.15.123:0/2472828246 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629984054 0 0) 0x7fec20074b20 con 0x7fec20154650 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.554+0000 7fec1e000640 1 -- 172.21.15.123:0/2472828246 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec04003520 con 0x7fec20154650 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.554+0000 7fec16000640 1 -- 172.21.15.123:0/2472828246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200730e0 msgr2=0x7fec200734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.554+0000 7fec16000640 1 --2- 172.21.15.123:0/2472828246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200730e0 0x7fec200734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.554+0000 7fec16000640 1 -- 172.21.15.123:0/2472828246 >> v1:172.21.15.123:6789/0 conn(0x7fec20154650 legacy=0x7fec2015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.554+0000 7fec16000640 1 -- 172.21.15.123:0/2472828246 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2015da50 con 0x7fec200c9f00 2024-04-25T12:35:12.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.555+0000 7fec16000640 1 --2- 172.21.15.123:0/2472828246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c9f00 0x7fec200c8170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fec08009d90 tx=0x7fec0802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57b5a5914e67012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.555+0000 7fec1e000640 1 -- 172.21.15.123:0/2472828246 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08007740 con 0x7fec200c9f00 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.555+0000 7fec1e000640 1 -- 172.21.15.123:0/2472828246 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec080078e0 con 0x7fec200c9f00 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.555+0000 7fec1e000640 1 -- 172.21.15.123:0/2472828246 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08007c90 con 0x7fec200c9f00 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 -- 172.21.15.123:0/2472828246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c9f00 msgr2=0x7fec200c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 --2- 172.21.15.123:0/2472828246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c9f00 0x7fec200c8170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fec08009d90 tx=0x7fec0802a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 -- 172.21.15.123:0/2472828246 shutdown_connections 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 --2- 172.21.15.123:0/2472828246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200730e0 0x7fec200734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 --2- 172.21.15.123:0/2472828246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c9f00 0x7fec200c8170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 -- 172.21.15.123:0/2472828246 >> 172.21.15.123:0/2472828246 conn(0x7fec2006f440 msgr2=0x7fec2006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 -- 172.21.15.123:0/2472828246 shutdown_connections 2024-04-25T12:35:12.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.556+0000 7fec25c00640 1 -- 172.21.15.123:0/2472828246 wait complete. 2024-04-25T12:35:12.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.559+0000 7fec25c00640 1 Processor -- start 2024-04-25T12:35:12.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.559+0000 7fec25c00640 1 -- start start 2024-04-25T12:35:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.560+0000 7fec25c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200730e0 0x7fec201f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.560+0000 7fec25c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200c9f00 0x7fec201e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.560+0000 7fec25c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec20154650 0x7fec2014a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.560+0000 7fec25c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec2015e2f0 con 0x7fec20154650 2024-04-25T12:35:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.560+0000 7fec25c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec2015e010 con 0x7fec200730e0 2024-04-25T12:35:12.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.560+0000 7fec25c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fec20154a60 con 0x7fec200c9f00 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec20154650 0x7fec2014a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec20154650 0x7fec2014a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52582/0 (socket says 172.21.15.123:52582) 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 -- 172.21.15.123:0/2234667708 learned_addr learned my addr 172.21.15.123:0/2234667708 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200c9f00 0x7fec201e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1ea00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200730e0 0x7fec201f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200c9f00 msgr2=0x7fec201e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec200c9f00 0x7fec201e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200730e0 msgr2=0x7fec201f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200730e0 0x7fec201f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec201f63b0 con 0x7fec20154650 2024-04-25T12:35:12.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.561+0000 7fec1f400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec20154650 0x7fec2014a0a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e820 tx=0x7fec1800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.562+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18017070 con 0x7fec20154650 2024-04-25T12:35:12.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.562+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1800ee70 con 0x7fec20154650 2024-04-25T12:35:12.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.562+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800cb60 con 0x7fec20154650 2024-04-25T12:35:12.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.562+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2015d590 con 0x7fec20154650 2024-04-25T12:35:12.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.562+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201e8660 con 0x7fec20154650 2024-04-25T12:35:12.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.563+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fec20076700 con 0x7fec20154650 2024-04-25T12:35:12.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.564+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec1801a070 con 0x7fec20154650 2024-04-25T12:35:12.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.565+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febfc0744e0 0x7febfc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.566+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fec180956b0 con 0x7fec20154650 2024-04-25T12:35:12.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.566+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7fec180959d0 con 0x7fec20154650 2024-04-25T12:35:12.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.566+0000 7fec1ea00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febfc0744e0 0x7febfc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.566+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c4420 0x7fec2014a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.566+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec200744e0 con 0x7fec200c4420 2024-04-25T12:35:12.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.567+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c4420 0x7fec2014a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.567+0000 7fec1ea00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febfc0744e0 0x7febfc0769f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fec10005880 tx=0x7fec10009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.567+0000 7fec16000640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7fec20073c20 con 0x7fec200c4420 2024-04-25T12:35:12.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.567+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c4420 0x7fec2014a5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec20071a20 tx=0x7fec08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.568+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0803b820 con 0x7fec200c4420 2024-04-25T12:35:12.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.568+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fec0803b820 con 0x7fec200c4420 2024-04-25T12:35:12.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.568+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c4420 msgr2=0x7fec2014a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.568+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c4420 0x7fec2014a5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec20071a20 tx=0x7fec08004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:12.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.568+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c4420 msgr2=0x7fec2014a5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:35:12.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.568+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec200c4420 0x7fec2014a5e0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec20071a20 tx=0x7fec08004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:12.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.575+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec201f6580 0x7fec201e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.575+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec201e8de0 con 0x7fec201f6580 2024-04-25T12:35:12.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.575+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec201f6580 0x7fec201e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.576+0000 7fec16000640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fec200c8960 con 0x7fec201f6580 2024-04-25T12:35:12.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.576+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec201f6580 0x7fec201e88a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec08000f80 tx=0x7fec08030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.576+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08049040 con 0x7fec201f6580 2024-04-25T12:35:12.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.578+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.0 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fec08004450 con 0x7fec201f6580 2024-04-25T12:35:12.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.578+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec201f6580 msgr2=0x7fec201e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.578+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec201f6580 0x7fec201e88a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec08000f80 tx=0x7fec08030000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.578+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec201f6580 msgr2=0x7fec201e88a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:35:12.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.578+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec201f6580 0x7fec201e88a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.578 INFO:teuthology.orchestra.run.smithi123.stdout:mon.a: {} 2024-04-25T12:35:12.579 INFO:teuthology.orchestra.run.smithi123.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T12:35:12.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.578+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200c7fd0 0x7fec201f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.578+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec2015d2b0 con 0x7fec200c7fd0 2024-04-25T12:35:12.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.579+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200c7fd0 0x7fec201f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.579+0000 7fec16000640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7fec2006dbf0 con 0x7fec200c7fd0 2024-04-25T12:35:12.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.579+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200c7fd0 0x7fec201f2820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fec08004a80 tx=0x7fec08040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.580+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08052840 con 0x7fec200c7fd0 2024-04-25T12:35:12.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.581+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.1 v2:172.21.15.153:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fec0804be50 con 0x7fec200c7fd0 2024-04-25T12:35:12.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.581+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200c7fd0 msgr2=0x7fec201f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.581+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200c7fd0 0x7fec201f2820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fec08004a80 tx=0x7fec08040000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.581+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200c7fd0 msgr2=0x7fec201f2820 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:35:12.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.581+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec200c7fd0 0x7fec201f2820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:12.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.581+0000 7fec1f400640 1 -- 172.21.15.123:0/2234667708 reap_dead start 2024-04-25T12:35:12.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.587+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec201f5e70 0x7fec201e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.587+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec2015bbc0 con 0x7fec201f5e70 2024-04-25T12:35:12.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.588+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec201f5e70 0x7fec201e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.588+0000 7fec16000640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fec201e6180 con 0x7fec201f5e70 2024-04-25T12:35:12.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.588+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec201f5e70 0x7fec201e5c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec0802a730 tx=0x7fec0802f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.589+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08067070 con 0x7fec201f5e70 2024-04-25T12:35:12.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.590+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.1 v2:172.21.15.153:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fec08063820 con 0x7fec201f5e70 2024-04-25T12:35:12.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.590+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec201f5e70 msgr2=0x7fec201e5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.590+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec201f5e70 0x7fec201e5c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec0802a730 tx=0x7fec0802f000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.590+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec201f5e70 msgr2=0x7fec201e5c40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:35:12.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.590+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec201f5e70 0x7fec201e5c40 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec0802a730 tx=0x7fec0802f000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:mon.b: {} 2024-04-25T12:35:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T12:35:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.591+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec201f2d60 0x7fec201f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.591+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fec201f36b0 con 0x7fec201f2d60 2024-04-25T12:35:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.591+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec201f2d60 0x7fec201f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.591+0000 7fec16000640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fec201f9100 con 0x7fec201f2d60 2024-04-25T12:35:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.592+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec201f2d60 0x7fec201f3170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec080049d0 tx=0x7fec08040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.592+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08052e60 con 0x7fec201f2d60 2024-04-25T12:35:12.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.592+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.2 v2:172.21.15.153:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fec08070070 con 0x7fec201f2d60 2024-04-25T12:35:12.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.592+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec201f2d60 msgr2=0x7fec201f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.592+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec201f2d60 0x7fec201f3170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec080049d0 tx=0x7fec08040000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.592+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec201f2d60 msgr2=0x7fec201f3170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:35:12.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.592+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec201f2d60 0x7fec201f3170 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec080049d0 tx=0x7fec08040000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.599+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec20157dd0 0x7fec201581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.599+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fec201f36b0 con 0x7fec20157dd0 2024-04-25T12:35:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.599+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec20157dd0 0x7fec201581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.599+0000 7fec16000640 1 -- 172.21.15.123:0/2234667708 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fec20207170 con 0x7fec20157dd0 2024-04-25T12:35:12.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.600+0000 7fec16000640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec20157dd0 0x7fec201581e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec08009d90 tx=0x7fec08067000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:12.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.600+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0804bbf0 con 0x7fec20157dd0 2024-04-25T12:35:12.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 <== mon.2 v2:172.21.15.153:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fec0807a230 con 0x7fec20157dd0 2024-04-25T12:35:12.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec20157dd0 msgr2=0x7fec201581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec20157dd0 0x7fec201581e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec08009d90 tx=0x7fec08067000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec17400640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec20157dd0 msgr2=0x7fec201581e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:35:12.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec17400640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fec20157dd0 0x7fec201581e0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec08009d90 tx=0x7fec08067000 comp rx=0 tx=0).stop 2024-04-25T12:35:12.601 INFO:teuthology.orchestra.run.smithi123.stdout:mon.c: {} 2024-04-25T12:35:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T12:35:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febfc0744e0 msgr2=0x7febfc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febfc0744e0 0x7febfc0769f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fec10005880 tx=0x7fec10009040 comp rx=0 tx=0).stop 2024-04-25T12:35:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec20154650 msgr2=0x7fec2014a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec25c00640 1 --2- 172.21.15.123:0/2234667708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec20154650 0x7fec2014a0a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e820 tx=0x7fec1800ec20 comp rx=0 tx=0).stop 2024-04-25T12:35:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec1f400640 1 -- 172.21.15.123:0/2234667708 reap_dead start 2024-04-25T12:35:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 shutdown_connections 2024-04-25T12:35:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.602+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 >> 172.21.15.123:0/2234667708 conn(0x7fec2006f440 msgr2=0x7fec200c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.603+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 shutdown_connections 2024-04-25T12:35:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:12.603+0000 7fec25c00640 1 -- 172.21.15.123:0/2234667708 wait complete. 2024-04-25T12:35:12.629 DEBUG:teuthology.orchestra.run.smithi123:> type ceph-objectstore-tool 2024-04-25T12:35:12.674 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-25T12:35:12.675 DEBUG:teuthology.orchestra.run.smithi153:> type ceph-objectstore-tool 2024-04-25T12:35:12.692 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-25T12:35:12.692 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T12:35:12.705 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T12:35:12.706 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T12:35:12.706 INFO:tasks.workunit:timeout=3h 2024-04-25T12:35:12.706 INFO:tasks.workunit:cleanup=True 2024-04-25T12:35:12.706 DEBUG:teuthology.orchestra.run.smithi123:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:12.706 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-25T12:35:12.706 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:35:12.707 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:35:12.707 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-25T12:35:12.707 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-25T12:35:12.707 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-25T12:35:12.708 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-25T12:35:12.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:12.708 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-25T12:35:12.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:12.708 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-25T12:35:12.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:35:12.717 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:12.718+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:12.772 INFO:teuthology.orchestra.run.smithi123.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:12.772 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T12:35:12.773 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-04-25T12:35:12.773 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T12:35:12.773 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T12:35:12.773 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T12:35:12.773 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 12:34:18.943053883 +0000 2024-04-25T12:35:12.773 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 12:34:29.531099926 +0000 2024-04-25T12:35:12.773 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 12:34:18.943053883 +0000 2024-04-25T12:35:12.773 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T12:35:12.773 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T12:35:12.809 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:12.810+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:12.910 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:12.911+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:12.926 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T12:35:13.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.004+0000 7fb3d2600640 1 Processor -- start 2024-04-25T12:35:13.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.004+0000 7fb3d2600640 1 -- start start 2024-04-25T12:35:13.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.007+0000 7fb3d2600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3cc1568c0 0x7fb3cc0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.007+0000 7fb3d2600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.007+0000 7fb3d2600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3cc0cedf0 con 0x7fb3cc0cdca0 2024-04-25T12:35:13.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.007+0000 7fb3d2600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb3cc0cebd0 con 0x7fb3cc1568c0 2024-04-25T12:35:13.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.007+0000 7fb3d2600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb3cc0cf0d0 con 0x7fb3cc0c6800 2024-04-25T12:35:13.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.007+0000 7fb3d1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3cc1568c0 0x7fb3cc0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.008+0000 7fb3cbe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb3cc0cdca0 0x7fb3cc0ccf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43518/0 (socket says 172.21.15.123:43518) 2024-04-25T12:35:13.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.008+0000 7fb3cbe00640 1 -- 172.21.15.123:0/3650228467 learned_addr learned my addr 172.21.15.123:0/3650228467 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:13.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.008+0000 7fb3d1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3cc1568c0 0x7fb3cc0ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51924/0 (socket says 172.21.15.123:51924) 2024-04-25T12:35:13.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.008+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/3650228467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.008+0000 7fb3d1c00640 1 -- 172.21.15.123:0/3650228467 >> v2:172.21.15.123:3300/0 conn(0x7fb3cc1568c0 msgr2=0x7fb3cc0ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:13.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.008+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/3650228467 >> v2:172.21.15.123:3300/0 conn(0x7fb3cc1568c0 0x7fb3cc0ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.008+0000 7fb3d1c00640 1 -- 172.21.15.123:0/3650228467 >> v1:172.21.15.123:6789/0 conn(0x7fb3cc0cdca0 legacy=0x7fb3cc0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.009+0000 7fb3cb400640 1 -- 172.21.15.123:0/3650228467 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252176668 0 0) 0x7fb3cc0cedf0 con 0x7fb3cc0cdca0 2024-04-25T12:35:13.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.009+0000 7fb3d1c00640 1 -- 172.21.15.123:0/3650228467 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3cc0c7df0 con 0x7fb3cc0c6800 2024-04-25T12:35:13.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.010+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/3650228467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc15c3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009d90 tx=0x7fb3c402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f078a40d3ed61d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.011+0000 7fb3cb400640 1 -- 172.21.15.123:0/3650228467 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4007740 con 0x7fb3cc0c6800 2024-04-25T12:35:13.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.011+0000 7fb3cb400640 1 -- 172.21.15.123:0/3650228467 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c40078e0 con 0x7fb3cc0c6800 2024-04-25T12:35:13.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.011+0000 7fb3cb400640 1 -- 172.21.15.123:0/3650228467 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4007cb0 con 0x7fb3cc0c6800 2024-04-25T12:35:13.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 -- 172.21.15.123:0/3650228467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 msgr2=0x7fb3cc15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 --2- 172.21.15.123:0/3650228467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc15c3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009d90 tx=0x7fb3c402a660 comp rx=0 tx=0).stop 2024-04-25T12:35:13.011 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:13.012+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 -- 172.21.15.123:0/3650228467 shutdown_connections 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 --2- 172.21.15.123:0/3650228467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc15c3f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 --2- 172.21.15.123:0/3650228467 >> v2:172.21.15.123:3300/0 conn(0x7fb3cc1568c0 0x7fb3cc0ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 -- 172.21.15.123:0/3650228467 >> 172.21.15.123:0/3650228467 conn(0x7fb3cc06dcd0 msgr2=0x7fb3cc06e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 -- 172.21.15.123:0/3650228467 shutdown_connections 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.012+0000 7fb3d2600640 1 -- 172.21.15.123:0/3650228467 wait complete. 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:13.012 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:13.013 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:13.013 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:13.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.013+0000 7f3712c00640 1 Processor -- start 2024-04-25T12:35:13.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.013+0000 7f3712c00640 1 -- start start 2024-04-25T12:35:13.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.016+0000 7fb3d2600640 1 Processor -- start 2024-04-25T12:35:13.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.016+0000 7fb3d2600640 1 -- start start 2024-04-25T12:35:13.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.016+0000 7f3712c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f370c0c90d0 0x7f370c0c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.016+0000 7f3712c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f370c15a730 0x7f370c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.016+0000 7f3712c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f370c0c4810 con 0x7f370c0c4240 2024-04-25T12:35:13.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.016+0000 7f3712c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f370c15b080 con 0x7f370c15a730 2024-04-25T12:35:13.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.016+0000 7f3712c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f370c0ce380 con 0x7f370c0c90d0 2024-04-25T12:35:13.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.017+0000 7f370be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f370c0c90d0 0x7f370c0c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.017+0000 7f370be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f370c0c90d0 0x7f370c0c2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33018/0 (socket says 172.21.15.123:33018) 2024-04-25T12:35:13.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.017+0000 7f370be00640 1 -- 172.21.15.123:0/595008462 learned_addr learned my addr 172.21.15.123:0/595008462 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:13.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.017+0000 7f3710c00640 1 --2- 172.21.15.123:0/595008462 >> v2:172.21.15.123:3300/0 conn(0x7f370c15a730 0x7f370c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7f370aa00640 1 -- 172.21.15.123:0/595008462 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4213140185 0 0) 0x7f370c0c4810 con 0x7f370c0c4240 2024-04-25T12:35:13.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7f370be00640 1 -- 172.21.15.123:0/595008462 >> v2:172.21.15.123:3300/0 conn(0x7f370c15a730 msgr2=0x7f370c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:13.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7f370be00640 1 --2- 172.21.15.123:0/595008462 >> v2:172.21.15.123:3300/0 conn(0x7f370c15a730 0x7f370c15ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7f370be00640 1 -- 172.21.15.123:0/595008462 >> v1:172.21.15.123:6789/0 conn(0x7f370c0c4240 legacy=0x7f370c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7f370be00640 1 -- 172.21.15.123:0/595008462 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c0c83f0 con 0x7f370c0c90d0 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d2600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc0d0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3cc0cdca0 0x7fb3cc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d2600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3cc154f60 0x7fb3cc154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d2600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc0c83b0 con 0x7fb3cc0cdca0 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d2600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc0c8720 con 0x7fb3cc154f60 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d2600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb3cc0cd630 con 0x7fb3cc0c6800 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc0d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc0d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37426/0 (socket says 172.21.15.123:37426) 2024-04-25T12:35:13.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d1200640 1 -- 172.21.15.123:0/1821441158 learned_addr learned my addr 172.21.15.123:0/1821441158 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:13.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3cc0cdca0 0x7fb3cc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.018+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3cc154f60 0x7fb3cc154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7f370be00640 1 --2- 172.21.15.123:0/595008462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c0c90d0 0x7f370c0c2870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009870 tx=0x7f36fc02a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27dd1cffabfe0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 msgr2=0x7fb3cc0d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3cc0c6800 0x7fb3cc0d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3cc0cdca0 msgr2=0x7fb3cc14e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3cc0cdca0 0x7fb3cc14e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3cc14ee10 con 0x7fb3cc154f60 2024-04-25T12:35:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3cc0cdca0 0x7fb3cc14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.019+0000 7f370aa00640 1 -- 172.21.15.123:0/595008462 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc037070 con 0x7f370c0c90d0 2024-04-25T12:35:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.020+0000 7f370aa00640 1 -- 172.21.15.123:0/595008462 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36fc032440 con 0x7f370c0c90d0 2024-04-25T12:35:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.020+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3cc154f60 0x7fb3cc154650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4031000 tx=0x7fb3c4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.020+0000 7f370aa00640 1 -- 172.21.15.123:0/595008462 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc032810 con 0x7f370c0c90d0 2024-04-25T12:35:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.020+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c403f070 con 0x7fb3cc154f60 2024-04-25T12:35:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.020+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c4031bc0 con 0x7fb3cc154f60 2024-04-25T12:35:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3cc14e2f0 con 0x7fb3cc154f60 2024-04-25T12:35:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4044460 con 0x7fb3cc154f60 2024-04-25T12:35:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3cc1e62d0 con 0x7fb3cc154f60 2024-04-25T12:35:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7f3712c00640 1 -- 172.21.15.123:0/595008462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c0c90d0 msgr2=0x7f370c0c2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7f3712c00640 1 --2- 172.21.15.123:0/595008462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c0c90d0 0x7f370c0c2870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009870 tx=0x7f36fc02a2a0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7f3712c00640 1 -- 172.21.15.123:0/595008462 shutdown_connections 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7f3712c00640 1 --2- 172.21.15.123:0/595008462 >> v2:172.21.15.123:3300/0 conn(0x7f370c15a730 0x7f370c15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7f3712c00640 1 --2- 172.21.15.123:0/595008462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c0c90d0 0x7f370c0c2870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.021+0000 7f3712c00640 1 -- 172.21.15.123:0/595008462 >> 172.21.15.123:0/595008462 conn(0x7f370c06c750 msgr2=0x7f370c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.022+0000 7f3712c00640 1 -- 172.21.15.123:0/595008462 shutdown_connections 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.022+0000 7f3712c00640 1 -- 172.21.15.123:0/595008462 wait complete. 2024-04-25T12:35:13.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.022+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb3cc076da0 con 0x7fb3cc154f60 2024-04-25T12:35:13.023 INFO:tasks.workunit.client.0.smithi123.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T12:35:13.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.025+0000 7f5f82200640 1 Processor -- start 2024-04-25T12:35:13.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.025+0000 7f5f82200640 1 -- start start 2024-04-25T12:35:13.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.025+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb3c4066a80 con 0x7fb3cc154f60 2024-04-25T12:35:13.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.025+0000 7f3712c00640 1 Processor -- start 2024-04-25T12:35:13.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.025+0000 7f3712c00640 1 -- start start 2024-04-25T12:35:13.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7fb3c8c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3ac074500 0x7fb3ac076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f3712c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f370c0c4240 0x7f370c14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7fb3c40b6c90 con 0x7fb3cc154f60 2024-04-25T12:35:13.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f3712c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370c0c90d0 0x7f370c158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f3712c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c15a730 0x7f370c1ec180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f3712c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f370c0c8c90 con 0x7f370c0c90d0 2024-04-25T12:35:13.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f3712c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f370c0c89b0 con 0x7f370c15a730 2024-04-25T12:35:13.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f3712c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f370c073db0 con 0x7f370c0c4240 2024-04-25T12:35:13.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v27) ==== 54+0+16 (secure 0 0 0) 0x7fb3c40b70a0 con 0x7fb3cc154f60 2024-04-25T12:35:13.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f370be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f370c0c4240 0x7f370c14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3ac074500 0x7fb3ac076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f370be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f370c0c4240 0x7f370c14e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37432/0 (socket says 172.21.15.123:37432) 2024-04-25T12:35:13.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.027+0000 7f370be00640 1 -- 172.21.15.123:0/224732565 learned_addr learned my addr 172.21.15.123:0/224732565 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:13.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f370b400640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370c0c90d0 0x7f370c158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3cc1f57c0 0x7fb3cc1f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc0b69c0 con 0x7fb3cc1f57c0 2024-04-25T12:35:13.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f3710c00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c15a730 0x7f370c1ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f5f82200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c1606e0 0x7f5f7c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f5f82200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5f7c156670 0x7f5f7c15d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f5f82200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f7c0c72c0 0x7f5f7c0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3cc1f57c0 0x7fb3cc1f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f5f82200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7c160af0 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f5f82200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7c15d570 con 0x7f5f7c156670 2024-04-25T12:35:13.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f5f82200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7c156a80 con 0x7f5f7c1606e0 2024-04-25T12:35:13.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c1606e0 0x7f5f7c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f370b400640 1 -- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f370c0c4240 msgr2=0x7f370c14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f370b400640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f370c0c4240 0x7f370c14e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f370b400640 1 -- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c15a730 msgr2=0x7f370c1ec180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.028+0000 7f370b400640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c15a730 0x7f370c1ec180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f370b400640 1 -- 172.21.15.123:0/224732565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c158cd0 con 0x7f370c0c90d0 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c1606e0 0x7f5f7c06d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37434/0 (socket says 172.21.15.123:37434) 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 -- 172.21.15.123:0/1042470060 learned_addr learned my addr 172.21.15.123:0/1042470060 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7aa00640 1 --2- 172.21.15.123:0/1042470060 >> v2:172.21.15.153:3300/0 conn(0x7f5f7c156670 0x7f5f7c15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3ac074500 0x7fb3ac076a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009b60 tx=0x7fb3c0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7be00640 1 --2- 172.21.15.123:0/1042470060 >> v2:172.21.15.123:3300/0 conn(0x7f5f7c0c72c0 0x7f5f7c0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f370b400640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370c0c90d0 0x7f370c158790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f370c0c7ee0 tx=0x7f36f800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 -- 172.21.15.123:0/1042470060 >> v2:172.21.15.153:3300/0 conn(0x7f5f7c156670 msgr2=0x7f5f7c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 --2- 172.21.15.123:0/1042470060 >> v2:172.21.15.153:3300/0 conn(0x7f5f7c156670 0x7f5f7c15d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 -- 172.21.15.123:0/1042470060 >> v2:172.21.15.123:3300/0 conn(0x7f5f7c0c72c0 msgr2=0x7f5f7c0c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 --2- 172.21.15.123:0/1042470060 >> v2:172.21.15.123:3300/0 conn(0x7f5f7c0c72c0 0x7f5f7c0c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7b400640 1 -- 172.21.15.123:0/1042470060 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c15fc40 con 0x7f5f7c1606e0 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.029+0000 7f5f7be00640 1 --2- 172.21.15.123:0/1042470060 >> v2:172.21.15.123:3300/0 conn(0x7f5f7c0c72c0 0x7f5f7c0c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.030+0000 7f3703e00640 1 -- 172.21.15.123:0/224732565 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8007030 con 0x7f370c0c90d0 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.030+0000 7f3703e00640 1 -- 172.21.15.123:0/224732565 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f80071f0 con 0x7f370c0c90d0 2024-04-25T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.030+0000 7f3703e00640 1 -- 172.21.15.123:0/224732565 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80075e0 con 0x7f370c0c90d0 2024-04-25T12:35:13.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.030+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3cc1f57c0 0x7fb3cc1f5bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.030+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c1f7210 con 0x7f370c0c90d0 2024-04-25T12:35:13.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.030+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c1f77d0 con 0x7f370c0c90d0 2024-04-25T12:35:13.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.031+0000 7f5f7b400640 1 --2- 172.21.15.123:0/1042470060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c1606e0 0x7f5f7c06d440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0cf590 tx=0x7f5f6c02a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24790606f911fbbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.031+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc0b69c0 con 0x7fb3cc1f57c0 2024-04-25T12:35:13.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.031+0000 7f5f7a000640 1 -- 172.21.15.123:0/1042470060 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c037070 con 0x7f5f7c1606e0 2024-04-25T12:35:13.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.031+0000 7f5f7a000640 1 -- 172.21.15.123:0/1042470060 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6c032440 con 0x7f5f7c1606e0 2024-04-25T12:35:13.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.031+0000 7f5f7a000640 1 -- 172.21.15.123:0/1042470060 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c032810 con 0x7f5f7c1606e0 2024-04-25T12:35:13.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.032+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e4005290 con 0x7f370c0c90d0 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.033+0000 7f5f82200640 1 -- 172.21.15.123:0/1042470060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c1606e0 msgr2=0x7f5f7c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.033+0000 7f5f82200640 1 --2- 172.21.15.123:0/1042470060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c1606e0 0x7f5f7c06d440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0cf590 tx=0x7f5f6c02a260 comp rx=0 tx=0).stop 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.034+0000 7f5f82200640 1 -- 172.21.15.123:0/1042470060 shutdown_connections 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.034+0000 7f5f82200640 1 --2- 172.21.15.123:0/1042470060 >> v2:172.21.15.123:3300/0 conn(0x7f5f7c0c72c0 0x7f5f7c0c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.034+0000 7f5f82200640 1 --2- 172.21.15.123:0/1042470060 >> v2:172.21.15.153:3300/0 conn(0x7f5f7c156670 0x7f5f7c15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.034+0000 7f5f82200640 1 --2- 172.21.15.123:0/1042470060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c1606e0 0x7f5f7c06d440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.034+0000 7f5f82200640 1 -- 172.21.15.123:0/1042470060 >> 172.21.15.123:0/1042470060 conn(0x7f5f7c068a50 msgr2=0x7f5f7c0d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.034+0000 7f5f82200640 1 -- 172.21.15.123:0/1042470060 shutdown_connections 2024-04-25T12:35:13.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.034+0000 7f5f82200640 1 -- 172.21.15.123:0/1042470060 wait complete. 2024-04-25T12:35:13.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.035+0000 7f3703e00640 1 -- 172.21.15.123:0/224732565 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f36f801a020 con 0x7f370c0c90d0 2024-04-25T12:35:13.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.036+0000 7f3703e00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36ec0744e0 0x7f36ec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.037+0000 7f3703e00640 1 -- 172.21.15.123:0/224732565 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f36f8094b40 con 0x7f370c0c90d0 2024-04-25T12:35:13.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.037+0000 7f370be00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36ec0744e0 0x7f36ec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.038+0000 7f5f82200640 1 Processor -- start 2024-04-25T12:35:13.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.038+0000 7f5f82200640 1 -- start start 2024-04-25T12:35:13.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.038+0000 7f370be00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36ec0744e0 0x7f36ec0769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36fc02a2e0 tx=0x7f36fc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f82200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c0c72c0 0x7f5f7c1e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f82200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f7c156670 0x7f5f7c1e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f82200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f7c1606e0 0x7f5f7c15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f82200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c15fee0 con 0x7f5f7c156670 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f82200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c0ccc00 con 0x7f5f7c1606e0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f82200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7c1601c0 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f7b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c0c72c0 0x7f5f7c1e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f7be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f7c1606e0 0x7f5f7c15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f7b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c0c72c0 0x7f5f7c1e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37442/0 (socket says 172.21.15.123:37442) 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.040+0000 7f5f7b400640 1 -- 172.21.15.123:0/2262943059 learned_addr learned my addr 172.21.15.123:0/2262943059 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f7b400640 1 -- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f7c1606e0 msgr2=0x7f5f7c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f7b400640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f7c1606e0 0x7f5f7c15df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f7b400640 1 -- 172.21.15.123:0/2262943059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f7c156670 msgr2=0x7f5f7c1e8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f7aa00640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f7c156670 0x7f5f7c1e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f7b400640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f7c156670 0x7f5f7c1e8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f7b400640 1 -- 172.21.15.123:0/2262943059 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c1e8790 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f7b400640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c0c72c0 0x7f5f7c1e7d10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009960 tx=0x7f5f6c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f73400640 1 -- 172.21.15.123:0/2262943059 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c042020 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f73400640 1 -- 172.21.15.123:0/2262943059 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6c03d730 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.041+0000 7f5f73400640 1 -- 172.21.15.123:0/2262943059 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c037040 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.042+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c0cc060 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.042+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c0cbd80 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.043+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f7c076700 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:13.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.046+0000 7f5f73400640 1 -- 172.21.15.123:0/2262943059 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f6c045020 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.047+0000 7f5f73400640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f5c074500 0x7f5f5c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.047+0000 7f5f73400640 1 -- 172.21.15.123:0/2262943059 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f5f6c0b6430 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.047+0000 7f5f7aa00640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f5c074500 0x7f5f5c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.046+0000 7f3703e00640 1 -- 172.21.15.123:0/224732565 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36f805efe0 con 0x7f370c0c90d0 2024-04-25T12:35:13.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.048+0000 7f5f7aa00640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f5c074500 0x7f5f5c076a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c159e30 tx=0x7f5f68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.050+0000 7f5f73400640 1 -- 172.21.15.123:0/2262943059 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f6c04c030 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.057+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc0cda80 con 0x7fb3cc1f57c0 2024-04-25T12:35:13.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.060+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc0cda80 con 0x7fb3cc1f57c0 2024-04-25T12:35:13.059 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:13.059 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:13.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.060+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3cc14ac00 0x7fb3cc14b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.060+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc1f6620 con 0x7fb3cc14ac00 2024-04-25T12:35:13.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.060+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3cc14ac00 0x7fb3cc14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.061+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3cc14ac00 0x7fb3cc14b010 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.062+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc1f6620 con 0x7fb3cc14ac00 2024-04-25T12:35:13.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.078+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc0ce7c0 con 0x7fb3cc14ac00 2024-04-25T12:35:13.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.080+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc0ce7c0 con 0x7fb3cc14ac00 2024-04-25T12:35:13.080 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:13.080 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:13.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.081+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3cc062fe0 0x7fb3cc0633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.081+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc14ba60 con 0x7fb3cc062fe0 2024-04-25T12:35:13.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.082+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3cc062fe0 0x7fb3cc0633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.082+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3cc062fe0 0x7fb3cc0633f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.083+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc14ba60 con 0x7fb3cc062fe0 2024-04-25T12:35:13.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.099+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc1e6510 con 0x7fb3cc062fe0 2024-04-25T12:35:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.102+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc1e6510 con 0x7fb3cc062fe0 2024-04-25T12:35:13.102 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:13.102 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:13.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.103+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3cc063cf0 0x7fb3cc1e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.103+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc1e6720 con 0x7fb3cc063cf0 2024-04-25T12:35:13.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.103+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3cc063cf0 0x7fb3cc1e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.104+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3cc063cf0 0x7fb3cc1e4dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.106+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc1e6720 con 0x7fb3cc063cf0 2024-04-25T12:35:13.112 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:13.113+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:13.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.121+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc1e55e0 con 0x7fb3cc063cf0 2024-04-25T12:35:13.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.124+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc1e55e0 con 0x7fb3cc063cf0 2024-04-25T12:35:13.124 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:13.124 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:13.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.125+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3cc02cee0 0x7fb3cc02d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.125+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc02d830 con 0x7fb3cc02cee0 2024-04-25T12:35:13.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.125+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3cc02cee0 0x7fb3cc02d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.126+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3cc02cee0 0x7fb3cc02d2f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.127+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc02d830 con 0x7fb3cc02cee0 2024-04-25T12:35:13.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.142+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc02da70 con 0x7fb3cc02cee0 2024-04-25T12:35:13.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.144+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc02da70 con 0x7fb3cc02cee0 2024-04-25T12:35:13.144 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:13.144 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:13.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.144+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb3cc206d50 0x7fb3cc209170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.144+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc20c010 con 0x7fb3cc206d50 2024-04-25T12:35:13.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.145+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb3cc206d50 0x7fb3cc209170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.146+0000 7fb3cbe00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb3cc206d50 0x7fb3cc209170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.147+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc20c010 con 0x7fb3cc206d50 2024-04-25T12:35:13.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.162+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc2096b0 con 0x7fb3cc206d50 2024-04-25T12:35:13.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.165+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc2096b0 con 0x7fb3cc206d50 2024-04-25T12:35:13.164 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:13.164 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:13.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.165+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3cc209dd0 0x7fb3cc20a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.165+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc20f460 con 0x7fb3cc209dd0 2024-04-25T12:35:13.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.165+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3cc209dd0 0x7fb3cc20a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.166+0000 7fb3d1c00640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3cc209dd0 0x7fb3cc20a1e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.167+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc20f460 con 0x7fb3cc209dd0 2024-04-25T12:35:13.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.185+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc20e260 con 0x7fb3cc209dd0 2024-04-25T12:35:13.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.187+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc20e260 con 0x7fb3cc209dd0 2024-04-25T12:35:13.186 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:13.187 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:13.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.187+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3cc20e930 0x7fb3cc20eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.187+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb3cc2127d0 con 0x7fb3cc20e930 2024-04-25T12:35:13.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.188+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3cc20e930 0x7fb3cc20eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.188+0000 7fb3d1200640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3cc20e930 0x7fb3cc20eda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.189+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3cc2127d0 con 0x7fb3cc20e930 2024-04-25T12:35:13.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.206+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb3cc211650 con 0x7fb3cc20e930 2024-04-25T12:35:13.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3c8c00640 1 -- 172.21.15.123:0/1821441158 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3cc211650 con 0x7fb3cc20e930 2024-04-25T12:35:13.207 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3cc1f57c0 msgr2=0x7fb3cc1f5bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3cc1f57c0 0x7fb3cc1f5bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3cc14ac00 msgr2=0x7fb3cc14b010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3cc14ac00 0x7fb3cc14b010 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3cc062fe0 msgr2=0x7fb3cc0633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3cc062fe0 0x7fb3cc0633f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3cc063cf0 msgr2=0x7fb3cc1e4dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3cc063cf0 0x7fb3cc1e4dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3cc02cee0 msgr2=0x7fb3cc02d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3cc02cee0 0x7fb3cc02d2f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 reap_dead start 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb3cc206d50 msgr2=0x7fb3cc209170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb3cc206d50 0x7fb3cc209170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3cc209dd0 msgr2=0x7fb3cc20a1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3cc209dd0 0x7fb3cc20a1e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3cc20e930 msgr2=0x7fb3cc20eda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.208+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3cc20e930 0x7fb3cc20eda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 reap_dead start 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3ac074500 msgr2=0x7fb3ac076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3ac074500 0x7fb3ac076a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009b60 tx=0x7fb3c0009190 comp rx=0 tx=0).stop 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3cc154f60 msgr2=0x7fb3cc154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 --2- 172.21.15.123:0/1821441158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3cc154f60 0x7fb3cc154650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4031000 tx=0x7fb3c4004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:13.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 reap_dead start 2024-04-25T12:35:13.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 reap_dead start 2024-04-25T12:35:13.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d1c00640 1 -- 172.21.15.123:0/1821441158 reap_dead start 2024-04-25T12:35:13.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 shutdown_connections 2024-04-25T12:35:13.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 >> 172.21.15.123:0/1821441158 conn(0x7fb3cc06dcd0 msgr2=0x7fb3cc0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 shutdown_connections 2024-04-25T12:35:13.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.209+0000 7fb3d2600640 1 -- 172.21.15.123:0/1821441158 wait complete. 2024-04-25T12:35:13.213 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:13.213+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:13.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.270+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f36e4005580 con 0x7f370c0c90d0 2024-04-25T12:35:13.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.292+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f5f7c0cd160 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.295+0000 7f3703e00640 1 -- 172.21.15.123:0/224732565 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v28) ==== 87+0+0 (secure 0 0 0) 0x7f36f8063e20 con 0x7f370c0c90d0 2024-04-25T12:35:13.295 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36ec0744e0 msgr2=0x7f36ec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36ec0744e0 0x7f36ec0769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36fc02a2e0 tx=0x7f36fc02e000 comp rx=0 tx=0).stop 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370c0c90d0 msgr2=0x7f370c158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370c0c90d0 0x7f370c158790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f370c0c7ee0 tx=0x7f36f800c870 comp rx=0 tx=0).stop 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 shutdown_connections 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36ec0744e0 0x7f36ec0769f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f370c15a730 0x7f370c1ec180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370c0c90d0 0x7f370c158790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 --2- 172.21.15.123:0/224732565 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f370c0c4240 0x7f370c14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 >> 172.21.15.123:0/224732565 conn(0x7f370c06c750 msgr2=0x7f370c075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 shutdown_connections 2024-04-25T12:35:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.298+0000 7f3712c00640 1 -- 172.21.15.123:0/224732565 wait complete. 2024-04-25T12:35:13.304 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:13.304 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:13.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.306+0000 7f5f73400640 1 -- 172.21.15.123:0/2262943059 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v28) ==== 81+0+37770 (secure 0 0 0) 0x7f5f6c0808c0 con 0x7f5f7c0c72c0 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "epoch": 28, 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "created": "2024-04-25T12:33:54.601604+0000", 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "modified": "2024-04-25T12:35:13.288064+0000", 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "last_up_change": "2024-04-25T12:34:07.013466+0000", 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "last_in_change": "2024-04-25T12:33:58.290536+0000", 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_num": 5801984, 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_set": [ 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "noscrub", 2024-04-25T12:35:13.306 INFO:teuthology.orchestra.run.smithi123.stdout: "pglog_hardlimit", 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "purged_snapdirs", 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_deletes", 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "sortbitwise" 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_version": 6, 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_snapshot": "", 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_max": 4, 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd": 8, 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "require_min_compat_client": "luminous", 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "min_compat_client": "luminous", 2024-04-25T12:35:13.307 INFO:teuthology.orchestra.run.smithi123.stdout: "require_osd_release": "squid", 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_crimson": false, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "pools": [ 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 1, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": ".mgr", 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:06.827674+0000", 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:13.308 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 1, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 1, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 1, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 1, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 1, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:13.309 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "17", 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:13.310 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:13.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_max": 32, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 1 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.312 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": {} 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 8, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 8, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 0.25, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 2, 2024-04-25T12:35:13.313 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_metadata", 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:11.030660+0000", 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "20", 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:13.316 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:13.317 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_bias": 4, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 16, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_priority": 5 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": "cephfs" 2024-04-25T12:35:13.318 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 2.25, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 2.25, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2.25, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2.25, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 3, 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data", 2024-04-25T12:35:13.319 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:12.395253+0000", 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 64, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 64, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 64, 2024-04-25T12:35:13.320 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 64, 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 64, 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "21", 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:13.321 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:13.322 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:13.323 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.05000000074505806 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 1.5, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 1.5, 2024-04-25T12:35:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 1.5, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 1.5, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 4, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data_ec", 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:15.975923+0000", 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 5, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 3, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 4, 2024-04-25T12:35:13.325 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 3, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 1, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:13.326 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "25", 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:13.327 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:13.328 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 8192, 2024-04-25T12:35:13.329 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: "osds": [ 2024-04-25T12:35:13.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "2d9ee216-abfa-45db-a0ac-ef834981be6f", 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 26, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.331 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6816", 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6817", 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.332 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6818", 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6819", 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6822", 2024-04-25T12:35:13.333 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6823", 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6820", 2024-04-25T12:35:13.334 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6821", 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6817/1405705428", 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6819/1405705428", 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6823/1405705428", 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6821/1405705428", 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.335 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "a60a3a6f-376a-4fb2-baf1-e13fc615f6e5", 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.336 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6800", 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6801", 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6802", 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6803", 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.338 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6806", 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6807", 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6804", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6805", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6801/1234958824", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6803/1234958824", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6807/1234958824", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6805/1234958824", 2024-04-25T12:35:13.340 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "cfb7cbbd-82cd-46fe-b89b-15775df1fe93", 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:13.341 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6808", 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6809", 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.342 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6810", 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6811", 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6814", 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6815", 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.344 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6812", 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6813", 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6809/2222173849", 2024-04-25T12:35:13.345 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6811/2222173849", 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6815/2222173849", 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6813/2222173849", 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9e8ccd97-ab68-4f03-bbb3-5820eccf4568", 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.346 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6824", 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6825", 2024-04-25T12:35:13.347 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6826", 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6827", 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6830", 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6831", 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6828", 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6829", 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.350 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6825/1491071423", 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6827/1491071423", 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6831/1491071423", 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6829/1491071423", 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "7d5bfd6e-173a-413d-abdb-df6d82c941d3", 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.351 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6802", 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.352 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6803", 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6804", 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6805", 2024-04-25T12:35:13.353 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6808", 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6809", 2024-04-25T12:35:13.354 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6806", 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6807", 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:13.355 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6803/3135982807", 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6805/3135982807", 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6809/3135982807", 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6807/3135982807", 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:35:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9141fa91-c547-4711-910a-9ef123017fb5", 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.357 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6826", 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6827", 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6828", 2024-04-25T12:35:13.358 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6829", 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6832", 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6833", 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6830", 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6831", 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6827/2327013656", 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6829/2327013656", 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6833/2327013656", 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6831/2327013656", 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "66fbc935-086e-4b03-aa03-96e0bc8b1d27", 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6810", 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6811", 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6812", 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6813", 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6816", 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6817", 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6814", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6815", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6811/3626282289", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6813/3626282289", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6817/3626282289", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6815/3626282289", 2024-04-25T12:35:13.366 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "de59a619-b84a-4623-ac89-447178815d65", 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:13.367 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 26, 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6818", 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6819", 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.368 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6820", 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6821", 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.369 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6824", 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6825", 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6822", 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6823", 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6819/3086604284", 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6821/3086604284", 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6825/3086604284", 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6823/3086604284", 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_xinfo": [ 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.805055+0000", 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.746266+0000", 2024-04-25T12:35:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.706883+0000", 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.374 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.826277+0000", 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.375 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.402228+0000", 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.536018+0000", 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:35:13.376 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.375016+0000", 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:13.377 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.454835+0000", 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap": [], 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_items": [ 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "3.15", 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 0, 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 7 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "4.d", 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 3, 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 0 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_primaries": [], 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_temp": [], 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_temp": [], 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklist": {}, 2024-04-25T12:35:13.379 INFO:teuthology.orchestra.run.smithi123.stdout: "range_blocklist": {}, 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profiles": { 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_data_ec": { 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-device-class": "", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-num-failure-domains": "0", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-osds-per-failure-domain": "0", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-root": "default", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "jerasure-per-chunk-alignment": "false", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "2", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van", 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "w": "8" 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "default": { 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "1", 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van" 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps_queue": [], 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "new_removed_snaps": [], 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "new_purged_snaps": [], 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_node_flags": {}, 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "device_class_flags": {}, 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode": { 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_enabled": false, 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_bucket_count": 0, 2024-04-25T12:35:13.381 INFO:teuthology.orchestra.run.smithi123.stdout: "degraded_stretch_mode": 0, 2024-04-25T12:35:13.382 INFO:teuthology.orchestra.run.smithi123.stdout: "recovering_stretch_mode": 0, 2024-04-25T12:35:13.382 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_bucket": 0 2024-04-25T12:35:13.382 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:13.382 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:13.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f5c074500 msgr2=0x7f5f5c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f5c074500 0x7f5f5c076a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c159e30 tx=0x7f5f68008040 comp rx=0 tx=0).stop 2024-04-25T12:35:13.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c0c72c0 msgr2=0x7f5f7c1e7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c0c72c0 0x7f5f7c1e7d10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009960 tx=0x7f5f6c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 shutdown_connections 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f5c074500 0x7f5f5c076a10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f7c1606e0 0x7f5f7c15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f7c156670 0x7f5f7c1e8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.308+0000 7f5f82200640 1 --2- 172.21.15.123:0/2262943059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f7c0c72c0 0x7f5f7c1e7d10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.309+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 >> 172.21.15.123:0/2262943059 conn(0x7f5f7c068a50 msgr2=0x7f5f7c0c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.309+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 shutdown_connections 2024-04-25T12:35:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.309+0000 7f5f82200640 1 -- 172.21.15.123:0/2262943059 wait complete. 2024-04-25T12:35:13.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:13.387 INFO:tasks.thrashosds.thrasher:j is {'epoch': 28, 'fsid': '2239057a-dc25-4ee3-8341-8c80e403f296', 'created': '2024-04-25T12:33:54.601604+0000', 'modified': '2024-04-25T12:35:13.288064+0000', 'last_up_change': '2024-04-25T12:34:07.013466+0000', 'last_in_change': '2024-04-25T12:33:58.290536+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': '2d9ee216-abfa-45db-a0ac-ef834981be6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 1405705428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 1405705428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 1405705428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 1405705428}]}, 'public_addr': '172.21.15.123:6817/1405705428', 'cluster_addr': '172.21.15.123:6819/1405705428', 'heartbeat_back_addr': '172.21.15.123:6823/1405705428', 'heartbeat_front_addr': '172.21.15.123:6821/1405705428', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a60a3a6f-376a-4fb2-baf1-e13fc615f6e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 1234958824}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 1234958824}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 1234958824}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 1234958824}]}, 'public_addr': '172.21.15.123:6801/1234958824', 'cluster_addr': '172.21.15.123:6803/1234958824', 'heartbeat_back_addr': '172.21.15.123:6807/1234958824', 'heartbeat_front_addr': '172.21.15.123:6805/1234958824', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'cfb7cbbd-82cd-46fe-b89b-15775df1fe93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 2222173849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 2222173849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 2222173849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 2222173849}]}, 'public_addr': '172.21.15.123:6809/2222173849', 'cluster_addr': '172.21.15.123:6811/2222173849', 'heartbeat_back_addr': '172.21.15.123:6815/2222173849', 'heartbeat_front_addr': '172.21.15.123:6813/2222173849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9e8ccd97-ab68-4f03-bbb3-5820eccf4568', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 1491071423}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 1491071423}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 1491071423}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 1491071423}]}, 'public_addr': '172.21.15.123:6825/1491071423', 'cluster_addr': '172.21.15.123:6827/1491071423', 'heartbeat_back_addr': '172.21.15.123:6831/1491071423', 'heartbeat_front_addr': '172.21.15.123:6829/1491071423', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d5bfd6e-173a-413d-abdb-df6d82c941d3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6802', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6803', 'nonce': 3135982807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6804', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6805', 'nonce': 3135982807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6808', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6809', 'nonce': 3135982807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6806', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6807', 'nonce': 3135982807}]}, 'public_addr': '172.21.15.153:6803/3135982807', 'cluster_addr': '172.21.15.153:6805/3135982807', 'heartbeat_back_addr': '172.21.15.153:6809/3135982807', 'heartbeat_front_addr': '172.21.15.153:6807/3135982807', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9141fa91-c547-4711-910a-9ef123017fb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6826', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6827', 'nonce': 2327013656}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6828', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6829', 'nonce': 2327013656}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 2327013656}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6830', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6831', 'nonce': 2327013656}]}, 'public_addr': '172.21.15.153:6827/2327013656', 'cluster_addr': '172.21.15.153:6829/2327013656', 'heartbeat_back_addr': '172.21.15.153:6833/2327013656', 'heartbeat_front_addr': '172.21.15.153:6831/2327013656', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '66fbc935-086e-4b03-aa03-96e0bc8b1d27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6810', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6811', 'nonce': 3626282289}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6812', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6813', 'nonce': 3626282289}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6816', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6817', 'nonce': 3626282289}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6814', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6815', 'nonce': 3626282289}]}, 'public_addr': '172.21.15.153:6811/3626282289', 'cluster_addr': '172.21.15.153:6813/3626282289', 'heartbeat_back_addr': '172.21.15.153:6817/3626282289', 'heartbeat_front_addr': '172.21.15.153:6815/3626282289', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'de59a619-b84a-4623-ac89-447178815d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6818', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6819', 'nonce': 3086604284}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6820', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6821', 'nonce': 3086604284}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6824', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6825', 'nonce': 3086604284}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6822', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6823', 'nonce': 3086604284}]}, 'public_addr': '172.21.15.153:6819/3086604284', 'cluster_addr': '172.21.15.153:6821/3086604284', 'heartbeat_back_addr': '172.21.15.153:6825/3086604284', 'heartbeat_front_addr': '172.21.15.153:6823/3086604284', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.805055+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.746266+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.706883+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.826277+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.402228+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.536018+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.375016+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.454835+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.15', 'mappings': [{'from': 0, 'to': 7}]}, {'pgid': '4.d', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-25T12:35:13.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:35:13.388 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:13.385+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:13.487 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:13.484+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:13.589 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:13.586+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:13.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.678+0000 7fe6e1600640 1 Processor -- start 2024-04-25T12:35:13.678 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:13.679 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.679+0000 7fe6e1600640 1 -- start start 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.680+0000 7fe6e1600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.680+0000 7fe6e1600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe6dc1567f0 0x7fe6dc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6e1600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6dc06e5a0 con 0x7fe6dc06fbd0 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6e1600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe6dc06cc20 con 0x7fe6dc1567f0 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6e1600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc06d990 con 0x7fe6dc0c3850 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6e0c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe6dc06fbd0 0x7fe6dc15c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43550/0 (socket says 172.21.15.123:43550) 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6e0c00640 1 -- 172.21.15.123:0/2319857895 learned_addr learned my addr 172.21.15.123:0/2319857895 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6dbe00640 1 --2- 172.21.15.123:0/2319857895 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6db400640 1 --2- 172.21.15.123:0/2319857895 >> v2:172.21.15.153:3300/0 conn(0x7fe6dc1567f0 0x7fe6dc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6daa00640 1 -- 172.21.15.123:0/2319857895 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2137953398 0 0) 0x7fe6dc06e5a0 con 0x7fe6dc06fbd0 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6daa00640 1 -- 172.21.15.123:0/2319857895 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6c0003520 con 0x7fe6dc06fbd0 2024-04-25T12:35:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6dbe00640 1 -- 172.21.15.123:0/2319857895 >> v2:172.21.15.153:3300/0 conn(0x7fe6dc1567f0 msgr2=0x7fe6dc06d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:13.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6dbe00640 1 --2- 172.21.15.123:0/2319857895 >> v2:172.21.15.153:3300/0 conn(0x7fe6dc1567f0 0x7fe6dc06d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6dbe00640 1 -- 172.21.15.123:0/2319857895 >> v1:172.21.15.123:6789/0 conn(0x7fe6dc06fbd0 legacy=0x7fe6dc15c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.681+0000 7fe6dbe00640 1 -- 172.21.15.123:0/2319857895 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc06dc70 con 0x7fe6dc0c3850 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.682+0000 7fe6dbe00640 1 --2- 172.21.15.123:0/2319857895 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc06c570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097a0 tx=0x7fe6c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c69198e7e7b937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.683+0000 7fe6daa00640 1 -- 172.21.15.123:0/2319857895 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4037070 con 0x7fe6dc0c3850 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.683+0000 7fe6daa00640 1 -- 172.21.15.123:0/2319857895 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c4032440 con 0x7fe6dc0c3850 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.683+0000 7fe6daa00640 1 -- 172.21.15.123:0/2319857895 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40327f0 con 0x7fe6dc0c3850 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.683+0000 7fe6e1600640 1 -- 172.21.15.123:0/2319857895 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 msgr2=0x7fe6dc06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.683+0000 7fe6e1600640 1 --2- 172.21.15.123:0/2319857895 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc06c570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097a0 tx=0x7fe6c402a180 comp rx=0 tx=0).stop 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.684+0000 7fe6e1600640 1 -- 172.21.15.123:0/2319857895 shutdown_connections 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.684+0000 7fe6e1600640 1 --2- 172.21.15.123:0/2319857895 >> v2:172.21.15.153:3300/0 conn(0x7fe6dc1567f0 0x7fe6dc06d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.684+0000 7fe6e1600640 1 --2- 172.21.15.123:0/2319857895 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc06c570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.684+0000 7fe6e1600640 1 -- 172.21.15.123:0/2319857895 >> 172.21.15.123:0/2319857895 conn(0x7fe6dc0ce710 msgr2=0x7fe6dc0ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.684+0000 7fe6e1600640 1 -- 172.21.15.123:0/2319857895 shutdown_connections 2024-04-25T12:35:13.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.684+0000 7fe6e1600640 1 -- 172.21.15.123:0/2319857895 wait complete. 2024-04-25T12:35:13.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.686+0000 7fe6e1600640 1 Processor -- start 2024-04-25T12:35:13.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.686+0000 7fe6e1600640 1 -- start start 2024-04-25T12:35:13.686 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:13.686+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.687+0000 7fe6e1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6dc06fbd0 0x7fe6dc1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.687+0000 7fe6e1600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6e1600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6dc1567f0 0x7fe6dc1e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6e1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc15ce40 con 0x7fe6dc06fbd0 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6e1600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc06e230 con 0x7fe6dc1567f0 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6e1600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc15cfc0 con 0x7fe6dc0c3850 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6dc06fbd0 0x7fe6dc1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6db400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6e0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6dc1567f0 0x7fe6dc1e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6dc06fbd0 0x7fe6dc1f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51964/0 (socket says 172.21.15.123:51964) 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6db400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37466/0 (socket says 172.21.15.123:37466) 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6dbe00640 1 -- 172.21.15.123:0/4099394475 learned_addr learned my addr 172.21.15.123:0/4099394475 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:13.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6db400640 1 -- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6dc1567f0 msgr2=0x7fe6dc1e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6db400640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6dc1567f0 0x7fe6dc1e7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6db400640 1 -- 172.21.15.123:0/4099394475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6dc06fbd0 msgr2=0x7fe6dc1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6db400640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6dc06fbd0 0x7fe6dc1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6db400640 1 -- 172.21.15.123:0/4099394475 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc14bcb0 con 0x7fe6dc0c3850 2024-04-25T12:35:13.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.688+0000 7fe6dbe00640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6dc06fbd0 0x7fe6dc1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:13.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.689+0000 7fe6db400640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc0cdc20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00ebf0 tx=0x7fe6cc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.689+0000 7fe6d3e00640 1 -- 172.21.15.123:0/4099394475 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc018020 con 0x7fe6dc0c3850 2024-04-25T12:35:13.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.689+0000 7fe6d3e00640 1 -- 172.21.15.123:0/4099394475 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6cc00cda0 con 0x7fe6dc0c3850 2024-04-25T12:35:13.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.689+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc14bf90 con 0x7fe6dc0c3850 2024-04-25T12:35:13.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.689+0000 7fe6d3e00640 1 -- 172.21.15.123:0/4099394475 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0146c0 con 0x7fe6dc0c3850 2024-04-25T12:35:13.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.689+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1f6020 con 0x7fe6dc0c3850 2024-04-25T12:35:13.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.690+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc076da0 con 0x7fe6dc0c3850 2024-04-25T12:35:13.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.693+0000 7fe6d3e00640 1 -- 172.21.15.123:0/4099394475 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe6cc005d10 con 0x7fe6dc0c3850 2024-04-25T12:35:13.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.693+0000 7fe6d3e00640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe6b80744b0 0x7fe6b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:13.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.693+0000 7fe6d3e00640 1 -- 172.21.15.123:0/4099394475 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fe6cc096130 con 0x7fe6dc0c3850 2024-04-25T12:35:13.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.694+0000 7fe6dbe00640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe6b80744b0 0x7fe6b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:13.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.694+0000 7fe6dbe00640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe6b80744b0 0x7fe6b80769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097a0 tx=0x7fe6c402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:13.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.698+0000 7fe6d3e00640 1 -- 172.21.15.123:0/4099394475 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6cc022030 con 0x7fe6dc0c3850 2024-04-25T12:35:13.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:13.786 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:13.786+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:13.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.888+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6dc0c9fa0 con 0x7fe6b80744b0 2024-04-25T12:35:13.889 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:13.890+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:13.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.897+0000 7fe6d3e00640 1 -- 172.21.15.123:0/4099394475 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332682 (secure 0 0 0) 0x7fe6dc0c9fa0 con 0x7fe6b80744b0 2024-04-25T12:35:13.898 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:13.906 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:35:13.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe6b80744b0 msgr2=0x7fe6b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe6b80744b0 0x7fe6b80769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097a0 tx=0x7fe6c402e000 comp rx=0 tx=0).stop 2024-04-25T12:35:13.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 msgr2=0x7fe6dc0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:13.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc0cdc20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00ebf0 tx=0x7fe6cc00c490 comp rx=0 tx=0).stop 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 shutdown_connections 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe6b80744b0 0x7fe6b80769c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6dc1567f0 0x7fe6dc1e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe6dc0c3850 0x7fe6dc0cdc20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 --2- 172.21.15.123:0/4099394475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6dc06fbd0 0x7fe6dc1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.902+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 >> 172.21.15.123:0/4099394475 conn(0x7fe6dc0ce710 msgr2=0x7fe6dc153db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.903+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 shutdown_connections 2024-04-25T12:35:13.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:13.903+0000 7fe6e1600640 1 -- 172.21.15.123:0/4099394475 wait complete. 2024-04-25T12:35:13.924 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-04-25T12:35:13.302132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598078,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":232,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218664,"kb_used_data":2496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756056,"statfs":{"total":773094113280,"available":772870201344,"internally_reserved":0,"allocated":2555904,"data_stored":2128852,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.497537"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694930+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:35:00.694930+0000","last_peered":"2024-04-25T12:35:00.694930+0000","last_clean":"2024-04-25T12:35:00.694930+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:35:00.694930+0000","last_undegraded":"2024-04-25T12:35:00.694930+0000","last_fullsized":"2024-04-25T12:35:00.694930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488862+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:35:00.488862+0000","last_peered":"2024-04-25T12:35:00.488862+0000","last_clean":"2024-04-25T12:35:00.488862+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:35:00.488862+0000","last_undegraded":"2024-04-25T12:35:00.488862+0000","last_fullsized":"2024-04-25T12:35:00.488862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238136+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:35:00.238136+0000","last_peered":"2024-04-25T12:35:00.238136+0000","last_clean":"2024-04-25T12:35:00.238136+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:35:00.238136+0000","last_undegraded":"2024-04-25T12:35:00.238136+0000","last_fullsized":"2024-04-25T12:35:00.238136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488973+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:35:00.488973+0000","last_peered":"2024-04-25T12:35:00.488973+0000","last_clean":"2024-04-25T12:35:00.488973+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:35:00.488973+0000","last_undegraded":"2024-04-25T12:35:00.488973+0000","last_fullsized":"2024-04-25T12:35:00.488973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488971+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:35:00.488971+0000","last_peered":"2024-04-25T12:35:00.488971+0000","last_clean":"2024-04-25T12:35:00.488971+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:35:00.488971+0000","last_undegraded":"2024-04-25T12:35:00.488971+0000","last_fullsized":"2024-04-25T12:35:00.488971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697778+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:35:00.697778+0000","last_peered":"2024-04-25T12:35:00.697778+0000","last_clean":"2024-04-25T12:35:00.697778+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:35:00.697778+0000","last_undegraded":"2024-04-25T12:35:00.697778+0000","last_fullsized":"2024-04-25T12:35:00.697778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244935+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:35:00.244935+0000","last_peered":"2024-04-25T12:35:00.244935+0000","last_clean":"2024-04-25T12:35:00.244935+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:35:00.244935+0000","last_undegraded":"2024-04-25T12:35:00.244935+0000","last_fullsized":"2024-04-25T12:35:00.244935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697494+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:35:00.697494+0000","last_peered":"2024-04-25T12:35:00.697494+0000","last_clean":"2024-04-25T12:35:00.697494+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:35:00.697494+0000","last_undegraded":"2024-04-25T12:35:00.697494+0000","last_fullsized":"2024-04-25T12:35:00.697494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699179+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:35:00.699179+0000","last_peered":"2024-04-25T12:35:00.699179+0000","last_clean":"2024-04-25T12:35:00.699179+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:35:00.699179+0000","last_undegraded":"2024-04-25T12:35:00.699179+0000","last_fullsized":"2024-04-25T12:35:00.699179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245331+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:35:00.245331+0000","last_peered":"2024-04-25T12:35:00.245331+0000","last_clean":"2024-04-25T12:35:00.245331+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:35:00.245331+0000","last_undegraded":"2024-04-25T12:35:00.245331+0000","last_fullsized":"2024-04-25T12:35:00.245331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237412+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:35:00.237412+0000","last_peered":"2024-04-25T12:35:00.237412+0000","last_clean":"2024-04-25T12:35:00.237412+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:35:00.237412+0000","last_undegraded":"2024-04-25T12:35:00.237412+0000","last_fullsized":"2024-04-25T12:35:00.237412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697778+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:35:00.697778+0000","last_peered":"2024-04-25T12:35:00.697778+0000","last_clean":"2024-04-25T12:35:00.697778+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:35:00.697778+0000","last_undegraded":"2024-04-25T12:35:00.697778+0000","last_fullsized":"2024-04-25T12:35:00.697778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236722+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:35:00.236722+0000","last_peered":"2024-04-25T12:35:00.236722+0000","last_clean":"2024-04-25T12:35:00.236722+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:35:00.236722+0000","last_undegraded":"2024-04-25T12:35:00.236722+0000","last_fullsized":"2024-04-25T12:35:00.236722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237528+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:35:00.237528+0000","last_peered":"2024-04-25T12:35:00.237528+0000","last_clean":"2024-04-25T12:35:00.237528+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:35:00.237528+0000","last_undegraded":"2024-04-25T12:35:00.237528+0000","last_fullsized":"2024-04-25T12:35:00.237528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.706022+0000","last_change":"2024-04-25T12:35:00.703950+0000","last_active":"2024-04-25T12:35:00.706022+0000","last_peered":"2024-04-25T12:35:00.706022+0000","last_clean":"2024-04-25T12:35:00.706022+0000","last_became_active":"2024-04-25T12:35:00.702548+0000","last_became_peered":"2024-04-25T12:35:00.702548+0000","last_unstale":"2024-04-25T12:35:00.706022+0000","last_undegraded":"2024-04-25T12:35:00.706022+0000","last_fullsized":"2024-04-25T12:35:00.706022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238506+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:35:00.238506+0000","last_peered":"2024-04-25T12:35:00.238506+0000","last_clean":"2024-04-25T12:35:00.238506+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:35:00.238506+0000","last_undegraded":"2024-04-25T12:35:00.238506+0000","last_fullsized":"2024-04-25T12:35:00.238506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235664+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:35:00.235664+0000","last_peered":"2024-04-25T12:35:00.235664+0000","last_clean":"2024-04-25T12:35:00.235664+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:35:00.235664+0000","last_undegraded":"2024-04-25T12:35:00.235664+0000","last_fullsized":"2024-04-25T12:35:00.235664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244076+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:35:00.244076+0000","last_peered":"2024-04-25T12:35:00.244076+0000","last_clean":"2024-04-25T12:35:00.244076+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:35:00.244076+0000","last_undegraded":"2024-04-25T12:35:00.244076+0000","last_fullsized":"2024-04-25T12:35:00.244076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694168+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:35:00.694168+0000","last_peered":"2024-04-25T12:35:00.694168+0000","last_clean":"2024-04-25T12:35:00.694168+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:35:00.694168+0000","last_undegraded":"2024-04-25T12:35:00.694168+0000","last_fullsized":"2024-04-25T12:35:00.694168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694486+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:35:00.694486+0000","last_peered":"2024-04-25T12:35:00.694486+0000","last_clean":"2024-04-25T12:35:00.694486+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:35:00.694486+0000","last_undegraded":"2024-04-25T12:35:00.694486+0000","last_fullsized":"2024-04-25T12:35:00.694486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238547+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:35:00.238547+0000","last_peered":"2024-04-25T12:35:00.238547+0000","last_clean":"2024-04-25T12:35:00.238547+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:35:00.238547+0000","last_undegraded":"2024-04-25T12:35:00.238547+0000","last_fullsized":"2024-04-25T12:35:00.238547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236478+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:35:00.236478+0000","last_peered":"2024-04-25T12:35:00.236478+0000","last_clean":"2024-04-25T12:35:00.236478+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:35:00.236478+0000","last_undegraded":"2024-04-25T12:35:00.236478+0000","last_fullsized":"2024-04-25T12:35:00.236478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242241+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:35:00.242241+0000","last_peered":"2024-04-25T12:35:00.242241+0000","last_clean":"2024-04-25T12:35:00.242241+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:35:00.242241+0000","last_undegraded":"2024-04-25T12:35:00.242241+0000","last_fullsized":"2024-04-25T12:35:00.242241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700917+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:35:00.700917+0000","last_peered":"2024-04-25T12:35:00.700917+0000","last_clean":"2024-04-25T12:35:00.700917+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:35:00.700917+0000","last_undegraded":"2024-04-25T12:35:00.700917+0000","last_fullsized":"2024-04-25T12:35:00.700917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694705+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:35:00.694705+0000","last_peered":"2024-04-25T12:35:00.694705+0000","last_clean":"2024-04-25T12:35:00.694705+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:35:00.694705+0000","last_undegraded":"2024-04-25T12:35:00.694705+0000","last_fullsized":"2024-04-25T12:35:00.694705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238249+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:35:00.238249+0000","last_peered":"2024-04-25T12:35:00.238249+0000","last_clean":"2024-04-25T12:35:00.238249+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:35:00.238249+0000","last_undegraded":"2024-04-25T12:35:00.238249+0000","last_fullsized":"2024-04-25T12:35:00.238249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699021+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:35:00.699021+0000","last_peered":"2024-04-25T12:35:00.699021+0000","last_clean":"2024-04-25T12:35:00.699021+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:35:00.699021+0000","last_undegraded":"2024-04-25T12:35:00.699021+0000","last_fullsized":"2024-04-25T12:35:00.699021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694870+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:35:00.694870+0000","last_peered":"2024-04-25T12:35:00.694870+0000","last_clean":"2024-04-25T12:35:00.694870+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:35:00.694870+0000","last_undegraded":"2024-04-25T12:35:00.694870+0000","last_fullsized":"2024-04-25T12:35:00.694870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694849+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:35:00.694849+0000","last_peered":"2024-04-25T12:35:00.694849+0000","last_clean":"2024-04-25T12:35:00.694849+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:35:00.694849+0000","last_undegraded":"2024-04-25T12:35:00.694849+0000","last_fullsized":"2024-04-25T12:35:00.694849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694157+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:35:00.694157+0000","last_peered":"2024-04-25T12:35:00.694157+0000","last_clean":"2024-04-25T12:35:00.694157+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:35:00.694157+0000","last_undegraded":"2024-04-25T12:35:00.694157+0000","last_fullsized":"2024-04-25T12:35:00.694157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236113+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:35:00.236113+0000","last_peered":"2024-04-25T12:35:00.236113+0000","last_clean":"2024-04-25T12:35:00.236113+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:35:00.236113+0000","last_undegraded":"2024-04-25T12:35:00.236113+0000","last_fullsized":"2024-04-25T12:35:00.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237837+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:35:00.237837+0000","last_peered":"2024-04-25T12:35:00.237837+0000","last_clean":"2024-04-25T12:35:00.237837+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:35:00.237837+0000","last_undegraded":"2024-04-25T12:35:00.237837+0000","last_fullsized":"2024-04-25T12:35:00.237837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235604+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:35:00.235604+0000","last_peered":"2024-04-25T12:35:00.235604+0000","last_clean":"2024-04-25T12:35:00.235604+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:35:00.235604+0000","last_undegraded":"2024-04-25T12:35:00.235604+0000","last_fullsized":"2024-04-25T12:35:00.235604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699987+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:35:00.699987+0000","last_peered":"2024-04-25T12:35:00.699987+0000","last_clean":"2024-04-25T12:35:00.699987+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:35:00.699987+0000","last_undegraded":"2024-04-25T12:35:00.699987+0000","last_fullsized":"2024-04-25T12:35:00.699987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488578+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:35:00.488578+0000","last_peered":"2024-04-25T12:35:00.488578+0000","last_clean":"2024-04-25T12:35:00.488578+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:35:00.488578+0000","last_undegraded":"2024-04-25T12:35:00.488578+0000","last_fullsized":"2024-04-25T12:35:00.488578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235386+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:35:00.235386+0000","last_peered":"2024-04-25T12:35:00.235386+0000","last_clean":"2024-04-25T12:35:00.235386+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:35:00.235386+0000","last_undegraded":"2024-04-25T12:35:00.235386+0000","last_fullsized":"2024-04-25T12:35:00.235386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.243856+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:35:00.243856+0000","last_peered":"2024-04-25T12:35:00.243856+0000","last_clean":"2024-04-25T12:35:00.243856+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:35:00.243856+0000","last_undegraded":"2024-04-25T12:35:00.243856+0000","last_fullsized":"2024-04-25T12:35:00.243856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237018+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:35:00.237018+0000","last_peered":"2024-04-25T12:35:00.237018+0000","last_clean":"2024-04-25T12:35:00.237018+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:35:00.237018+0000","last_undegraded":"2024-04-25T12:35:00.237018+0000","last_fullsized":"2024-04-25T12:35:00.237018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235850+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:35:00.235850+0000","last_peered":"2024-04-25T12:35:00.235850+0000","last_clean":"2024-04-25T12:35:00.235850+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:35:00.235850+0000","last_undegraded":"2024-04-25T12:35:00.235850+0000","last_fullsized":"2024-04-25T12:35:00.235850+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488045+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:35:00.488045+0000","last_peered":"2024-04-25T12:35:00.488045+0000","last_clean":"2024-04-25T12:35:00.488045+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:35:00.488045+0000","last_undegraded":"2024-04-25T12:35:00.488045+0000","last_fullsized":"2024-04-25T12:35:00.488045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238322+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:35:00.238322+0000","last_peered":"2024-04-25T12:35:00.238322+0000","last_clean":"2024-04-25T12:35:00.238322+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:35:00.238322+0000","last_undegraded":"2024-04-25T12:35:00.238322+0000","last_fullsized":"2024-04-25T12:35:00.238322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236008+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:35:00.236008+0000","last_peered":"2024-04-25T12:35:00.236008+0000","last_clean":"2024-04-25T12:35:00.236008+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:35:00.236008+0000","last_undegraded":"2024-04-25T12:35:00.236008+0000","last_fullsized":"2024-04-25T12:35:00.236008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235351+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:35:00.235351+0000","last_peered":"2024-04-25T12:35:00.235351+0000","last_clean":"2024-04-25T12:35:00.235351+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:35:00.235351+0000","last_undegraded":"2024-04-25T12:35:00.235351+0000","last_fullsized":"2024-04-25T12:35:00.235351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236908+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:35:00.236908+0000","last_peered":"2024-04-25T12:35:00.236908+0000","last_clean":"2024-04-25T12:35:00.236908+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:35:00.236908+0000","last_undegraded":"2024-04-25T12:35:00.236908+0000","last_fullsized":"2024-04-25T12:35:00.236908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488596+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:35:00.488596+0000","last_peered":"2024-04-25T12:35:00.488596+0000","last_clean":"2024-04-25T12:35:00.488596+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:35:00.488596+0000","last_undegraded":"2024-04-25T12:35:00.488596+0000","last_fullsized":"2024-04-25T12:35:00.488596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694405+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:35:00.694405+0000","last_peered":"2024-04-25T12:35:00.694405+0000","last_clean":"2024-04-25T12:35:00.694405+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:35:00.694405+0000","last_undegraded":"2024-04-25T12:35:00.694405+0000","last_fullsized":"2024-04-25T12:35:00.694405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237670+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:35:00.237670+0000","last_peered":"2024-04-25T12:35:00.237670+0000","last_clean":"2024-04-25T12:35:00.237670+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:35:00.237670+0000","last_undegraded":"2024-04-25T12:35:00.237670+0000","last_fullsized":"2024-04-25T12:35:00.237670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697341+0000","last_change":"2024-04-25T12:34:08.560989+0000","last_active":"2024-04-25T12:35:00.697341+0000","last_peered":"2024-04-25T12:35:00.697341+0000","last_clean":"2024-04-25T12:35:00.697341+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:35:00.697341+0000","last_undegraded":"2024-04-25T12:35:00.697341+0000","last_fullsized":"2024-04-25T12:35:00.697341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:49.130809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237678+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:35:00.237678+0000","last_peered":"2024-04-25T12:35:00.237678+0000","last_clean":"2024-04-25T12:35:00.237678+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:35:00.237678+0000","last_undegraded":"2024-04-25T12:35:00.237678+0000","last_fullsized":"2024-04-25T12:35:00.237678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237549+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:35:00.237549+0000","last_peered":"2024-04-25T12:35:00.237549+0000","last_clean":"2024-04-25T12:35:00.237549+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:35:00.237549+0000","last_undegraded":"2024-04-25T12:35:00.237549+0000","last_fullsized":"2024-04-25T12:35:00.237549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236215+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:35:00.236215+0000","last_peered":"2024-04-25T12:35:00.236215+0000","last_clean":"2024-04-25T12:35:00.236215+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:35:00.236215+0000","last_undegraded":"2024-04-25T12:35:00.236215+0000","last_fullsized":"2024-04-25T12:35:00.236215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698145+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:35:00.698145+0000","last_peered":"2024-04-25T12:35:00.698145+0000","last_clean":"2024-04-25T12:35:00.698145+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:35:00.698145+0000","last_undegraded":"2024-04-25T12:35:00.698145+0000","last_fullsized":"2024-04-25T12:35:00.698145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245030+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:35:00.245030+0000","last_peered":"2024-04-25T12:35:00.245030+0000","last_clean":"2024-04-25T12:35:00.245030+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:35:00.245030+0000","last_undegraded":"2024-04-25T12:35:00.245030+0000","last_fullsized":"2024-04-25T12:35:00.245030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488705+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:35:00.488705+0000","last_peered":"2024-04-25T12:35:00.488705+0000","last_clean":"2024-04-25T12:35:00.488705+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:35:00.488705+0000","last_undegraded":"2024-04-25T12:35:00.488705+0000","last_fullsized":"2024-04-25T12:35:00.488705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488389+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:35:00.488389+0000","last_peered":"2024-04-25T12:35:00.488389+0000","last_clean":"2024-04-25T12:35:00.488389+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:35:00.488389+0000","last_undegraded":"2024-04-25T12:35:00.488389+0000","last_fullsized":"2024-04-25T12:35:00.488389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245496+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:35:00.245496+0000","last_peered":"2024-04-25T12:35:00.245496+0000","last_clean":"2024-04-25T12:35:00.245496+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:35:00.245496+0000","last_undegraded":"2024-04-25T12:35:00.245496+0000","last_fullsized":"2024-04-25T12:35:00.245496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694766+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:35:00.694766+0000","last_peered":"2024-04-25T12:35:00.694766+0000","last_clean":"2024-04-25T12:35:00.694766+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:35:00.694766+0000","last_undegraded":"2024-04-25T12:35:00.694766+0000","last_fullsized":"2024-04-25T12:35:00.694766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236118+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:35:00.236118+0000","last_peered":"2024-04-25T12:35:00.236118+0000","last_clean":"2024-04-25T12:35:00.236118+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:35:00.236118+0000","last_undegraded":"2024-04-25T12:35:00.236118+0000","last_fullsized":"2024-04-25T12:35:00.236118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.701276+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:35:00.701276+0000","last_peered":"2024-04-25T12:35:00.701276+0000","last_clean":"2024-04-25T12:35:00.701276+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:35:00.701276+0000","last_undegraded":"2024-04-25T12:35:00.701276+0000","last_fullsized":"2024-04-25T12:35:00.701276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245657+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:35:00.245657+0000","last_peered":"2024-04-25T12:35:00.245657+0000","last_clean":"2024-04-25T12:35:00.245657+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:35:00.245657+0000","last_undegraded":"2024-04-25T12:35:00.245657+0000","last_fullsized":"2024-04-25T12:35:00.245657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.697896+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:35:00.697896+0000","last_peered":"2024-04-25T12:35:00.697896+0000","last_clean":"2024-04-25T12:35:00.697896+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:35:00.697896+0000","last_undegraded":"2024-04-25T12:35:00.697896+0000","last_fullsized":"2024-04-25T12:35:00.697896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236512+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:35:00.236512+0000","last_peered":"2024-04-25T12:35:00.236512+0000","last_clean":"2024-04-25T12:35:00.236512+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:35:00.236512+0000","last_undegraded":"2024-04-25T12:35:00.236512+0000","last_fullsized":"2024-04-25T12:35:00.236512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694315+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:35:00.694315+0000","last_peered":"2024-04-25T12:35:00.694315+0000","last_clean":"2024-04-25T12:35:00.694315+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:35:00.694315+0000","last_undegraded":"2024-04-25T12:35:00.694315+0000","last_fullsized":"2024-04-25T12:35:00.694315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488781+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:35:00.488781+0000","last_peered":"2024-04-25T12:35:00.488781+0000","last_clean":"2024-04-25T12:35:00.488781+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:35:00.488781+0000","last_undegraded":"2024-04-25T12:35:00.488781+0000","last_fullsized":"2024-04-25T12:35:00.488781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.696800+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:35:00.696800+0000","last_peered":"2024-04-25T12:35:00.696800+0000","last_clean":"2024-04-25T12:35:00.696800+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:35:00.696800+0000","last_undegraded":"2024-04-25T12:35:00.696800+0000","last_fullsized":"2024-04-25T12:35:00.696800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242577+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:35:00.242577+0000","last_peered":"2024-04-25T12:35:00.242577+0000","last_clean":"2024-04-25T12:35:00.242577+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:35:00.242577+0000","last_undegraded":"2024-04-25T12:35:00.242577+0000","last_fullsized":"2024-04-25T12:35:00.242577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488857+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:35:00.488857+0000","last_peered":"2024-04-25T12:35:00.488857+0000","last_clean":"2024-04-25T12:35:00.488857+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:35:00.488857+0000","last_undegraded":"2024-04-25T12:35:00.488857+0000","last_fullsized":"2024-04-25T12:35:00.488857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237299+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:35:00.237299+0000","last_peered":"2024-04-25T12:35:00.237299+0000","last_clean":"2024-04-25T12:35:00.237299+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:35:00.237299+0000","last_undegraded":"2024-04-25T12:35:00.237299+0000","last_fullsized":"2024-04-25T12:35:00.237299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236954+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:35:00.236954+0000","last_peered":"2024-04-25T12:35:00.236954+0000","last_clean":"2024-04-25T12:35:00.236954+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:35:00.236954+0000","last_undegraded":"2024-04-25T12:35:00.236954+0000","last_fullsized":"2024-04-25T12:35:00.236954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235770+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:35:00.235770+0000","last_peered":"2024-04-25T12:35:00.235770+0000","last_clean":"2024-04-25T12:35:00.235770+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:35:00.235770+0000","last_undegraded":"2024-04-25T12:35:00.235770+0000","last_fullsized":"2024-04-25T12:35:00.235770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235564+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:35:00.235564+0000","last_peered":"2024-04-25T12:35:00.235564+0000","last_clean":"2024-04-25T12:35:00.235564+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:35:00.235564+0000","last_undegraded":"2024-04-25T12:35:00.235564+0000","last_fullsized":"2024-04-25T12:35:00.235564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238662+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:35:00.238662+0000","last_peered":"2024-04-25T12:35:00.238662+0000","last_clean":"2024-04-25T12:35:00.238662+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:35:00.238662+0000","last_undegraded":"2024-04-25T12:35:00.238662+0000","last_fullsized":"2024-04-25T12:35:00.238662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237434+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:35:00.237434+0000","last_peered":"2024-04-25T12:35:00.237434+0000","last_clean":"2024-04-25T12:35:00.237434+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:35:00.237434+0000","last_undegraded":"2024-04-25T12:35:00.237434+0000","last_fullsized":"2024-04-25T12:35:00.237434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.240432+0000","last_change":"2024-04-25T12:35:00.239744+0000","last_active":"2024-04-25T12:35:00.240432+0000","last_peered":"2024-04-25T12:35:00.240432+0000","last_clean":"2024-04-25T12:35:00.240432+0000","last_became_active":"2024-04-25T12:35:00.239414+0000","last_became_peered":"2024-04-25T12:35:00.239414+0000","last_unstale":"2024-04-25T12:35:00.240432+0000","last_undegraded":"2024-04-25T12:35:00.240432+0000","last_fullsized":"2024-04-25T12:35:00.240432+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:17.015606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'14","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:08.877704+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:35:08.877704+0000","last_peered":"2024-04-25T12:35:08.877704+0000","last_clean":"2024-04-25T12:35:08.877704+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:35:08.877704+0000","last_undegraded":"2024-04-25T12:35:08.877704+0000","last_fullsized":"2024-04-25T12:35:08.877704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4100,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694248+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:35:00.694248+0000","last_peered":"2024-04-25T12:35:00.694248+0000","last_clean":"2024-04-25T12:35:00.694248+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:35:00.694248+0000","last_undegraded":"2024-04-25T12:35:00.694248+0000","last_fullsized":"2024-04-25T12:35:00.694248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236992+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:35:00.236992+0000","last_peered":"2024-04-25T12:35:00.236992+0000","last_clean":"2024-04-25T12:35:00.236992+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:35:00.236992+0000","last_undegraded":"2024-04-25T12:35:00.236992+0000","last_fullsized":"2024-04-25T12:35:00.236992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238965+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:35:00.238965+0000","last_peered":"2024-04-25T12:35:00.238965+0000","last_clean":"2024-04-25T12:35:00.238965+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:35:00.238965+0000","last_undegraded":"2024-04-25T12:35:00.238965+0000","last_fullsized":"2024-04-25T12:35:00.238965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699488+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:35:00.699488+0000","last_peered":"2024-04-25T12:35:00.699488+0000","last_clean":"2024-04-25T12:35:00.699488+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:35:00.699488+0000","last_undegraded":"2024-04-25T12:35:00.699488+0000","last_fullsized":"2024-04-25T12:35:00.699488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.489002+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:35:00.489002+0000","last_peered":"2024-04-25T12:35:00.489002+0000","last_clean":"2024-04-25T12:35:00.489002+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:35:00.489002+0000","last_undegraded":"2024-04-25T12:35:00.489002+0000","last_fullsized":"2024-04-25T12:35:00.489002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245001+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:35:00.245001+0000","last_peered":"2024-04-25T12:35:00.245001+0000","last_clean":"2024-04-25T12:35:00.245001+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:35:00.245001+0000","last_undegraded":"2024-04-25T12:35:00.245001+0000","last_fullsized":"2024-04-25T12:35:00.245001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238024+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:35:00.238024+0000","last_peered":"2024-04-25T12:35:00.238024+0000","last_clean":"2024-04-25T12:35:00.238024+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:35:00.238024+0000","last_undegraded":"2024-04-25T12:35:00.238024+0000","last_fullsized":"2024-04-25T12:35:00.238024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.701303+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:35:00.701303+0000","last_peered":"2024-04-25T12:35:00.701303+0000","last_clean":"2024-04-25T12:35:00.701303+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:35:00.701303+0000","last_undegraded":"2024-04-25T12:35:00.701303+0000","last_fullsized":"2024-04-25T12:35:00.701303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700940+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:35:00.700940+0000","last_peered":"2024-04-25T12:35:00.700940+0000","last_clean":"2024-04-25T12:35:00.700940+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:35:00.700940+0000","last_undegraded":"2024-04-25T12:35:00.700940+0000","last_fullsized":"2024-04-25T12:35:00.700940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694320+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:35:00.694320+0000","last_peered":"2024-04-25T12:35:00.694320+0000","last_clean":"2024-04-25T12:35:00.694320+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:35:00.694320+0000","last_undegraded":"2024-04-25T12:35:00.694320+0000","last_fullsized":"2024-04-25T12:35:00.694320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694611+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:35:00.694611+0000","last_peered":"2024-04-25T12:35:00.694611+0000","last_clean":"2024-04-25T12:35:00.694611+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:35:00.694611+0000","last_undegraded":"2024-04-25T12:35:00.694611+0000","last_fullsized":"2024-04-25T12:35:00.694611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694483+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:35:00.694483+0000","last_peered":"2024-04-25T12:35:00.694483+0000","last_clean":"2024-04-25T12:35:00.694483+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:35:00.694483+0000","last_undegraded":"2024-04-25T12:35:00.694483+0000","last_fullsized":"2024-04-25T12:35:00.694483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236757+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:35:00.236757+0000","last_peered":"2024-04-25T12:35:00.236757+0000","last_clean":"2024-04-25T12:35:00.236757+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:35:00.236757+0000","last_undegraded":"2024-04-25T12:35:00.236757+0000","last_fullsized":"2024-04-25T12:35:00.236757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237774+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:35:00.237774+0000","last_peered":"2024-04-25T12:35:00.237774+0000","last_clean":"2024-04-25T12:35:00.237774+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:35:00.237774+0000","last_undegraded":"2024-04-25T12:35:00.237774+0000","last_fullsized":"2024-04-25T12:35:00.237774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488252+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:35:00.488252+0000","last_peered":"2024-04-25T12:35:00.488252+0000","last_clean":"2024-04-25T12:35:00.488252+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:35:00.488252+0000","last_undegraded":"2024-04-25T12:35:00.488252+0000","last_fullsized":"2024-04-25T12:35:00.488252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235386+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:35:00.235386+0000","last_peered":"2024-04-25T12:35:00.235386+0000","last_clean":"2024-04-25T12:35:00.235386+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:35:00.235386+0000","last_undegraded":"2024-04-25T12:35:00.235386+0000","last_fullsized":"2024-04-25T12:35:00.235386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236858+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:35:00.236858+0000","last_peered":"2024-04-25T12:35:00.236858+0000","last_clean":"2024-04-25T12:35:00.236858+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:35:00.236858+0000","last_undegraded":"2024-04-25T12:35:00.236858+0000","last_fullsized":"2024-04-25T12:35:00.236858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694382+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:35:00.694382+0000","last_peered":"2024-04-25T12:35:00.694382+0000","last_clean":"2024-04-25T12:35:00.694382+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:35:00.694382+0000","last_undegraded":"2024-04-25T12:35:00.694382+0000","last_fullsized":"2024-04-25T12:35:00.694382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694541+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:35:00.694541+0000","last_peered":"2024-04-25T12:35:00.694541+0000","last_clean":"2024-04-25T12:35:00.694541+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:35:00.694541+0000","last_undegraded":"2024-04-25T12:35:00.694541+0000","last_fullsized":"2024-04-25T12:35:00.694541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488638+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:35:00.488638+0000","last_peered":"2024-04-25T12:35:00.488638+0000","last_clean":"2024-04-25T12:35:00.488638+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:35:00.488638+0000","last_undegraded":"2024-04-25T12:35:00.488638+0000","last_fullsized":"2024-04-25T12:35:00.488638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699483+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:35:00.699483+0000","last_peered":"2024-04-25T12:35:00.699483+0000","last_clean":"2024-04-25T12:35:00.699483+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:35:00.699483+0000","last_undegraded":"2024-04-25T12:35:00.699483+0000","last_fullsized":"2024-04-25T12:35:00.699483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238812+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:35:00.238812+0000","last_peered":"2024-04-25T12:35:00.238812+0000","last_clean":"2024-04-25T12:35:00.238812+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:35:00.238812+0000","last_undegraded":"2024-04-25T12:35:00.238812+0000","last_fullsized":"2024-04-25T12:35:00.238812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699291+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:35:00.699291+0000","last_peered":"2024-04-25T12:35:00.699291+0000","last_clean":"2024-04-25T12:35:00.699291+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:35:00.699291+0000","last_undegraded":"2024-04-25T12:35:00.699291+0000","last_fullsized":"2024-04-25T12:35:00.699291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238474+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:35:00.238474+0000","last_peered":"2024-04-25T12:35:00.238474+0000","last_clean":"2024-04-25T12:35:00.238474+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:35:00.238474+0000","last_undegraded":"2024-04-25T12:35:00.238474+0000","last_fullsized":"2024-04-25T12:35:00.238474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236497+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:35:00.236497+0000","last_peered":"2024-04-25T12:35:00.236497+0000","last_clean":"2024-04-25T12:35:00.236497+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:35:00.236497+0000","last_undegraded":"2024-04-25T12:35:00.236497+0000","last_fullsized":"2024-04-25T12:35:00.236497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488552+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:35:00.488552+0000","last_peered":"2024-04-25T12:35:00.488552+0000","last_clean":"2024-04-25T12:35:00.488552+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:35:00.488552+0000","last_undegraded":"2024-04-25T12:35:00.488552+0000","last_fullsized":"2024-04-25T12:35:00.488552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235986+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:35:00.235986+0000","last_peered":"2024-04-25T12:35:00.235986+0000","last_clean":"2024-04-25T12:35:00.235986+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:35:00.235986+0000","last_undegraded":"2024-04-25T12:35:00.235986+0000","last_fullsized":"2024-04-25T12:35:00.235986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236085+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:35:00.236085+0000","last_peered":"2024-04-25T12:35:00.236085+0000","last_clean":"2024-04-25T12:35:00.236085+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:35:00.236085+0000","last_undegraded":"2024-04-25T12:35:00.236085+0000","last_fullsized":"2024-04-25T12:35:00.236085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700980+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:35:00.700980+0000","last_peered":"2024-04-25T12:35:00.700980+0000","last_clean":"2024-04-25T12:35:00.700980+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:35:00.700980+0000","last_undegraded":"2024-04-25T12:35:00.700980+0000","last_fullsized":"2024-04-25T12:35:00.700980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488827+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:35:00.488827+0000","last_peered":"2024-04-25T12:35:00.488827+0000","last_clean":"2024-04-25T12:35:00.488827+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:35:00.488827+0000","last_undegraded":"2024-04-25T12:35:00.488827+0000","last_fullsized":"2024-04-25T12:35:00.488827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236512+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:35:00.236512+0000","last_peered":"2024-04-25T12:35:00.236512+0000","last_clean":"2024-04-25T12:35:00.236512+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:35:00.236512+0000","last_undegraded":"2024-04-25T12:35:00.236512+0000","last_fullsized":"2024-04-25T12:35:00.236512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488137+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:35:00.488137+0000","last_peered":"2024-04-25T12:35:00.488137+0000","last_clean":"2024-04-25T12:35:00.488137+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:35:00.488137+0000","last_undegraded":"2024-04-25T12:35:00.488137+0000","last_fullsized":"2024-04-25T12:35:00.488137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.699318+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:35:00.699318+0000","last_peered":"2024-04-25T12:35:00.699318+0000","last_clean":"2024-04-25T12:35:00.699318+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:35:00.699318+0000","last_undegraded":"2024-04-25T12:35:00.699318+0000","last_fullsized":"2024-04-25T12:35:00.699318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.694330+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:35:00.694330+0000","last_peered":"2024-04-25T12:35:00.694330+0000","last_clean":"2024-04-25T12:35:00.694330+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:35:00.694330+0000","last_undegraded":"2024-04-25T12:35:00.694330+0000","last_fullsized":"2024-04-25T12:35:00.694330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244838+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:35:00.244838+0000","last_peered":"2024-04-25T12:35:00.244838+0000","last_clean":"2024-04-25T12:35:00.244838+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:35:00.244838+0000","last_undegraded":"2024-04-25T12:35:00.244838+0000","last_fullsized":"2024-04-25T12:35:00.244838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.237777+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:35:00.237777+0000","last_peered":"2024-04-25T12:35:00.237777+0000","last_clean":"2024-04-25T12:35:00.237777+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:35:00.237777+0000","last_undegraded":"2024-04-25T12:35:00.237777+0000","last_fullsized":"2024-04-25T12:35:00.237777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238887+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:35:00.238887+0000","last_peered":"2024-04-25T12:35:00.238887+0000","last_clean":"2024-04-25T12:35:00.238887+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:35:00.238887+0000","last_undegraded":"2024-04-25T12:35:00.238887+0000","last_fullsized":"2024-04-25T12:35:00.238887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.238159+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:35:00.238159+0000","last_peered":"2024-04-25T12:35:00.238159+0000","last_clean":"2024-04-25T12:35:00.238159+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:35:00.238159+0000","last_undegraded":"2024-04-25T12:35:00.238159+0000","last_fullsized":"2024-04-25T12:35:00.238159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.236425+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:35:00.236425+0000","last_peered":"2024-04-25T12:35:00.236425+0000","last_clean":"2024-04-25T12:35:00.236425+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:35:00.236425+0000","last_undegraded":"2024-04-25T12:35:00.236425+0000","last_fullsized":"2024-04-25T12:35:00.236425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488070+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:35:00.488070+0000","last_peered":"2024-04-25T12:35:00.488070+0000","last_clean":"2024-04-25T12:35:00.488070+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:35:00.488070+0000","last_undegraded":"2024-04-25T12:35:00.488070+0000","last_fullsized":"2024-04-25T12:35:00.488070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235433+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:35:00.235433+0000","last_peered":"2024-04-25T12:35:00.235433+0000","last_clean":"2024-04-25T12:35:00.235433+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:35:00.235433+0000","last_undegraded":"2024-04-25T12:35:00.235433+0000","last_fullsized":"2024-04-25T12:35:00.235433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.235696+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:35:00.235696+0000","last_peered":"2024-04-25T12:35:00.235696+0000","last_clean":"2024-04-25T12:35:00.235696+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:35:00.235696+0000","last_undegraded":"2024-04-25T12:35:00.235696+0000","last_fullsized":"2024-04-25T12:35:00.235696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244713+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:35:00.244713+0000","last_peered":"2024-04-25T12:35:00.244713+0000","last_clean":"2024-04-25T12:35:00.244713+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:35:00.244713+0000","last_undegraded":"2024-04-25T12:35:00.244713+0000","last_fullsized":"2024-04-25T12:35:00.244713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244441+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:35:00.244441+0000","last_peered":"2024-04-25T12:35:00.244441+0000","last_clean":"2024-04-25T12:35:00.244441+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:35:00.244441+0000","last_undegraded":"2024-04-25T12:35:00.244441+0000","last_fullsized":"2024-04-25T12:35:00.244441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.243194+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:35:00.243194+0000","last_peered":"2024-04-25T12:35:00.243194+0000","last_clean":"2024-04-25T12:35:00.243194+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:35:00.243194+0000","last_undegraded":"2024-04-25T12:35:00.243194+0000","last_fullsized":"2024-04-25T12:35:00.243194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.700840+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:35:00.700840+0000","last_peered":"2024-04-25T12:35:00.700840+0000","last_clean":"2024-04-25T12:35:00.700840+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:35:00.700840+0000","last_undegraded":"2024-04-25T12:35:00.700840+0000","last_fullsized":"2024-04-25T12:35:00.700840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488044+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:35:00.488044+0000","last_peered":"2024-04-25T12:35:00.488044+0000","last_clean":"2024-04-25T12:35:00.488044+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:35:00.488044+0000","last_undegraded":"2024-04-25T12:35:00.488044+0000","last_fullsized":"2024-04-25T12:35:00.488044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.242559+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:35:00.242559+0000","last_peered":"2024-04-25T12:35:00.242559+0000","last_clean":"2024-04-25T12:35:00.242559+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:35:00.242559+0000","last_undegraded":"2024-04-25T12:35:00.242559+0000","last_fullsized":"2024-04-25T12:35:00.242559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698632+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:35:00.698632+0000","last_peered":"2024-04-25T12:35:00.698632+0000","last_clean":"2024-04-25T12:35:00.698632+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:35:00.698632+0000","last_undegraded":"2024-04-25T12:35:00.698632+0000","last_fullsized":"2024-04-25T12:35:00.698632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.244141+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:35:00.244141+0000","last_peered":"2024-04-25T12:35:00.244141+0000","last_clean":"2024-04-25T12:35:00.244141+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:35:00.244141+0000","last_undegraded":"2024-04-25T12:35:00.244141+0000","last_fullsized":"2024-04-25T12:35:00.244141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.698101+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:35:00.698101+0000","last_peered":"2024-04-25T12:35:00.698101+0000","last_clean":"2024-04-25T12:35:00.698101+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:35:00.698101+0000","last_undegraded":"2024-04-25T12:35:00.698101+0000","last_fullsized":"2024-04-25T12:35:00.698101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488547+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:35:00.488547+0000","last_peered":"2024-04-25T12:35:00.488547+0000","last_clean":"2024-04-25T12:35:00.488547+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:35:00.488547+0000","last_undegraded":"2024-04-25T12:35:00.488547+0000","last_fullsized":"2024-04-25T12:35:00.488547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.488685+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:35:00.488685+0000","last_peered":"2024-04-25T12:35:00.488685+0000","last_clean":"2024-04-25T12:35:00.488685+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:35:00.488685+0000","last_undegraded":"2024-04-25T12:35:00.488685+0000","last_fullsized":"2024-04-25T12:35:00.488685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T12:35:00.245426+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:35:00.245426+0000","last_peered":"2024-04-25T12:35:00.245426+0000","last_clean":"2024-04-25T12:35:00.245426+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:35:00.245426+0000","last_undegraded":"2024-04-25T12:35:00.245426+0000","last_fullsized":"2024-04-25T12:35:00.245426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7710,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":117903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":307200,"data_stored":117495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":707759,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":327680,"data_stored":122255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":120721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":707115,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:35:13.929 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [3, 7, 5, 4] 2024-04-25T12:35:13.929 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.0', '3', '7', '5', '4'] 2024-04-25T12:35:13.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.0 3 7 5 4 2024-04-25T12:35:13.938 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:13.938 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:13.960 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:13.988 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:13.989+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:14.089 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:14.090+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:14.190 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:14.191+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:14.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.207+0000 7fd57f200640 1 Processor -- start 2024-04-25T12:35:14.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.207+0000 7fd57f200640 1 -- start start 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.209+0000 7fd57f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd5780c5340 0x7fd57815d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.209+0000 7fd57f200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd578156770 0x7fd5780c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.209+0000 7fd57f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5780738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.209+0000 7fd57f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd57815d670 con 0x7fd5780c5340 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.209+0000 7fd57f200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd5780c5750 con 0x7fd578156770 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.209+0000 7fd57f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780c7110 con 0x7fd578156b80 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd5780c5340 0x7fd57815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd578156770 0x7fd5780c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd5780c5340 0x7fd57815d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51970/0 (socket says 172.21.15.123:51970) 2024-04-25T12:35:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 -- 172.21.15.123:0/202054893 learned_addr learned my addr 172.21.15.123:0/202054893 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd57d200640 1 --2- 172.21.15.123:0/202054893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5780738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 -- 172.21.15.123:0/202054893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 msgr2=0x7fd5780738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 --2- 172.21.15.123:0/202054893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5780738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 -- 172.21.15.123:0/202054893 >> v2:172.21.15.153:3300/0 conn(0x7fd578156770 msgr2=0x7fd5780c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 --2- 172.21.15.123:0/202054893 >> v2:172.21.15.153:3300/0 conn(0x7fd578156770 0x7fd5780c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.210+0000 7fd577e00640 1 -- 172.21.15.123:0/202054893 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5780c59f0 con 0x7fd5780c5340 2024-04-25T12:35:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.211+0000 7fd577e00640 1 --2- 172.21.15.123:0/202054893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd57815d130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd578073690 tx=0x7fd56802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d798e5599183d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.211+0000 7fd576a00640 1 -- 172.21.15.123:0/202054893 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568037070 con 0x7fd5780c5340 2024-04-25T12:35:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.212+0000 7fd576a00640 1 -- 172.21.15.123:0/202054893 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd568032440 con 0x7fd5780c5340 2024-04-25T12:35:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.212+0000 7fd576a00640 1 -- 172.21.15.123:0/202054893 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568032810 con 0x7fd5780c5340 2024-04-25T12:35:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.212+0000 7fd57f200640 1 -- 172.21.15.123:0/202054893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 msgr2=0x7fd57815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.212+0000 7fd57f200640 1 --2- 172.21.15.123:0/202054893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd57815d130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd578073690 tx=0x7fd56802a270 comp rx=0 tx=0).stop 2024-04-25T12:35:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.213+0000 7fd57f200640 1 -- 172.21.15.123:0/202054893 shutdown_connections 2024-04-25T12:35:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.213+0000 7fd57f200640 1 --2- 172.21.15.123:0/202054893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5780738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.213+0000 7fd57f200640 1 --2- 172.21.15.123:0/202054893 >> v2:172.21.15.153:3300/0 conn(0x7fd578156770 0x7fd5780c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.213+0000 7fd57f200640 1 --2- 172.21.15.123:0/202054893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd57815d130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.213+0000 7fd57f200640 1 -- 172.21.15.123:0/202054893 >> 172.21.15.123:0/202054893 conn(0x7fd5780cde40 msgr2=0x7fd5780ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.213+0000 7fd57f200640 1 -- 172.21.15.123:0/202054893 shutdown_connections 2024-04-25T12:35:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.213+0000 7fd57f200640 1 -- 172.21.15.123:0/202054893 wait complete. 2024-04-25T12:35:14.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.215+0000 7fd57f200640 1 Processor -- start 2024-04-25T12:35:14.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.215+0000 7fd57f200640 1 -- start start 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd57f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd5781597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd57f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd578156770 0x7fd5781f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd57f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5781e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd57f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780c7990 con 0x7fd5780c5340 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd57f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780c76b0 con 0x7fd578156770 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd57f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780c7c70 con 0x7fd578156b80 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd5781597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd5781597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51976/0 (socket says 172.21.15.123:51976) 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 -- 172.21.15.123:0/4183026390 learned_addr learned my addr 172.21.15.123:0/4183026390 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577400640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd578156770 0x7fd5781f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd57d200640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5781e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 -- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 msgr2=0x7fd5781e2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5781e2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 -- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd578156770 msgr2=0x7fd5781f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd578156770 0x7fd5781f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.217+0000 7fd577e00640 1 -- 172.21.15.123:0/4183026390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd578159d10 con 0x7fd5780c5340 2024-04-25T12:35:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.218+0000 7fd577e00640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd5781597d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd56802a2b0 tx=0x7fd5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.218+0000 7fd56fe00640 1 -- 172.21.15.123:0/4183026390 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568041020 con 0x7fd5780c5340 2024-04-25T12:35:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.218+0000 7fd56fe00640 1 -- 172.21.15.123:0/4183026390 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56803d620 con 0x7fd5780c5340 2024-04-25T12:35:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.218+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5781f3690 con 0x7fd5780c5340 2024-04-25T12:35:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.218+0000 7fd56fe00640 1 -- 172.21.15.123:0/4183026390 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568037040 con 0x7fd5780c5340 2024-04-25T12:35:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.218+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd578157ef0 con 0x7fd5780c5340 2024-04-25T12:35:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.220+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd578076700 con 0x7fd5780c5340 2024-04-25T12:35:14.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.221+0000 7fd56fe00640 1 -- 172.21.15.123:0/4183026390 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd568044020 con 0x7fd5780c5340 2024-04-25T12:35:14.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.221+0000 7fd56fe00640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd5580744b0 0x7fd5580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:14.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.222+0000 7fd56fe00640 1 -- 172.21.15.123:0/4183026390 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fd5680b5c70 con 0x7fd5780c5340 2024-04-25T12:35:14.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.222+0000 7fd577400640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd5580744b0 0x7fd5580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:14.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.223+0000 7fd577400640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd5580744b0 0x7fd5580769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5780cda80 tx=0x7fd564009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:14.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.226+0000 7fd56fe00640 1 -- 172.21.15.123:0/4183026390 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd56804b030 con 0x7fd5780c5340 2024-04-25T12:35:14.261 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:14.261 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:14.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:14.323 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:14.324+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:14.391 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:14.392+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:14.432+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "1.0", "id": [3, 7, 5, 4]} v 0) -- 0x7fd5780c8e60 con 0x7fd5780c5340 2024-04-25T12:35:14.493 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:14.493+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:14.576 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:14.577 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:14.593 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:14.594+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:14.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:14.694 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:14.695+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:14.794 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:14.795+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:14.895 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:14.896+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:14.914 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:14.915 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:14.935 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:14.996 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:14.997+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:15.097 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:15.098+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:15.198 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:15.198+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:15.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:15.254 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:15.254 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:15.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:15.298 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:15.299+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:15.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.307+0000 7fd56fe00640 1 -- 172.21.15.123:0/4183026390 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "1.0", "id": [3, 7, 5, 4]}]=0 set 1.0 pg_upmap_items mapping to [3->7,5->4] v29) ==== 146+0+0 (secure 0 0 0) 0x7fd568088090 con 0x7fd5780c5340 2024-04-25T12:35:15.306 INFO:teuthology.orchestra.run.smithi123.stderr:set 1.0 pg_upmap_items mapping to [3->7,5->4] 2024-04-25T12:35:15.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd5580744b0 msgr2=0x7fd5580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd5580744b0 0x7fd5580769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5780cda80 tx=0x7fd564009190 comp rx=0 tx=0).stop 2024-04-25T12:35:15.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 msgr2=0x7fd5781597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd5781597d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd56802a2b0 tx=0x7fd5680046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 shutdown_connections 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd5580744b0 0x7fd5580769c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd578156b80 0x7fd5781e2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd578156770 0x7fd5781f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 --2- 172.21.15.123:0/4183026390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5780c5340 0x7fd5781597d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 >> 172.21.15.123:0/4183026390 conn(0x7fd5780cde40 msgr2=0x7fd57815bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.309+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 shutdown_connections 2024-04-25T12:35:15.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.310+0000 7fd57f200640 1 -- 172.21.15.123:0/4183026390 wait complete. 2024-04-25T12:35:15.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:35:15.399 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:15.400+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:15.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.497+0000 7ff6ef600640 1 Processor -- start 2024-04-25T12:35:15.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.497+0000 7ff6ef600640 1 -- start start 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.499+0000 7ff6ef600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e815aca0 0x7ff6e8161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.499+0000 7ff6ef600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6e80c3740 0x7ff6e80ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.499+0000 7ff6ef600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff6e80c3b50 0x7ff6e80ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.499+0000 7ff6ef600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e8074170 con 0x7ff6e80c3b50 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.499+0000 7ff6ef600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e8073230 con 0x7ff6e80c3740 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.499+0000 7ff6ef600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e8071220 con 0x7ff6e815aca0 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff6e80c3b50 0x7ff6e80ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff6e80c3b50 0x7ff6e80ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51992/0 (socket says 172.21.15.123:51992) 2024-04-25T12:35:15.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 -- 172.21.15.123:0/3092112747 learned_addr learned my addr 172.21.15.123:0/3092112747 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:15.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/3092112747 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e815aca0 0x7ff6e8161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/3092112747 >> v2:172.21.15.153:3300/0 conn(0x7ff6e80c3740 0x7ff6e80ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 -- 172.21.15.123:0/3092112747 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e815aca0 msgr2=0x7ff6e8161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 --2- 172.21.15.123:0/3092112747 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e815aca0 0x7ff6e8161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 -- 172.21.15.123:0/3092112747 >> v2:172.21.15.153:3300/0 conn(0x7ff6e80c3740 msgr2=0x7ff6e80ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 --2- 172.21.15.123:0/3092112747 >> v2:172.21.15.153:3300/0 conn(0x7ff6e80c3740 0x7ff6e80ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.500+0000 7ff6ed600640 1 -- 172.21.15.123:0/3092112747 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e815b0b0 con 0x7ff6e80c3b50 2024-04-25T12:35:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.501+0000 7ff6ed600640 1 --2- 172.21.15.123:0/3092112747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e80c3b50 0x7ff6e80ca5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80713a0 tx=0x7ff6e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111fe53a458585a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.502+0000 7ff6e7400640 1 -- 172.21.15.123:0/3092112747 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0036070 con 0x7ff6e80c3b50 2024-04-25T12:35:15.501 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:15.502+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:15.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.502+0000 7ff6e7400640 1 -- 172.21.15.123:0/3092112747 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e0031600 con 0x7ff6e80c3b50 2024-04-25T12:35:15.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.502+0000 7ff6e7400640 1 -- 172.21.15.123:0/3092112747 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00319d0 con 0x7ff6e80c3b50 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 -- 172.21.15.123:0/3092112747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e80c3b50 msgr2=0x7ff6e80ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 --2- 172.21.15.123:0/3092112747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e80c3b50 0x7ff6e80ca5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80713a0 tx=0x7ff6e002a710 comp rx=0 tx=0).stop 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 -- 172.21.15.123:0/3092112747 shutdown_connections 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 --2- 172.21.15.123:0/3092112747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e80c3b50 0x7ff6e80ca5a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 --2- 172.21.15.123:0/3092112747 >> v2:172.21.15.153:3300/0 conn(0x7ff6e80c3740 0x7ff6e80ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 --2- 172.21.15.123:0/3092112747 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e815aca0 0x7ff6e8161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 -- 172.21.15.123:0/3092112747 >> 172.21.15.123:0/3092112747 conn(0x7ff6e806f440 msgr2=0x7ff6e806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 -- 172.21.15.123:0/3092112747 shutdown_connections 2024-04-25T12:35:15.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.503+0000 7ff6ef600640 1 -- 172.21.15.123:0/3092112747 wait complete. 2024-04-25T12:35:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.506+0000 7ff6ef600640 1 Processor -- start 2024-04-25T12:35:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.506+0000 7ff6ef600640 1 -- start start 2024-04-25T12:35:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.507+0000 7ff6ef600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e80c3740 0x7ff6e815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.507+0000 7ff6ef600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6e80c3b50 0x7ff6e8150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.507+0000 7ff6ef600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e815aca0 0x7ff6e8150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.507+0000 7ff6ef600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e80708f0 con 0x7ff6e815aca0 2024-04-25T12:35:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.507+0000 7ff6ef600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e8073760 con 0x7ff6e80c3b50 2024-04-25T12:35:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.507+0000 7ff6ef600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e8070bd0 con 0x7ff6e80c3740 2024-04-25T12:35:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6ed600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e815aca0 0x7ff6e8150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6ecc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e80c3740 0x7ff6e815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6e7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6e80c3b50 0x7ff6e8150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6ed600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e815aca0 0x7ff6e8150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52004/0 (socket says 172.21.15.123:52004) 2024-04-25T12:35:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6ed600640 1 -- 172.21.15.123:0/2038163038 learned_addr learned my addr 172.21.15.123:0/2038163038 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6e7e00640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e80c3740 msgr2=0x7ff6e815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6e80c3740 0x7ff6e815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6e7e00640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e815aca0 msgr2=0x7ff6e8150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6e815aca0 0x7ff6e8150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.508+0000 7ff6e7e00640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e815de30 con 0x7ff6e80c3b50 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.509+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6e80c3b50 0x7ff6e8150a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80cad70 tx=0x7ff6d400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.509+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4017070 con 0x7ff6e80c3b50 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.509+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d400c700 con 0x7ff6e80c3b50 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.509+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400caf0 con 0x7ff6e80c3b50 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.509+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e815c0f0 con 0x7ff6e80c3b50 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.509+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e8152660 con 0x7ff6e80c3b50 2024-04-25T12:35:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.510+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff6e815bf10 con 0x7ff6e80c3b50 2024-04-25T12:35:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.512+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff6d401a070 con 0x7ff6e80c3b50 2024-04-25T12:35:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.513+0000 7ff6e4c00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff6c80744e0 0x7ff6c80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.514+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7ff6d403ce00 con 0x7ff6e80c3b50 2024-04-25T12:35:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.514+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v29) ==== 54+0+16 (secure 0 0 0) 0x7ff6e815bf10 con 0x7ff6e80c3b50 2024-04-25T12:35:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.514+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff6c80744e0 0x7ff6c80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.514+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff6e81588b0 0x7ff6e80cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.514+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6e80cdf10 con 0x7ff6e81588b0 2024-04-25T12:35:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.515+0000 7ff6ed600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff6e81588b0 0x7ff6e80cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.515+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff6c80744e0 0x7ff6c80769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009a20 tx=0x7ff6dc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.515+0000 7ff6ed600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff6e81588b0 0x7ff6e80cca40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.516+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e0037040 con 0x7ff6e81588b0 2024-04-25T12:35:15.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.533+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e80b3b10 con 0x7ff6e81588b0 2024-04-25T12:35:15.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.535+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e80b3b10 con 0x7ff6e81588b0 2024-04-25T12:35:15.534 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.535+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff6e80cd490 0x7ff6e81fa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.535+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff6e80cdaa0 con 0x7ff6e80cd490 2024-04-25T12:35:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.536+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff6e80cd490 0x7ff6e81fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.537+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff6e80cd490 0x7ff6e81fa920 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.538+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e80cdaa0 con 0x7ff6e80cd490 2024-04-25T12:35:15.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.555+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e8076700 con 0x7ff6e80cd490 2024-04-25T12:35:15.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.558+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e8076700 con 0x7ff6e80cd490 2024-04-25T12:35:15.557 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:15.558 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:15.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.558+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff6e81fb370 0x7ff6e8062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.558+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff6e81fb780 con 0x7ff6e81fb370 2024-04-25T12:35:15.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.559+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff6e81fb370 0x7ff6e8062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.559+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff6e81fb370 0x7ff6e8062fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.560+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e81fb780 con 0x7ff6e81fb370 2024-04-25T12:35:15.559 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:15.559 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:15.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.575+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e80c2960 con 0x7ff6e81fb370 2024-04-25T12:35:15.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.578+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e80c2960 con 0x7ff6e81fb370 2024-04-25T12:35:15.577 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:15.577 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:15.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.578+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff6e80c7e60 0x7ff6e80c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.578+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff6e8074630 con 0x7ff6e80c7e60 2024-04-25T12:35:15.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.578+0000 7ff6ed600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff6e80c7e60 0x7ff6e80c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.579+0000 7ff6ed600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff6e80c7e60 0x7ff6e80c8270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.581+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e8074630 con 0x7ff6e80c7e60 2024-04-25T12:35:15.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:15.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.597+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e815f060 con 0x7ff6e80c7e60 2024-04-25T12:35:15.601 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:15.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.599+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e815f060 con 0x7ff6e80c7e60 2024-04-25T12:35:15.602 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:15.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.600+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff6e802c9d0 0x7ff6e802cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.600+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff6e815ed80 con 0x7ff6e802c9d0 2024-04-25T12:35:15.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.600+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff6e802c9d0 0x7ff6e802cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.601+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff6e802c9d0 0x7ff6e802cde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.602+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e815ed80 con 0x7ff6e802c9d0 2024-04-25T12:35:15.603 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:15.602+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:15.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.617+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e815ed80 con 0x7ff6e802c9d0 2024-04-25T12:35:15.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.620+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e815ed80 con 0x7ff6e802c9d0 2024-04-25T12:35:15.619 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:15.619 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:15.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.620+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff6e802d830 0x7ff6e820dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.620+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff6e80c8cc0 con 0x7ff6e802d830 2024-04-25T12:35:15.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.620+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff6e802d830 0x7ff6e820dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.621+0000 7ff6e7e00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff6e802d830 0x7ff6e820dcb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.622+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e80c8cc0 con 0x7ff6e802d830 2024-04-25T12:35:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.637+0000 7f61e7e00640 1 Processor -- start 2024-04-25T12:35:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.637+0000 7f61e7e00640 1 -- start start 2024-04-25T12:35:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.638+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e815eaf0 con 0x7ff6e802d830 2024-04-25T12:35:15.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e7e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61e80cba00 0x7f61e80ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61e80cc420 0x7f61e80cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e7e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e80ca730 con 0x7f61e815a5f0 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e7e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f61e80c4d20 con 0x7f61e80cc420 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e7e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f61e80c5000 con 0x7f61e80cba00 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e6000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61e80cc420 0x7f61e80cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e6a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61e80cba00 0x7f61e80ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e7400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f61e815a5f0 0x7f61e815aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43566/0 (socket says 172.21.15.123:43566) 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e6a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61e80cba00 0x7f61e80ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33116/0 (socket says 172.21.15.123:33116) 2024-04-25T12:35:15.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.639+0000 7f61e7400640 1 -- 172.21.15.123:0/3809585838 learned_addr learned my addr 172.21.15.123:0/3809585838 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.640+0000 7f61e6a00640 1 -- 172.21.15.123:0/3809585838 >> v2:172.21.15.123:3300/0 conn(0x7f61e80cc420 msgr2=0x7f61e80cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.640+0000 7f61e6a00640 1 --2- 172.21.15.123:0/3809585838 >> v2:172.21.15.123:3300/0 conn(0x7f61e80cc420 0x7f61e80cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.640+0000 7f61e6a00640 1 -- 172.21.15.123:0/3809585838 >> v1:172.21.15.123:6789/0 conn(0x7f61e815a5f0 legacy=0x7f61e815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.640+0000 7f61e6a00640 1 -- 172.21.15.123:0/3809585838 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e80c52e0 con 0x7f61e80cba00 2024-04-25T12:35:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.640+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e815eaf0 con 0x7ff6e802d830 2024-04-25T12:35:15.641 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.641+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff6e802dc40 0x7ff6e8210710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.641+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff6e8213560 con 0x7ff6e802dc40 2024-04-25T12:35:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.641+0000 7f61e6a00640 1 --2- 172.21.15.123:0/3809585838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cba00 0x7f61e80ca0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f61e8071e10 tx=0x7f61d802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d4ed8b931dcf66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.641+0000 7ff6ed600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff6e802dc40 0x7ff6e8210710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.641+0000 7f61e5600640 1 -- 172.21.15.123:0/3809585838 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8037070 con 0x7f61e80cba00 2024-04-25T12:35:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.641+0000 7f61e5600640 1 -- 172.21.15.123:0/3809585838 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d8032440 con 0x7f61e80cba00 2024-04-25T12:35:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.641+0000 7f61e5600640 1 -- 172.21.15.123:0/3809585838 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8032810 con 0x7f61e80cba00 2024-04-25T12:35:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.642+0000 7ff6ed600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff6e802dc40 0x7ff6e8210710 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.642+0000 7f61e7e00640 1 -- 172.21.15.123:0/3809585838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cba00 msgr2=0x7f61e80ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.642+0000 7f61e7e00640 1 --2- 172.21.15.123:0/3809585838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cba00 0x7f61e80ca0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f61e8071e10 tx=0x7f61d802a1a0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.642+0000 7f61e7e00640 1 -- 172.21.15.123:0/3809585838 shutdown_connections 2024-04-25T12:35:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.642+0000 7f61e7e00640 1 --2- 172.21.15.123:0/3809585838 >> v2:172.21.15.123:3300/0 conn(0x7f61e80cc420 0x7f61e80cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.642+0000 7f61e7e00640 1 --2- 172.21.15.123:0/3809585838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cba00 0x7f61e80ca0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.642+0000 7f61e7e00640 1 -- 172.21.15.123:0/3809585838 >> 172.21.15.123:0/3809585838 conn(0x7f61e8072cb0 msgr2=0x7f61e80730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.643+0000 7f61e7e00640 1 -- 172.21.15.123:0/3809585838 shutdown_connections 2024-04-25T12:35:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.643+0000 7f61e7e00640 1 -- 172.21.15.123:0/3809585838 wait complete. 2024-04-25T12:35:15.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.643+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e8213560 con 0x7ff6e802dc40 2024-04-25T12:35:15.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.646+0000 7f61e7e00640 1 Processor -- start 2024-04-25T12:35:15.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.647+0000 7f61e7e00640 1 -- start start 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.649+0000 7f61e7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61e80cba00 0x7f61e8156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.649+0000 7f61e7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cc420 0x7f61e80cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.649+0000 7f61e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e815a5f0 0x7f61e8149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.649+0000 7f61e7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80c66f0 con 0x7f61e815a5f0 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.649+0000 7f61e7e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80c58a0 con 0x7f61e80cc420 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.649+0000 7f61e7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f61e8154540 con 0x7f61e80cba00 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e815a5f0 0x7f61e8149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61e80cba00 0x7f61e8156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61e80cba00 0x7f61e8156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37510/0 (socket says 172.21.15.123:37510) 2024-04-25T12:35:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 -- 172.21.15.123:0/515325451 learned_addr learned my addr 172.21.15.123:0/515325451 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e815a5f0 0x7f61e8149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52032/0 (socket says 172.21.15.123:52032) 2024-04-25T12:35:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6000640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cc420 0x7f61e80cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 -- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cc420 msgr2=0x7f61e80cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cc420 0x7f61e80cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 -- 172.21.15.123:0/515325451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e815a5f0 msgr2=0x7f61e8149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e815a5f0 0x7f61e8149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6a00640 1 -- 172.21.15.123:0/515325451 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e81573a0 con 0x7f61e80cba00 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e6000640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cc420 0x7f61e80cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.650+0000 7f61e7400640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e815a5f0 0x7f61e8149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.651+0000 7f61e6a00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61e80cba00 0x7f61e8156e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000c00 tx=0x7f61d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.651+0000 7f61df400640 1 -- 172.21.15.123:0/515325451 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8037040 con 0x7f61e80cba00 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.651+0000 7f61df400640 1 -- 172.21.15.123:0/515325451 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d8042070 con 0x7f61e80cba00 2024-04-25T12:35:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.651+0000 7f61df400640 1 -- 172.21.15.123:0/515325451 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8030960 con 0x7f61e80cba00 2024-04-25T12:35:15.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.651+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e8157f90 con 0x7f61e80cba00 2024-04-25T12:35:15.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.651+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e81e70a0 con 0x7f61e80cba00 2024-04-25T12:35:15.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.652+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e8076da0 con 0x7f61e80cba00 2024-04-25T12:35:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.654+0000 7f61df400640 1 -- 172.21.15.123:0/515325451 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61d8030b00 con 0x7f61e80cba00 2024-04-25T12:35:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.655+0000 7f61df400640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61c80744e0 0x7f61c80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.655+0000 7f61df400640 1 -- 172.21.15.123:0/515325451 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 6619+0+0 (secure 0 0 0) 0x7f61d80b5eb0 con 0x7f61e80cba00 2024-04-25T12:35:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.656+0000 7f61e6000640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61c80744e0 0x7f61c80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.656+0000 7f61e6000640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61c80744e0 0x7f61c80769f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61e81f01b0 tx=0x7f61d4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.660+0000 7f61df400640 1 -- 172.21.15.123:0/515325451 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61d807fb40 con 0x7f61e80cba00 2024-04-25T12:35:15.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.663+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e8210c50 con 0x7ff6e802dc40 2024-04-25T12:35:15.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.665+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e8210c50 con 0x7ff6e802dc40 2024-04-25T12:35:15.664 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:15.665 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:15.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.665+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff6e82112d0 0x7ff6e8211700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:15.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.665+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff6e82168d0 con 0x7ff6e82112d0 2024-04-25T12:35:15.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.666+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff6e82112d0 0x7ff6e8211700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:15.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.667+0000 7ff6ecc00640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff6e82112d0 0x7ff6e8211700 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:15.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.668+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e82168d0 con 0x7ff6e82112d0 2024-04-25T12:35:15.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.683+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6e8211d50 con 0x7ff6e82112d0 2024-04-25T12:35:15.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.685+0000 7ff6e4c00640 1 -- 172.21.15.123:0/2038163038 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6e8211d50 con 0x7ff6e82112d0 2024-04-25T12:35:15.684 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff6e81588b0 msgr2=0x7ff6e80cca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff6e81588b0 0x7ff6e80cca40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff6e80cd490 msgr2=0x7ff6e81fa920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff6e80cd490 0x7ff6e81fa920 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff6e81fb370 msgr2=0x7ff6e8062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff6e81fb370 0x7ff6e8062fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff6e80c7e60 msgr2=0x7ff6e80c8270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff6e80c7e60 0x7ff6e80c8270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff6e802c9d0 msgr2=0x7ff6e802cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff6e802c9d0 0x7ff6e802cde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff6e802d830 msgr2=0x7ff6e820dcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff6e802d830 0x7ff6e820dcb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff6e802dc40 msgr2=0x7ff6e8210710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff6e802dc40 0x7ff6e8210710 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff6e82112d0 msgr2=0x7ff6e8211700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff6e82112d0 0x7ff6e8211700 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ed600640 1 -- 172.21.15.123:0/2038163038 reap_dead start 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ed600640 1 -- 172.21.15.123:0/2038163038 reap_dead start 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ed600640 1 -- 172.21.15.123:0/2038163038 reap_dead start 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ed600640 1 -- 172.21.15.123:0/2038163038 reap_dead start 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ed600640 1 -- 172.21.15.123:0/2038163038 reap_dead start 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff6c80744e0 msgr2=0x7ff6c80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff6c80744e0 0x7ff6c80769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009a20 tx=0x7ff6dc009190 comp rx=0 tx=0).stop 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6e80c3b50 msgr2=0x7ff6e8150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6e80c3b50 0x7ff6e8150a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80cad70 tx=0x7ff6d400ecd0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 shutdown_connections 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff6c80744e0 0x7ff6c80769f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 --2- 172.21.15.123:0/2038163038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6e80c3b50 0x7ff6e8150a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 >> 172.21.15.123:0/2038163038 conn(0x7ff6e806f440 msgr2=0x7ff6e8159740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 shutdown_connections 2024-04-25T12:35:15.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.686+0000 7ff6ef600640 1 -- 172.21.15.123:0/2038163038 wait complete. 2024-04-25T12:35:15.702 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:15.703+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:15.803 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:15.804+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:15.850 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:15.851 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:15.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:15.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:15.877+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f61e80c9a60 con 0x7f61e80cba00 2024-04-25T12:35:15.918 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:15.919+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:16.004 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:16.006+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:16.105 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:16.106+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:16.123 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:16.123 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:16.144 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:16.206 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:16.207+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:16.307 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:16.307+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:16.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.316+0000 7f61df400640 1 -- 172.21.15.123:0/515325451 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v30) ==== 97+0+0 (secure 0 0 0) 0x7f61d8083e40 con 0x7f61e80cba00 2024-04-25T12:35:16.316 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.323+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61c80744e0 msgr2=0x7f61c80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.323+0000 7f61e7e00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61c80744e0 0x7f61c80769f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61e81f01b0 tx=0x7f61d4009190 comp rx=0 tx=0).stop 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.323+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61e80cba00 msgr2=0x7f61e8156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.323+0000 7f61e7e00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61e80cba00 0x7f61e8156e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000c00 tx=0x7f61d80046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.324+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 shutdown_connections 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.324+0000 7f61e7e00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61c80744e0 0x7f61c80769f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.324+0000 7f61e7e00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e815a5f0 0x7f61e8149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.324+0000 7f61e7e00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e80cc420 0x7f61e80cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.324+0000 7f61e7e00640 1 --2- 172.21.15.123:0/515325451 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61e80cba00 0x7f61e8156e60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:16.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.324+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 >> 172.21.15.123:0/515325451 conn(0x7f61e8072cb0 msgr2=0x7f61e814c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:16.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.326+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 shutdown_connections 2024-04-25T12:35:16.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:16.326+0000 7f61e7e00640 1 -- 172.21.15.123:0/515325451 wait complete. 2024-04-25T12:35:16.407 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:16.407+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.534490251000001, 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:35:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:35:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:35:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.533806069000001, 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:35:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:35:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:35:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.532355527999997, 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:35:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:35:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:35:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.532094712999999, 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.528244325999999, 2024-04-25T12:35:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:35:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.528021957999997, 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.527668169000002, 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:35:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:35:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.367049220999998, 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:35:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366841729000001, 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:35:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:35:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:35:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366697239000001, 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:35:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:35:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:35:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366565444999999, 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:35:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:35:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:35:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366491150000002, 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366391408999998, 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:35:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:35:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366321946999999, 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:35:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:35:16.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.553 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366221833000001, 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:35:16.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:35:16.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366124742, 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:35:16.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:35:16.563 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.366034336999999, 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.567 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.365886364000001, 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:16.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:35:16.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:35:16.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.571 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.365778959, 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:16.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:35:16.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:35:16.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 21.331527660999999, 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:16.575 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:16.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:16.580 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:16.581 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:16.582 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:16.582 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:16.582+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:16.684 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:16.684+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:16.707 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:16.707 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:16.707 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:16.707 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:16.717 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:16.784 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:16.784+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:16.829 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:16.829 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:16.829 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:16.829 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:16.829 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:16.839 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:16.923 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:16.924+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:16.986 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:16.987+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:16.995 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:16.995 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:16.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045567536, 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:16.996 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:35:16.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:16.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:16.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045479827000001, 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.001 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:35:17.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:35:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045353898999998, 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.006 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045292240000002, 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.012 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:35:17.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045235476999999, 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:35:17.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:35:17.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.020 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045172006999998, 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:35:17.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.025 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045120058000002, 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:17.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:35:17.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:35:17.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:17.030 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.045068108000002, 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:17.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.035 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.044950999000001, 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:17.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:35:17.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.044912552, 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.040 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:35:17.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 26.879777498999999, 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:35:17.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:35:17.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 26.879719447999999, 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:35:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:35:17.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:35:17.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:35:17.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 26.879659754999999, 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:35:17.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 21.842739673000001, 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:35:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:35:17.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.062 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 21.842650707000001, 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 21.842606866000001, 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 21.841092962000001, 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:35:17.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.077 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 21.840173746000001, 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:35:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 21.839718392999998, 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:35:17.082 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:17.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.085 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 21.839689414999999, 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.086 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.089 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:17.091 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:35:17.091 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:17.091+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:17.191 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:17.191+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:17.242 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:17.243 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:17.243 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:17.243 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:17.251 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:35:17.292 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:17.293+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:17.393 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:17.393+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:17.405 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:17.405 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:17.405 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:17.405 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:17.405 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:17.414 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:35:17.494 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:17.495+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:17.568 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:17.568 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:23 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'13, mlcod=16'8)", 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 68.976079485, 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039927010000000004, 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:17.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7399999999999998e-07 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592515+0000", 2024-04-25T12:35:17.570 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9832e-05 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592524+0000", 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2429999999999994e-06 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592532+0000", 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.576e-06 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.571 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592580+0000", 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8093000000000002e-05 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592597+0000", 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6688999999999999e-05 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.596424+0000", 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0038271770000000002 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.596457+0000", 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.3617e-05 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:24 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'14, mlcod=16'9)", 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 68.974500117000005, 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052768190000000003, 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.573 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7599999999999997e-07 2024-04-25T12:35:17.574 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594111+0000", 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6662999999999995e-05 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594121+0000", 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8949999999999993e-06 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594129+0000", 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2679999999999994e-06 2024-04-25T12:35:17.575 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594173+0000", 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3340999999999997e-05 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594188+0000", 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5838999999999999e-05 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599274+0000", 2024-04-25T12:35:17.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0050853210000000003 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599321+0000", 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7015999999999998e-05 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:25 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'15, mlcod=16'9)", 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:08.594612+0000", 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 68.973931648000004, 2024-04-25T12:35:17.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047303079999999999, 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594612+0000", 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594612+0000", 2024-04-25T12:35:17.578 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594613+0000", 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500000000000001e-07 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594618+0000", 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1089999999999997e-06 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.579 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594626+0000", 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8390000000000007e-06 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594634+0000", 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7510000000000008e-06 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594667+0000", 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.3776000000000003e-05 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594681+0000", 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3949e-05 2024-04-25T12:35:17.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599324+0000", 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0046431099999999998 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599343+0000", 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8399000000000001e-05 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:65 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'40)", 2024-04-25T12:35:17.581 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.251375+0000", 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 68.317168636999995, 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036189350000000002, 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251375+0000", 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.582 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251375+0000", 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251376+0000", 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7599999999999997e-07 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251383+0000", 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5719999999999999e-06 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251389+0000", 2024-04-25T12:35:17.583 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0970000000000002e-06 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251394+0000", 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6760000000000004e-06 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251439+0000", 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4836999999999998e-05 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251461+0000", 2024-04-25T12:35:17.584 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1756e-05 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.254963+0000", 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003501905 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.254994+0000", 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1615999999999997e-05 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.585 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:79 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'57, mlcod=16'53)", 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.274232+0000", 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 68.294312063999996, 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003717354, 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274232+0000", 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.586 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274232+0000", 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274233+0000", 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.58e-07 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274322+0000", 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9189000000000003e-05 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.587 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274330+0000", 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1089999999999998e-06 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274337+0000", 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.255e-06 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274384+0000", 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6873999999999999e-05 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274399+0000", 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4987999999999999e-05 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.277912+0000", 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035133930000000001 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.277949+0000", 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7088000000000001e-05 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:80 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'58, mlcod=16'53)", 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 68.292582315000004, 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047846920000000001, 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4200000000000001e-07 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275995+0000", 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2805999999999997e-05 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276004+0000", 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.5960000000000004e-06 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276011+0000", 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3540000000000002e-06 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.592 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276056+0000", 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4888000000000001e-05 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276071+0000", 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5560999999999999e-05 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.280709+0000", 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0046372460000000002 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.593 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.280746+0000", 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7799000000000002e-05 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:90 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'64, mlcod=16'60)", 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.940815330000007, 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047246010000000001, 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.594 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:17.595 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3799999999999997e-07 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627853+0000", 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012308499999999999 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627863+0000", 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0135999999999999e-05 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.596 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627869+0000", 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7190000000000004e-06 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627906+0000", 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.65e-05 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627919+0000", 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2955e-05 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.597 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632420+0000", 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004501672 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632453+0000", 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2895999999999998e-05 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:91 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'65, mlcod=17'61)", 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:17.598 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.939614583999997, 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035550040000000001, 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:17.599 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2099999999999996e-07 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628938+0000", 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5490000000000001e-06 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628948+0000", 2024-04-25T12:35:17.600 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7929999999999992e-06 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628955+0000", 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4760000000000001e-06 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628997+0000", 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.193e-05 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.629011+0000", 2024-04-25T12:35:17.601 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632455+0000", 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003444253 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632485+0000", 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9059000000000001e-05 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.602 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:92 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'66, mlcod=17'61)", 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.938509316999998, 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0050398830000000002, 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2799999999999996e-07 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630042+0000", 2024-04-25T12:35:17.604 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.861e-06 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630052+0000", 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630059+0000", 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.63e-06 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630093+0000", 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4567000000000003e-05 2024-04-25T12:35:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630107+0000", 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.322e-05 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635043+0000", 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049359740000000001 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635075+0000", 2024-04-25T12:35:17.606 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2076000000000003e-05 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:93 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'61)", 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.937923342999994, 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044888319999999999, 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.607 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2600000000000002e-07 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.608 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630657+0000", 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5537000000000002e-05 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630667+0000", 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0107e-05 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630675+0000", 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8699999999999992e-06 2024-04-25T12:35:17.609 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630718+0000", 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3050999999999998e-05 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630732+0000", 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4154e-05 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635078+0000", 2024-04-25T12:35:17.610 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043455569999999999 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635110+0000", 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1930000000000001e-05 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:94 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'62)", 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.631743+0000", 2024-04-25T12:35:17.611 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.936800640000001, 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034012959999999998, 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631743+0000", 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631743+0000", 2024-04-25T12:35:17.612 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631744+0000", 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631751+0000", 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7499999999999997e-06 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.613 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631761+0000", 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8430000000000005e-06 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631767+0000", 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1820000000000003e-06 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631811+0000", 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3634999999999998e-05 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.614 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631824+0000", 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3584000000000001e-05 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635111+0000", 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0032869750000000001 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635145+0000", 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.3707000000000001e-05 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.615 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.950975+0000", 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.617568726999998, 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033063559999999999, 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.616 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950975+0000", 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950975+0000", 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950976+0000", 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.850000000000001e-07 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4749999999999999e-06 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951002+0000", 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6501e-05 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951014+0000", 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1234e-05 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951081+0000", 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6964000000000005e-05 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7339999999999998e-05 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954232+0000", 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031145080000000002 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.619 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954282+0000", 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9348999999999998e-05 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:30 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953314+0000", 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.615230556, 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010402056, 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.620 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953314+0000", 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953314+0000", 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953328+0000", 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4114000000000001e-05 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953341+0000", 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3162e-05 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953356+0000", 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5218000000000001e-05 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953368+0000", 2024-04-25T12:35:17.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2150999999999999e-05 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953488+0000", 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000119907 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953638+0000", 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015016300000000001 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953791+0000", 2024-04-25T12:35:17.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015301900000000001 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955457+0000", 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001665435 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.963628+0000", 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081709910000000007 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.963667+0000", 2024-04-25T12:35:17.624 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9196999999999998e-05 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.963716+0000", 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954270+0000", 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.614274166999998, 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0045854889999999999, 2024-04-25T12:35:17.625 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954270+0000", 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954270+0000", 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.626 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954271+0000", 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.085e-06 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954287+0000", 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6257999999999999e-05 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954298+0000", 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0616e-05 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954308+0000", 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0139999999999999e-05 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954381+0000", 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2992999999999997e-05 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954501+0000", 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000120127 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.628 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958855+0000", 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043542700000000004 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 58.613949820999999, 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043590520000000004, 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.629 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954595+0000", 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0440000000000001e-06 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954601+0000", 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.818e-06 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954654+0000", 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2880000000000002e-05 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.631 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954668+0000", 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4136e-05 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954732+0000", 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3540999999999994e-05 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954883+0000", 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015099199999999999 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958953+0000", 2024-04-25T12:35:17.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040706409999999998 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115422+0000", 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.453122373999999, 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004970716, 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:17.633 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115422+0000", 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115422+0000", 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115424+0000", 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2400000000000002e-06 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115434+0000", 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0338e-05 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115453+0000", 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9040999999999999e-05 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115468+0000", 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4187e-05 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115555+0000", 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7397000000000004e-05 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115703+0000", 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001484 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 6", 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115862+0000", 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015840099999999999 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.636 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119676+0000", 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0038141529999999998 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120295+0000", 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00061928999999999999 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120336+0000", 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1304000000000003e-05 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120392+0000", 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5964999999999997e-05 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1 epoch 25/18 rep_tid 7 entries 25'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116283+0000", 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.452260902999999, 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003323505, 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.638 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116283+0000", 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116283+0000", 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116285+0000", 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.77e-06 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116300+0000", 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5498999999999998e-05 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116328+0000", 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7024000000000001e-05 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116367+0000", 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9910000000000002e-05 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116599+0000", 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000231757 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119607+0000", 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030075449999999999 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.641 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117004+0000", 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.451540484999999, 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039571210000000001, 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117004+0000", 2024-04-25T12:35:17.642 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117004+0000", 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117005+0000", 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.372e-06 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117011+0000", 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9689999999999999e-06 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117022+0000", 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1521e-05 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117035+0000", 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.216e-05 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.644 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117101+0000", 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6558999999999998e-05 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117253+0000", 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000151736 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120961+0000", 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037078039999999999 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.645 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117569+0000", 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.450975611, 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035307899999999998, 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.646 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117569+0000", 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117569+0000", 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.9800000000000002e-07 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.647 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117577+0000", 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0330000000000003e-06 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117587+0000", 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0767e-05 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117598+0000", 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1174e-05 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.648 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117677+0000", 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8745000000000006e-05 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117852+0000", 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017459600000000001 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121099+0000", 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0032474769999999999 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.649 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152648+0000", 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.415896226000001, 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047121940000000003, 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152648+0000", 2024-04-25T12:35:17.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152648+0000", 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152650+0000", 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.075e-06 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152658+0000", 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5850000000000002e-06 2024-04-25T12:35:17.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152674+0000", 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6121999999999999e-05 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152686+0000", 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2446000000000001e-05 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152822+0000", 2024-04-25T12:35:17.652 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135771 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152972+0000", 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014999499999999999 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157360+0000", 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043882000000000001 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.653 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:17.654 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:17.654 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:17.655 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:17.655 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:17.656+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:17.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:17.764 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:17.765+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:17.806 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:17.806 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:17.814 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:17.856 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:17.857+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:17.924 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:17.925 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:17.925 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:17.933 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:17.957 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:17.958+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:17.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:17.997+0000 7fb5aee00640 1 Processor -- start 2024-04-25T12:35:17.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:17.998+0000 7fb5aee00640 1 -- start start 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.000+0000 7fb5aee00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb5a815e320 0x7fb5a8165a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.000+0000 7fb5aee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a80b9d90 con 0x7fb5a8072d60 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.000+0000 7fb5aee00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a815a410 con 0x7fb5a8154350 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.000+0000 7fb5aee00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb5a815d780 con 0x7fb5a815e320 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.000+0000 7fb5ada00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb5a8154350 0x7fb5a80c6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:58924/0 (socket says 172.21.15.123:58924) 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.000+0000 7fb5ada00640 1 -- 172.21.15.123:0/1797182261 learned_addr learned my addr 172.21.15.123:0/1797182261 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.000+0000 7fb5ae400640 1 --2- 172.21.15.123:0/1797182261 >> v2:172.21.15.153:3300/0 conn(0x7fb5a815e320 0x7fb5a8165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.001+0000 7fb5a7e00640 1 -- 172.21.15.123:0/1797182261 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721371134 0 0) 0x7fb5a815a410 con 0x7fb5a8154350 2024-04-25T12:35:18.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.001+0000 7fb5a7e00640 1 -- 172.21.15.123:0/1797182261 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb590003520 con 0x7fb5a8154350 2024-04-25T12:35:18.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.001+0000 7fb5a7e00640 1 -- 172.21.15.123:0/1797182261 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768927919 0 0) 0x7fb5a80b9d90 con 0x7fb5a8072d60 2024-04-25T12:35:18.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.001+0000 7fb5ae400640 1 -- 172.21.15.123:0/1797182261 >> v1:172.21.15.153:6789/0 conn(0x7fb5a8154350 legacy=0x7fb5a80c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.001+0000 7fb5ae400640 1 -- 172.21.15.123:0/1797182261 >> v1:172.21.15.123:6789/0 conn(0x7fb5a8072d60 legacy=0x7fb5a815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.001+0000 7fb5ae400640 1 -- 172.21.15.123:0/1797182261 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a815a6d0 con 0x7fb5a815e320 2024-04-25T12:35:18.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.002+0000 7fb5ae400640 1 --2- 172.21.15.123:0/1797182261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a815e320 0x7fb5a8165a90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009d90 tx=0x7fb5a002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b9a5c1fd7d8d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.002+0000 7fb5a7e00640 1 -- 172.21.15.123:0/1797182261 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0007740 con 0x7fb5a815e320 2024-04-25T12:35:18.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.002+0000 7fb5a7e00640 1 -- 172.21.15.123:0/1797182261 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a00078e0 con 0x7fb5a815e320 2024-04-25T12:35:18.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.002+0000 7fb5a7e00640 1 -- 172.21.15.123:0/1797182261 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0007cb0 con 0x7fb5a815e320 2024-04-25T12:35:18.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.003+0000 7fb5aee00640 1 -- 172.21.15.123:0/1797182261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a815e320 msgr2=0x7fb5a8165a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.003+0000 7fb5aee00640 1 --2- 172.21.15.123:0/1797182261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a815e320 0x7fb5a8165a90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009d90 tx=0x7fb5a002a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.003+0000 7fb5aee00640 1 -- 172.21.15.123:0/1797182261 shutdown_connections 2024-04-25T12:35:18.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.003+0000 7fb5aee00640 1 --2- 172.21.15.123:0/1797182261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a815e320 0x7fb5a8165a90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.003+0000 7fb5aee00640 1 -- 172.21.15.123:0/1797182261 >> 172.21.15.123:0/1797182261 conn(0x7fb5a806cba0 msgr2=0x7fb5a80cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:18.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.003+0000 7fb5aee00640 1 -- 172.21.15.123:0/1797182261 shutdown_connections 2024-04-25T12:35:18.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.003+0000 7fb5aee00640 1 -- 172.21.15.123:0/1797182261 wait complete. 2024-04-25T12:35:18.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.005+0000 7fb5aee00640 1 Processor -- start 2024-04-25T12:35:18.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.005+0000 7fb5aee00640 1 -- start start 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5aee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5a8072d60 0x7fb5a815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5aee00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a8154350 0x7fb5a80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5aee00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb5a815e320 0x7fb5a8148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5aee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a815af70 con 0x7fb5a8072d60 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5aee00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a815ac90 con 0x7fb5a8154350 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5aee00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a815a100 con 0x7fb5a815e320 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5ada00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5a8072d60 0x7fb5a815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5ada00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5a8072d60 0x7fb5a815c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52044/0 (socket says 172.21.15.123:52044) 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5ada00640 1 -- 172.21.15.123:0/165068575 learned_addr learned my addr 172.21.15.123:0/165068575 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5ae400640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb5a815e320 0x7fb5a8148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.007+0000 7fb5ad000640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a8154350 0x7fb5a80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5ada00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb5a815e320 msgr2=0x7fb5a8148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb5a815e320 0x7fb5a8148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5ada00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a8154350 msgr2=0x7fb5a80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5a8154350 0x7fb5a80c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5ada00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a815c940 con 0x7fb5a8072d60 2024-04-25T12:35:18.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5a8072d60 0x7fb5a815c400 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c410 tx=0x7fb59c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c017070 con 0x7fb5a8072d60 2024-04-25T12:35:18.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c005ce0 con 0x7fb5a8072d60 2024-04-25T12:35:18.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a80cca40 con 0x7fb5a8072d60 2024-04-25T12:35:18.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0072f0 con 0x7fb5a8072d60 2024-04-25T12:35:18.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.008+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a80cbdb0 con 0x7fb5a8072d60 2024-04-25T12:35:18.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.009+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb5a8076da0 con 0x7fb5a8072d60 2024-04-25T12:35:18.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.011+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb59c01a070 con 0x7fb5a8072d60 2024-04-25T12:35:18.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.012+0000 7fb5a5600640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb5880744e0 0x7fb5880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.012+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6619+0+0 (secure 0 0 0) 0x7fb59c0948f0 con 0x7fb5a8072d60 2024-04-25T12:35:18.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.012+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v30) ==== 54+0+16 (secure 0 0 0) 0x7fb59c099220 con 0x7fb5a8072d60 2024-04-25T12:35:18.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.012+0000 7fb5ad000640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb5880744e0 0x7fb5880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.012+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb5a80c89c0 0x7fb5a8148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.012+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5a80cc490 con 0x7fb5a80c89c0 2024-04-25T12:35:18.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.013+0000 7fb5ae400640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb5a80c89c0 0x7fb5a8148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.013+0000 7fb5ad000640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb5880744e0 0x7fb5880769f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80cac40 tx=0x7fb594009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.013+0000 7fb5ae400640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb5a80c89c0 0x7fb5a8148570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.014+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a0003a90 con 0x7fb5a80c89c0 2024-04-25T12:35:18.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.031+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a80b99d0 con 0x7fb5a80c89c0 2024-04-25T12:35:18.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.033+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a80b99d0 con 0x7fb5a80c89c0 2024-04-25T12:35:18.032 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:18.032 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:18.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.033+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb5a8142720 0x7fb5a8142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.033+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb5a8073850 con 0x7fb5a8142720 2024-04-25T12:35:18.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.033+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb5a8142720 0x7fb5a8142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.034+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb5a8142720 0x7fb5a8142b30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.035+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a8073850 con 0x7fb5a8142720 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 24/18 rep_tid 1 entries 24'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:18.048 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 59.093489787000003, 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075768199999999997, 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954840+0000", 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6869999999999999e-06 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954864+0000", 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3824000000000001e-05 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954902+0000", 2024-04-25T12:35:18.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500999999999999e-05 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954919+0000", 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7618e-05 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955011+0000", 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2250000000000006e-05 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955596+0000", 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058480200000000004 2024-04-25T12:35:18.051 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:25 2.1b e25/18 2:dbae76b5:::mds1_inotable:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.931804705000001, 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036130300000000001, 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116525+0000", 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.455e-06 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116539+0000", 2024-04-25T12:35:18.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4e-05 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116563+0000", 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4111999999999999e-05 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116577+0000", 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4059999999999999e-05 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116689+0000", 2024-04-25T12:35:18.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011209000000000001 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116719+0000", 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.989e-05 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120082+0000", 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033628130000000001 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120137+0000", 2024-04-25T12:35:18.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4610000000000001e-05 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 4 entries 25'3 (0'0) error 2:db457a8a:::201.00000001:head by mds.1.9:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.930748164000001, 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030622480000000001, 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117581+0000", 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0279999999999999e-06 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117596+0000", 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5109999999999999e-05 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117631+0000", 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4230000000000003e-05 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117655+0000", 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4987e-05 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117751+0000", 2024-04-25T12:35:18.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5311999999999998e-05 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120642+0000", 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028915809999999998 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 5 entries 25'4 (0'0) error 2:d80f8093:::201.00000002:head by mds.1.9:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.930246932999999, 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0025905469999999999, 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.059 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118082+0000", 2024-04-25T12:35:18.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4000000000000001e-07 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118091+0000", 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9919999999999996e-06 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118108+0000", 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7207999999999999e-05 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118124+0000", 2024-04-25T12:35:18.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5378000000000001e-05 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118203+0000", 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8929000000000003e-05 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120672+0000", 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024692999999999998 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:34 2.1b e25/18 2:db457a8a:::201.00000001:head v 25'5, mlcod=25'3)", 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.926772599, 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016516759999999999, 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121557+0000", 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.423e-06 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121568+0000", 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0499000000000001e-05 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121585+0000", 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7283e-05 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121597+0000", 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1821000000000001e-05 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121653+0000", 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6533000000000002e-05 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121671+0000", 2024-04-25T12:35:18.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7285000000000001e-05 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123158+0000", 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0014869519999999999 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123207+0000", 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9879999999999997e-05 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:35 2.a 2:54ba6923:::mds1_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.904660282000002, 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016436459999999999, 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143670+0000", 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4040000000000002e-06 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143685+0000", 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4768e-05 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143703+0000", 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7424000000000001e-05 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143721+0000", 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8685000000000002e-05 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143824+0000", 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010286800000000001 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144032+0000", 2024-04-25T12:35:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020812700000000001 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144205+0000", 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017276999999999999 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144982+0000", 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077719300000000002 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145196+0000", 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000213308 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145254+0000", 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8619000000000002e-05 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145312+0000", 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7479999999999999e-05 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.1.9:35 2.a e25/18 ondisk, result = 0)", 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.903233714999999, 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00022686700000000001, 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145096+0000", 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2309999999999999e-06 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145102+0000", 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1639999999999998e-06 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145115+0000", 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2809000000000001e-05 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145124+0000", 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9989999999999997e-06 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145178+0000", 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4287999999999999e-05 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145189+0000", 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.084e-05 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145321+0000", 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013253599999999999 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:11 2.c 2:36924187:::618.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:18.075 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.895984449, 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070050249999999998, 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:18.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152345+0000", 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6270000000000001e-06 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152355+0000", 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4069999999999999e-06 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152384+0000", 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9479e-05 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152399+0000", 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152478+0000", 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9702000000000004e-05 2024-04-25T12:35:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152615+0000", 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013665500000000001 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152730+0000", 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011504499999999999 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156394+0000", 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036643259999999999 2024-04-25T12:35:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028370299999999999 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159289+0000", 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7268999999999997e-05 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159349+0000", 2024-04-25T12:35:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0161999999999998e-05 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.895897874999999, 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070895239999999998, 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152431+0000", 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6899999999999996e-07 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152440+0000", 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3259999999999996e-06 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152450+0000", 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0874e-05 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152470+0000", 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9658999999999999e-05 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153265+0000", 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079454099999999995 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153349+0000", 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4381000000000004e-05 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153485+0000", 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136365 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:18.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156400+0000", 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029144990000000001 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159418+0000", 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030182299999999998 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159477+0000", 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159520+0000", 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3127000000000002e-05 2024-04-25T12:35:18.085 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.895834261000001, 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0068584939999999997, 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.086 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152495+0000", 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.16e-07 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152501+0000", 2024-04-25T12:35:18.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7819999999999999e-06 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152510+0000", 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9199999999999993e-06 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152518+0000", 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9090000000000006e-06 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152651+0000", 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000132349 2024-04-25T12:35:18.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152795+0000", 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014392800000000001 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152930+0000", 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013561399999999999 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156365+0000", 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034348830000000001 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.089 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159266+0000", 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029006959999999999 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159296+0000", 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0026e-05 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159353+0000", 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6570999999999998e-05 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.090 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.895748298000001, 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069239999999999996, 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152581+0000", 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3399999999999998e-07 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152587+0000", 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4049999999999997e-06 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152598+0000", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1043000000000001e-05 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152607+0000", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9409999999999996e-06 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153448+0000", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00084054699999999997 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153563+0000", 2024-04-25T12:35:18.093 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000115584 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153719+0000", 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015566199999999999 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156373+0000", 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002653617 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:18.094 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159423+0000", 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030508060000000001 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159472+0000", 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9042000000000002e-05 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159504+0000", 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1519000000000001e-05 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.095 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.895665415, 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0086667419999999998, 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.096 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152664+0000", 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0100000000000003e-07 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152671+0000", 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8870000000000003e-06 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152679+0000", 2024-04-25T12:35:18.097 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2730000000000002e-06 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152696+0000", 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7373000000000001e-05 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153747+0000", 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001050647 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.098 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153967+0000", 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00021982699999999999 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154122+0000", 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015472399999999999 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156405+0000", 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022838110000000002 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161248+0000", 2024-04-25T12:35:18.099 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048425669999999999 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161283+0000", 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5166000000000002e-05 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161330+0000", 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6566000000000001e-05 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.100 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.2 epoch 25/18 rep_tid 20 entries 25'1 (0'0) error 2:40dbbc66:::202.00000003:head by mds.2.13:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.894059534, 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0021562719999999999, 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.101 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154270+0000", 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.638e-06 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154290+0000", 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392e-05 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154315+0000", 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4858000000000002e-05 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154335+0000", 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0044e-05 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154711+0000", 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037622099999999999 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.103 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156425+0000", 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017141190000000001 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:11 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.889203640000002, 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00023418899999999999, 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:18.104 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159126+0000", 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7699999999999992e-07 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.105 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159131+0000", 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8109999999999999e-06 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159143+0000", 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1745e-05 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159153+0000", 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6520000000000001e-06 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.106 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159211+0000", 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8582999999999998e-05 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159225+0000", 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3088000000000001e-05 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159359+0000", 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013433299999999999 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.107 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:15 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.889153243999999, 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000189507, 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.108 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159176+0000", 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6899999999999998e-07 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159180+0000", 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1320000000000004e-06 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.109 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159188+0000", 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9319999999999996e-06 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159199+0000", 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1165999999999999e-05 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5071000000000002e-05 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159262+0000", 2024-04-25T12:35:18.110 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0160000000000008e-06 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159365+0000", 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010272099999999999 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:12 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.889114314, 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000315354, 2024-04-25T12:35:18.111 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.112 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2099999999999998e-07 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159217+0000", 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.875e-06 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159223+0000", 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2999999999999998e-06 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.113 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159229+0000", 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6740000000000002e-06 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159401+0000", 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017201599999999999 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159412+0000", 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0349e-05 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.114 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159529+0000", 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000117819 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:16 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.889086044999999, 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00026996799999999999, 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:18.115 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159243+0000", 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0100000000000001e-07 2024-04-25T12:35:18.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159246+0000", 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.687e-06 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.072e-06 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159261+0000", 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2169999999999999e-06 2024-04-25T12:35:18.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159408+0000", 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000147637 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159417+0000", 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999996e-06 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.118 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159512+0000", 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5163999999999999e-05 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:23 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 22.887171890000001, 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000182072, 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.119 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161157+0000", 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7400000000000002e-07 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.120 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161164+0000", 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161180+0000", 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6189999999999999e-05 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161191+0000", 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0981e-05 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.121 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161238+0000", 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7509999999999997e-05 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161244+0000", 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.694e-06 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161338+0000", 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4432999999999995e-05 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.122 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:18.123 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:18.123 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:18.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.051+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a80cd5d0 con 0x7fb5a8142720 2024-04-25T12:35:18.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.055+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a80cd5d0 con 0x7fb5a8142720 2024-04-25T12:35:18.124 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:18.125 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:18.125 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:18.124+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.055+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb5a8062fe0 0x7fb5a80633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.055+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb5a8143580 con 0x7fb5a8062fe0 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.056+0000 7fb5ad000640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb5a8062fe0 0x7fb5a80633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.057+0000 7fb5ad000640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb5a8062fe0 0x7fb5a80633f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.058+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a8143580 con 0x7fb5a8062fe0 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.074+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a80ccf90 con 0x7fb5a8062fe0 2024-04-25T12:35:18.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.077+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a80ccf90 con 0x7fb5a8062fe0 2024-04-25T12:35:18.127 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:18.127 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.077+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb5a8158310 0x7fb5a8158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.077+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb5a80c8f40 con 0x7fb5a8158310 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.078+0000 7fb5ae400640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb5a8158310 0x7fb5a8158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.078+0000 7fb5ae400640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb5a8158310 0x7fb5a8158720 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.079+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a80c8f40 con 0x7fb5a8158310 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.094+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a8157cc0 con 0x7fb5a8158310 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.096+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a8157cc0 con 0x7fb5a8158310 2024-04-25T12:35:18.128 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.097+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb5a802c9d0 0x7fb5a802cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.097+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb5a8159170 con 0x7fb5a802c9d0 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.097+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb5a802c9d0 0x7fb5a802cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.098+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb5a802c9d0 0x7fb5a802cde0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.099+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a8159170 con 0x7fb5a802c9d0 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.114+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a8159170 con 0x7fb5a802c9d0 2024-04-25T12:35:18.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.116+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a8159170 con 0x7fb5a802c9d0 2024-04-25T12:35:18.130 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:18.130 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:18.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.116+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb5a802d830 0x7fb5a8208e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.116+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb5a802dcc0 con 0x7fb5a802d830 2024-04-25T12:35:18.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.117+0000 7fb5ad000640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb5a802d830 0x7fb5a8208e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.117+0000 7fb5ad000640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb5a802d830 0x7fb5a8208e70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.118+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a802dcc0 con 0x7fb5a802d830 2024-04-25T12:35:18.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.133+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a80008d0 con 0x7fb5a802d830 2024-04-25T12:35:18.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.135+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a80008d0 con 0x7fb5a802d830 2024-04-25T12:35:18.134 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:18.135 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:18.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.135+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb5a82098c0 0x7fb5a820bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.135+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb5a820ebe0 con 0x7fb5a82098c0 2024-04-25T12:35:18.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.136+0000 7fb5ae400640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb5a82098c0 0x7fb5a820bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.136+0000 7fb5ae400640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb5a82098c0 0x7fb5a820bce0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.138+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a820ebe0 con 0x7fb5a82098c0 2024-04-25T12:35:18.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.152+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a802dcc0 con 0x7fb5a82098c0 2024-04-25T12:35:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.155+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a802dcc0 con 0x7fb5a82098c0 2024-04-25T12:35:18.154 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.155+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb5a820c730 0x7fb5a820cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.155+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb5a8211fb0 con 0x7fb5a820c730 2024-04-25T12:35:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.156+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb5a820c730 0x7fb5a820cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.156+0000 7fb5ada00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb5a820c730 0x7fb5a820cb40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.157+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5a8211fb0 con 0x7fb5a820c730 2024-04-25T12:35:18.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.173+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb5a820d190 con 0x7fb5a820c730 2024-04-25T12:35:18.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.175+0000 7fb5a5600640 1 -- 172.21.15.123:0/165068575 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb5a820d190 con 0x7fb5a820c730 2024-04-25T12:35:18.174 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:18.176 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:18.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.175+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb5a80c89c0 msgr2=0x7fb5a8148570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.175+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb5a80c89c0 0x7fb5a8148570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.175+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb5a8142720 msgr2=0x7fb5a8142b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.175+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb5a8142720 0x7fb5a8142b30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.175+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb5a8062fe0 msgr2=0x7fb5a80633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.175+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb5a8062fe0 0x7fb5a80633f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb5a8158310 msgr2=0x7fb5a8158720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb5a8158310 0x7fb5a8158720 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5ae400640 1 -- 172.21.15.123:0/165068575 reap_dead start 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb5a802c9d0 msgr2=0x7fb5a802cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb5a802c9d0 0x7fb5a802cde0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb5a802d830 msgr2=0x7fb5a8208e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb5a802d830 0x7fb5a8208e70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb5a82098c0 msgr2=0x7fb5a820bce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb5a82098c0 0x7fb5a820bce0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb5a820c730 msgr2=0x7fb5a820cb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb5a820c730 0x7fb5a820cb40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5ae400640 1 -- 172.21.15.123:0/165068575 reap_dead start 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb5880744e0 msgr2=0x7fb5880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb5880744e0 0x7fb5880769f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80cac40 tx=0x7fb594009040 comp rx=0 tx=0).stop 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5a8072d60 msgr2=0x7fb5a815c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5a8072d60 0x7fb5a815c400 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c410 tx=0x7fb59c00c8d0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 shutdown_connections 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb5880744e0 0x7fb5880769f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 --2- 172.21.15.123:0/165068575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5a8072d60 0x7fb5a815c400 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 >> 172.21.15.123:0/165068575 conn(0x7fb5a806cba0 msgr2=0x7fb5a80c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 shutdown_connections 2024-04-25T12:35:18.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.176+0000 7fb5aee00640 1 -- 172.21.15.123:0/165068575 wait complete. 2024-04-25T12:35:18.224 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:18.225+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:18.276 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:18.276 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:18.284 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:18.325 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:18.326+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:18.365 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:35:18.426 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:18.427+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:18.435 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:18.435 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:18.444 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:18.527 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:18.528+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:18.596 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:18.596 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:18.605 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:18.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.647+0000 7fc2cca00640 1 Processor -- start 2024-04-25T12:35:18.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.647+0000 7fc2cca00640 1 -- start start 2024-04-25T12:35:18.647 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:18.648+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.649+0000 7fc2cca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c8154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.649+0000 7fc2cca00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc2c815afa0 0x7fc2c80cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.649+0000 7fc2cca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2c815b3b0 con 0x7fc2c80c4d50 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.649+0000 7fc2cca00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc2c80c61f0 con 0x7fc2c815afa0 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.649+0000 7fc2cca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c80c87d0 con 0x7fc2c8074870 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.649+0000 7fc2c5e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc2c80c4d50 0x7fc2c80c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43574/0 (socket says 172.21.15.123:43574) 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.649+0000 7fc2c5e00640 1 -- 172.21.15.123:0/4125312099 learned_addr learned my addr 172.21.15.123:0/4125312099 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c6800640 1 --2- 172.21.15.123:0/4125312099 >> v2:172.21.15.153:3300/0 conn(0x7fc2c815afa0 0x7fc2c80cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c5400640 1 --2- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c4a00640 1 -- 172.21.15.123:0/4125312099 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947700870 0 0) 0x7fc2c815b3b0 con 0x7fc2c80c4d50 2024-04-25T12:35:18.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c4a00640 1 -- 172.21.15.123:0/4125312099 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2ac003520 con 0x7fc2c80c4d50 2024-04-25T12:35:18.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c6800640 1 -- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 msgr2=0x7fc2c8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:18.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c6800640 1 --2- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c8154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c6800640 1 -- 172.21.15.123:0/4125312099 >> v1:172.21.15.123:6789/0 conn(0x7fc2c80c4d50 legacy=0x7fc2c80c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c6800640 1 -- 172.21.15.123:0/4125312099 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c8073ce0 con 0x7fc2c815afa0 2024-04-25T12:35:18.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.650+0000 7fc2c5400640 1 --2- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:18.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.652+0000 7fc2c6800640 1 --2- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c815afa0 0x7fc2c80cda90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009d90 tx=0x7fc2c002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf09d411e0a005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.652+0000 7fc2c4a00640 1 -- 172.21.15.123:0/4125312099 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0007740 con 0x7fc2c815afa0 2024-04-25T12:35:18.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.652+0000 7fc2c4a00640 1 -- 172.21.15.123:0/4125312099 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c00078e0 con 0x7fc2c815afa0 2024-04-25T12:35:18.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.652+0000 7fc2c4a00640 1 -- 172.21.15.123:0/4125312099 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0007cb0 con 0x7fc2c815afa0 2024-04-25T12:35:18.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 -- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c815afa0 msgr2=0x7fc2c80cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 --2- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c815afa0 0x7fc2c80cda90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009d90 tx=0x7fc2c002a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 -- 172.21.15.123:0/4125312099 shutdown_connections 2024-04-25T12:35:18.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 --2- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c815afa0 0x7fc2c80cda90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 --2- 172.21.15.123:0/4125312099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 -- 172.21.15.123:0/4125312099 >> 172.21.15.123:0/4125312099 conn(0x7fc2c806c750 msgr2=0x7fc2c806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:18.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 -- 172.21.15.123:0/4125312099 shutdown_connections 2024-04-25T12:35:18.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.653+0000 7fc2cca00640 1 -- 172.21.15.123:0/4125312099 wait complete. 2024-04-25T12:35:18.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.655+0000 7fc2cca00640 1 Processor -- start 2024-04-25T12:35:18.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.655+0000 7fc2cca00640 1 -- start start 2024-04-25T12:35:18.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2cca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c80ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2cca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c80c4d50 0x7fc2c80cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2cca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2c815afa0 0x7fc2c80c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2cca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c80c8460 con 0x7fc2c815afa0 2024-04-25T12:35:18.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2cca00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c80742a0 con 0x7fc2c80c4d50 2024-04-25T12:35:18.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2cca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c80c9370 con 0x7fc2c8074870 2024-04-25T12:35:18.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2c815afa0 0x7fc2c80c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2c815afa0 0x7fc2c80c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52048/0 (socket says 172.21.15.123:52048) 2024-04-25T12:35:18.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c6800640 1 -- 172.21.15.123:0/1449811820 learned_addr learned my addr 172.21.15.123:0/1449811820 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:18.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c5400640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c80c4d50 0x7fc2c80cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c5e00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c80ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c5400640 1 -- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 msgr2=0x7fc2c80ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c5400640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c80ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c5400640 1 -- 172.21.15.123:0/1449811820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2c815afa0 msgr2=0x7fc2c80c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:18.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.657+0000 7fc2c5400640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2c815afa0 0x7fc2c80c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:18.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2c5400640 1 -- 172.21.15.123:0/1449811820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c80cd530 con 0x7fc2c80c4d50 2024-04-25T12:35:18.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2c6800640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2c815afa0 0x7fc2c80c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:18.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2c5400640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c80c4d50 0x7fc2c80cc7c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400e870 tx=0x7fc2b400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2be000640 1 -- 172.21.15.123:0/1449811820 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4017070 con 0x7fc2c80c4d50 2024-04-25T12:35:18.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2be000640 1 -- 172.21.15.123:0/1449811820 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b400c940 con 0x7fc2c80c4d50 2024-04-25T12:35:18.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c80ccd00 con 0x7fc2c80c4d50 2024-04-25T12:35:18.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2be000640 1 -- 172.21.15.123:0/1449811820 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400cd30 con 0x7fc2c80c4d50 2024-04-25T12:35:18.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.658+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c80cba00 con 0x7fc2c80c4d50 2024-04-25T12:35:18.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.660+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c8069920 con 0x7fc2c80c4d50 2024-04-25T12:35:18.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.661+0000 7fc2be000640 1 -- 172.21.15.123:0/1449811820 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc2b401a070 con 0x7fc2c80c4d50 2024-04-25T12:35:18.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.662+0000 7fc2be000640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc2a80744b0 0x7fc2a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:18.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.662+0000 7fc2be000640 1 -- 172.21.15.123:0/1449811820 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6619+0+0 (secure 0 0 0) 0x7fc2b4095350 con 0x7fc2c80c4d50 2024-04-25T12:35:18.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.662+0000 7fc2c5e00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc2a80744b0 0x7fc2a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:18.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.663+0000 7fc2c5e00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc2a80744b0 0x7fc2a80769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005880 tx=0x7fc2b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.666+0000 7fc2be000640 1 -- 172.21.15.123:0/1449811820 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2c8069920 con 0x7fc2c80c4d50 2024-04-25T12:35:18.729 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:18.730+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:18.830 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:18.830+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:18.864 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:18.864 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:18.878+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fc2c8076700 con 0x7fc2c80c4d50 2024-04-25T12:35:18.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:18.932 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:18.933+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:19.032 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:19.032+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:19.132 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:19.134+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:19.199 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:19.200 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:19.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:19.233 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:19.234+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:19.334 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:19.335+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:19.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.353+0000 7fc2be000640 1 -- 172.21.15.123:0/1449811820 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v31) ==== 91+0+0 (secure 0 0 0) 0x7fc2b405f0f0 con 0x7fc2c80c4d50 2024-04-25T12:35:19.353 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.356+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc2a80744b0 msgr2=0x7fc2a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.356+0000 7fc2cca00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc2a80744b0 0x7fc2a80769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005880 tx=0x7fc2b8009040 comp rx=0 tx=0).stop 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.356+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c80c4d50 msgr2=0x7fc2c80cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.356+0000 7fc2cca00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c80c4d50 0x7fc2c80cc7c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400e870 tx=0x7fc2b400ed30 comp rx=0 tx=0).stop 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 shutdown_connections 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc2a80744b0 0x7fc2a80769c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2c815afa0 0x7fc2c80c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2c80c4d50 0x7fc2c80cc7c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 --2- 172.21.15.123:0/1449811820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2c8074870 0x7fc2c80ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:19.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 >> 172.21.15.123:0/1449811820 conn(0x7fc2c806c750 msgr2=0x7fc2c80ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:19.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 shutdown_connections 2024-04-25T12:35:19.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:19.357+0000 7fc2cca00640 1 -- 172.21.15.123:0/1449811820 wait complete. 2024-04-25T12:35:19.435 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:19.436+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:19.498 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:19.498 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:19.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:19.567 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:19.568+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:19.637 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:19.638+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:19.737 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:19.738+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:19.796 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:19.796 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:19.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:19.838 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:19.839+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:19.939 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:19.940+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:20.040 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:20.041+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:20.136 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:20.136 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:20.141 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:20.142+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:20.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:20.202 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:20.251 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:20.252+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:20.339 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:35:20.339 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:35:20.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:35:20.342 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:20.343+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:20.443 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:20.444+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:20.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.449+0000 7f5591600640 1 Processor -- start 2024-04-25T12:35:20.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.449+0000 7f5591600640 1 -- start start 2024-04-25T12:35:20.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f5591600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f5591600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f558c0ca060 0x7f558c0ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f5591600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f558c073c10 con 0x7f558c0b2a10 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f5591600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f558c15d7b0 con 0x7f558c0ca060 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f5591600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f558c156fb0 con 0x7f558c073460 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f558b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f558c0ca060 0x7f558c0ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f558aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f558a000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f558c0b2a10 0x7f558c0c3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:58928/0 (socket says 172.21.15.123:58928) 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.451+0000 7f558a000640 1 -- 172.21.15.123:0/1103719866 learned_addr learned my addr 172.21.15.123:0/1103719866 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.452+0000 7f558aa00640 1 -- 172.21.15.123:0/1103719866 >> v2:172.21.15.123:3300/0 conn(0x7f558c0ca060 msgr2=0x7f558c0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.452+0000 7f558aa00640 1 --2- 172.21.15.123:0/1103719866 >> v2:172.21.15.123:3300/0 conn(0x7f558c0ca060 0x7f558c0ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.452+0000 7f558aa00640 1 -- 172.21.15.123:0/1103719866 >> v1:172.21.15.153:6789/0 conn(0x7f558c0b2a10 legacy=0x7f558c0c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.452+0000 7f5589600640 1 -- 172.21.15.123:0/1103719866 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873642714 0 0) 0x7f558c073c10 con 0x7f558c0b2a10 2024-04-25T12:35:20.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.452+0000 7f558aa00640 1 -- 172.21.15.123:0/1103719866 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c074100 con 0x7f558c073460 2024-04-25T12:35:20.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.453+0000 7f558aa00640 1 --2- 172.21.15.123:0/1103719866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c15d270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097a0 tx=0x7f557c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd48f5bfccdbbcf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.453+0000 7f5589600640 1 -- 172.21.15.123:0/1103719866 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c037070 con 0x7f558c073460 2024-04-25T12:35:20.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.453+0000 7f5589600640 1 -- 172.21.15.123:0/1103719866 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c032440 con 0x7f558c073460 2024-04-25T12:35:20.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.453+0000 7f5589600640 1 -- 172.21.15.123:0/1103719866 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c032810 con 0x7f558c073460 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 -- 172.21.15.123:0/1103719866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 msgr2=0x7f558c15d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 --2- 172.21.15.123:0/1103719866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c15d270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097a0 tx=0x7f557c02a180 comp rx=0 tx=0).stop 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 -- 172.21.15.123:0/1103719866 shutdown_connections 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 --2- 172.21.15.123:0/1103719866 >> v2:172.21.15.123:3300/0 conn(0x7f558c0ca060 0x7f558c0ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 --2- 172.21.15.123:0/1103719866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c15d270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 -- 172.21.15.123:0/1103719866 >> 172.21.15.123:0/1103719866 conn(0x7f558c06f440 msgr2=0x7f558c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 -- 172.21.15.123:0/1103719866 shutdown_connections 2024-04-25T12:35:20.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.454+0000 7f5591600640 1 -- 172.21.15.123:0/1103719866 wait complete. 2024-04-25T12:35:20.455 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:20.455 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:20.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.457+0000 7f5591600640 1 Processor -- start 2024-04-25T12:35:20.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.457+0000 7f5591600640 1 -- start start 2024-04-25T12:35:20.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.458+0000 7f5591600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c1e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.458+0000 7f5591600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f558c0b2a10 0x7f558c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.458+0000 7f5591600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c0ca060 0x7f558c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.458+0000 7f5591600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f558c070a20 con 0x7f558c0ca060 2024-04-25T12:35:20.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.458+0000 7f5591600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f558c071310 con 0x7f558c0b2a10 2024-04-25T12:35:20.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.458+0000 7f5591600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f558c0717a0 con 0x7f558c073460 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c0ca060 0x7f558c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c1e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f558c0b2a10 0x7f558c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f558c0b2a10 0x7f558c151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33172/0 (socket says 172.21.15.123:33172) 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c0ca060 0x7f558c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52068/0 (socket says 172.21.15.123:52068) 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c1e7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37552/0 (socket says 172.21.15.123:37552) 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558a000640 1 -- 172.21.15.123:0/443205362 learned_addr learned my addr 172.21.15.123:0/443205362 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558a000640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 msgr2=0x7f558c1e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558a000640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f558c073460 0x7f558c1e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558a000640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c0ca060 msgr2=0x7f558c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.459+0000 7f558a000640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c0ca060 0x7f558c15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.460+0000 7f558a000640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c1f3430 con 0x7f558c0b2a10 2024-04-25T12:35:20.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.460+0000 7f558a000640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f558c0b2a10 0x7f558c151ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f557800c440 tx=0x7f557800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.460+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578018070 con 0x7f558c0b2a10 2024-04-25T12:35:20.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.460+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55780074e0 con 0x7f558c0b2a10 2024-04-25T12:35:20.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.460+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c1e7980 con 0x7f558c0b2a10 2024-04-25T12:35:20.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.460+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578004a20 con 0x7f558c0b2a10 2024-04-25T12:35:20.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.460+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1f36b0 con 0x7f558c0b2a10 2024-04-25T12:35:20.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.461+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f558c15a450 con 0x7f558c0b2a10 2024-04-25T12:35:20.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.464+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f557801b070 con 0x7f558c0b2a10 2024-04-25T12:35:20.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.465+0000 7f5582a00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f556c0744b0 0x7f556c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.465+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6619+0+0 (secure 0 0 0) 0x7f5578094730 con 0x7f558c0b2a10 2024-04-25T12:35:20.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.465+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f556c0744b0 0x7f556c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.465+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v31) ==== 54+0+16 (secure 0 0 0) 0x7f558c15a450 con 0x7f558c0b2a10 2024-04-25T12:35:20.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.466+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f558c159eb0 0x7f558c15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.466+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f558c0c4400 con 0x7f558c159eb0 2024-04-25T12:35:20.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.466+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f556c0744b0 0x7f556c0769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f557c034000 tx=0x7f557c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.466+0000 7f558b400640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f558c159eb0 0x7f558c15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.467+0000 7f558b400640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f558c159eb0 0x7f558c15c1d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.468+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c0c4400 con 0x7f558c159eb0 2024-04-25T12:35:20.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:20.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.487+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c076700 con 0x7f558c159eb0 2024-04-25T12:35:20.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.490+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c076700 con 0x7f558c159eb0 2024-04-25T12:35:20.489 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:20.489 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:20.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.490+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f558c1f5f50 0x7f558c1f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.490+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f558c0c9b90 con 0x7f558c1f5f50 2024-04-25T12:35:20.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.490+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f558c1f5f50 0x7f558c1f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.491+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f558c1f5f50 0x7f558c1f6360 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.492+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c0c9b90 con 0x7f558c1f5f50 2024-04-25T12:35:20.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.509+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c073e00 con 0x7f558c1f5f50 2024-04-25T12:35:20.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.511+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c073e00 con 0x7f558c1f5f50 2024-04-25T12:35:20.510 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:20.511 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:20.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.511+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f558c062fe0 0x7f558c0633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.511+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f558c1f6db0 con 0x7f558c062fe0 2024-04-25T12:35:20.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.511+0000 7f558a000640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f558c062fe0 0x7f558c0633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.512+0000 7f558a000640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f558c062fe0 0x7f558c0633f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.513+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c1f6db0 con 0x7f558c062fe0 2024-04-25T12:35:20.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.528+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c1e8e10 con 0x7f558c062fe0 2024-04-25T12:35:20.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.531+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c1e8e10 con 0x7f558c062fe0 2024-04-25T12:35:20.530 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:20.531 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:20.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.531+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f558c0c7e60 0x7f558c0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.531+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f558c063e40 con 0x7f558c0c7e60 2024-04-25T12:35:20.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.532+0000 7f558b400640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f558c0c7e60 0x7f558c0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.532+0000 7f558b400640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f558c0c7e60 0x7f558c0c8270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.533+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c063e40 con 0x7f558c0c7e60 2024-04-25T12:35:20.543 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:20.544+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:20.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.549+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c0c87b0 con 0x7f558c0c7e60 2024-04-25T12:35:20.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.551+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c0c87b0 con 0x7f558c0c7e60 2024-04-25T12:35:20.550 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:20.551 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:20.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.551+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f558c02c9d0 0x7f558c02cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.551+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f558c0c87b0 con 0x7f558c02c9d0 2024-04-25T12:35:20.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.552+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f558c02c9d0 0x7f558c02cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.553+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f558c02c9d0 0x7f558c02cde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.554+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c0c87b0 con 0x7f558c02c9d0 2024-04-25T12:35:20.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.585+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c0c87b0 con 0x7f558c02c9d0 2024-04-25T12:35:20.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.588+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c0c87b0 con 0x7f558c02c9d0 2024-04-25T12:35:20.587 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:20.587 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:20.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.588+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f558c02d830 0x7f558c2092e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.588+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f558c02dc40 con 0x7f558c02d830 2024-04-25T12:35:20.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.588+0000 7f558a000640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f558c02d830 0x7f558c2092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.589+0000 7f558a000640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f558c02d830 0x7f558c2092e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.590+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c02dc40 con 0x7f558c02d830 2024-04-25T12:35:20.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.601+0000 7f0fe1800640 1 Processor -- start 2024-04-25T12:35:20.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.602+0000 7f0fe1800640 1 -- start start 2024-04-25T12:35:20.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fe1800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c9b80 0x7f0fdc0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fe1800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fdc156e60 con 0x7f0fdc0c8250 2024-04-25T12:35:20.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fe1800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fdc0c7130 con 0x7f0fdc159f30 2024-04-25T12:35:20.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fe1800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0fdc15ca70 con 0x7f0fdc0c9b80 2024-04-25T12:35:20.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fd3e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0fdc0c8250 0x7f0fdc159820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43584/0 (socket says 172.21.15.123:43584) 2024-04-25T12:35:20.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fd3e00640 1 -- 172.21.15.123:0/2168985818 learned_addr learned my addr 172.21.15.123:0/2168985818 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:20.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fdaa00640 1 --2- 172.21.15.123:0/2168985818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c9b80 0x7f0fdc0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fda000640 1 -- 172.21.15.123:0/2168985818 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358368989 0 0) 0x7f0fdc156e60 con 0x7f0fdc0c8250 2024-04-25T12:35:20.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.604+0000 7f0fda000640 1 -- 172.21.15.123:0/2168985818 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fc0003520 con 0x7f0fdc0c8250 2024-04-25T12:35:20.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.605+0000 7f0fda000640 1 -- 172.21.15.123:0/2168985818 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567947659 0 0) 0x7f0fdc0c7130 con 0x7f0fdc159f30 2024-04-25T12:35:20.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.605+0000 7f0fda000640 1 -- 172.21.15.123:0/2168985818 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fdc156e60 con 0x7f0fdc159f30 2024-04-25T12:35:20.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.605+0000 7f0fdaa00640 1 -- 172.21.15.123:0/2168985818 >> v1:172.21.15.153:6789/0 conn(0x7f0fdc159f30 legacy=0x7f0fdc163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.605+0000 7f0fdaa00640 1 -- 172.21.15.123:0/2168985818 >> v1:172.21.15.123:6789/0 conn(0x7f0fdc0c8250 legacy=0x7f0fdc159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.605+0000 7f0fdaa00640 1 -- 172.21.15.123:0/2168985818 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc15cd50 con 0x7f0fdc0c9b80 2024-04-25T12:35:20.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.606+0000 7f0fdaa00640 1 --2- 172.21.15.123:0/2168985818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c9b80 0x7f0fdc0742b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0097d0 tx=0x7f0fcc02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb66eb785679f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.606+0000 7f0fda000640 1 -- 172.21.15.123:0/2168985818 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc037070 con 0x7f0fdc0c9b80 2024-04-25T12:35:20.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.606+0000 7f0fda000640 1 -- 172.21.15.123:0/2168985818 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc032440 con 0x7f0fdc0c9b80 2024-04-25T12:35:20.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.606+0000 7f0fda000640 1 -- 172.21.15.123:0/2168985818 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc032810 con 0x7f0fdc0c9b80 2024-04-25T12:35:20.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.607+0000 7f0fe1800640 1 -- 172.21.15.123:0/2168985818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c9b80 msgr2=0x7f0fdc0742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.607+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2168985818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c9b80 0x7f0fdc0742b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0097d0 tx=0x7f0fcc02a230 comp rx=0 tx=0).stop 2024-04-25T12:35:20.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.607+0000 7f0fe1800640 1 -- 172.21.15.123:0/2168985818 shutdown_connections 2024-04-25T12:35:20.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.607+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2168985818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c9b80 0x7f0fdc0742b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.607+0000 7f0fe1800640 1 -- 172.21.15.123:0/2168985818 >> 172.21.15.123:0/2168985818 conn(0x7f0fdc06c750 msgr2=0x7f0fdc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:20.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.607+0000 7f0fe1800640 1 -- 172.21.15.123:0/2168985818 shutdown_connections 2024-04-25T12:35:20.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.607+0000 7f0fe1800640 1 -- 172.21.15.123:0/2168985818 wait complete. 2024-04-25T12:35:20.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.609+0000 7f0fe1800640 1 Processor -- start 2024-04-25T12:35:20.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.609+0000 7f0fe1800640 1 -- start start 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fe1800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c8250 0x7f0fdc15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fe1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fdc0c9b80 0x7f0fdc14b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fe1800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc159f30 0x7f0fdc14de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fe1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc15d5f0 con 0x7f0fdc0c9b80 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fe1800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc15d310 con 0x7f0fdc159f30 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fe1800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0fdc0747f0 con 0x7f0fdc0c8250 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fd3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fdc0c9b80 0x7f0fdc14b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fd3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fdc0c9b80 0x7f0fdc14b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52082/0 (socket says 172.21.15.123:52082) 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fd3e00640 1 -- 172.21.15.123:0/2770886293 learned_addr learned my addr 172.21.15.123:0/2770886293 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:20.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fdaa00640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c8250 0x7f0fdc15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.611+0000 7f0fdb400640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc159f30 0x7f0fdc14de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.612+0000 7f0fdaa00640 1 -- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc159f30 msgr2=0x7f0fdc14de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.612+0000 7f0fdaa00640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc159f30 0x7f0fdc14de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.612+0000 7f0fdaa00640 1 -- 172.21.15.123:0/2770886293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fdc0c9b80 msgr2=0x7f0fdc14b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.612+0000 7f0fdaa00640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fdc0c9b80 0x7f0fdc14b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.612+0000 7f0fdaa00640 1 -- 172.21.15.123:0/2770886293 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc15b850 con 0x7f0fdc0c8250 2024-04-25T12:35:20.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.612+0000 7f0fdaa00640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c8250 0x7f0fdc15b310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc031000 tx=0x7f0fcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.612+0000 7f0fd2a00640 1 -- 172.21.15.123:0/2770886293 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc041020 con 0x7f0fdc0c8250 2024-04-25T12:35:20.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.613+0000 7f0fd2a00640 1 -- 172.21.15.123:0/2770886293 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc03d720 con 0x7f0fdc0c8250 2024-04-25T12:35:20.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.613+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fdc15bfc0 con 0x7f0fdc0c8250 2024-04-25T12:35:20.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.613+0000 7f0fd2a00640 1 -- 172.21.15.123:0/2770886293 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc037040 con 0x7f0fdc0c8250 2024-04-25T12:35:20.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.613+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fdc15c580 con 0x7f0fdc0c8250 2024-04-25T12:35:20.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.614+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fdc076700 con 0x7f0fdc0c8250 2024-04-25T12:35:20.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.615+0000 7f0fd2a00640 1 -- 172.21.15.123:0/2770886293 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0fcc044020 con 0x7f0fdc0c8250 2024-04-25T12:35:20.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.616+0000 7f0fd2a00640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fb80744b0 0x7f0fb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.616+0000 7f0fd2a00640 1 -- 172.21.15.123:0/2770886293 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 6619+0+0 (secure 0 0 0) 0x7f0fcc0b5c00 con 0x7f0fdc0c8250 2024-04-25T12:35:20.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.616+0000 7f0fd3e00640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fb80744b0 0x7f0fb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.617+0000 7f0fd3e00640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fb80744b0 0x7f0fb80769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc1ed100 tx=0x7f0fc4007db0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.622+0000 7f0fd2a00640 1 -- 172.21.15.123:0/2770886293 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fcc04b030 con 0x7f0fdc0c8250 2024-04-25T12:35:20.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.622+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c02dc40 con 0x7f558c02d830 2024-04-25T12:35:20.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.624+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c02dc40 con 0x7f558c02d830 2024-04-25T12:35:20.624 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:20.624 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:20.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.625+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f558c209d30 0x7f558c20c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.625+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f558c20ef20 con 0x7f558c209d30 2024-04-25T12:35:20.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.625+0000 7f558b400640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f558c209d30 0x7f558c20c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.626+0000 7f558b400640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f558c209d30 0x7f558c20c150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.627+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c20ef20 con 0x7f558c209d30 2024-04-25T12:35:20.645 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:20.645+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:20.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.646+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c20c690 con 0x7f558c209d30 2024-04-25T12:35:20.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.648+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c20c690 con 0x7f558c209d30 2024-04-25T12:35:20.648 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:20.648 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:20.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.649+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f558c20cd10 0x7f558c20f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:20.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.649+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f558c2121b0 con 0x7f558c20cd10 2024-04-25T12:35:20.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.649+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f558c20cd10 0x7f558c20f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:20.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.650+0000 7f558aa00640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f558c20cd10 0x7f558c20f130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:20.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.651+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f558c2121b0 con 0x7f558c20cd10 2024-04-25T12:35:20.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.666+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f558c212180 con 0x7f558c20cd10 2024-04-25T12:35:20.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.668+0000 7f5582a00640 1 -- 172.21.15.123:0/443205362 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f558c212180 con 0x7f558c20cd10 2024-04-25T12:35:20.667 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:20.669 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:20.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f558c159eb0 msgr2=0x7f558c15c1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f558c159eb0 0x7f558c15c1d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f558c1f5f50 msgr2=0x7f558c1f6360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f558c1f5f50 0x7f558c1f6360 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f558c062fe0 msgr2=0x7f558c0633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f558c062fe0 0x7f558c0633f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f558c0c7e60 msgr2=0x7f558c0c8270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f558c0c7e60 0x7f558c0c8270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f558c02c9d0 msgr2=0x7f558c02cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f558c02c9d0 0x7f558c02cde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f558c02d830 msgr2=0x7f558c2092e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f558c02d830 0x7f558c2092e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f558c209d30 msgr2=0x7f558c20c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f558c209d30 0x7f558c20c150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f558c20cd10 msgr2=0x7f558c20f130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f558c20cd10 0x7f558c20f130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f558b400640 1 -- 172.21.15.123:0/443205362 reap_dead start 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f558b400640 1 -- 172.21.15.123:0/443205362 reap_dead start 2024-04-25T12:35:20.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f558b400640 1 -- 172.21.15.123:0/443205362 reap_dead start 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f558b400640 1 -- 172.21.15.123:0/443205362 reap_dead start 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f558b400640 1 -- 172.21.15.123:0/443205362 reap_dead start 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f556c0744b0 msgr2=0x7f556c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f556c0744b0 0x7f556c0769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f557c034000 tx=0x7f557c02e000 comp rx=0 tx=0).stop 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f558c0b2a10 msgr2=0x7f558c151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f558c0b2a10 0x7f558c151ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f557800c440 tx=0x7f557800c900 comp rx=0 tx=0).stop 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 shutdown_connections 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f556c0744b0 0x7f556c0769c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 --2- 172.21.15.123:0/443205362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f558c0b2a10 0x7f558c151ba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 >> 172.21.15.123:0/443205362 conn(0x7f558c06f440 msgr2=0x7f558c15b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.669+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 shutdown_connections 2024-04-25T12:35:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.670+0000 7f5591600640 1 -- 172.21.15.123:0/443205362 wait complete. 2024-04-25T12:35:20.746 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:20.747+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:20.750 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:20.750 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:20.750 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:20.750 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:20.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.821+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0fdc15bd70 con 0x7f0fb80744b0 2024-04-25T12:35:20.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.829+0000 7f0fd2a00640 1 -- 172.21.15.123:0/2770886293 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332752 (secure 0 0 0) 0x7f0fdc15bd70 con 0x7f0fb80744b0 2024-04-25T12:35:20.833 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:20.835 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.834+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fb80744b0 msgr2=0x7f0fb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.834+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fb80744b0 0x7f0fb80769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc1ed100 tx=0x7f0fc4007db0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.834+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c8250 msgr2=0x7f0fdc15b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.834+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c8250 0x7f0fdc15b310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc031000 tx=0x7f0fcc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 shutdown_connections 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fb80744b0 0x7f0fb80769c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc159f30 0x7f0fdc14de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fdc0c9b80 0x7f0fdc14b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 --2- 172.21.15.123:0/2770886293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0fdc0c8250 0x7f0fdc15b310 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:20.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 >> 172.21.15.123:0/2770886293 conn(0x7f0fdc06c750 msgr2=0x7f0fdc0c2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:20.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 shutdown_connections 2024-04-25T12:35:20.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:20.835+0000 7f0fe1800640 1 -- 172.21.15.123:0/2770886293 wait complete. 2024-04-25T12:35:20.847 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:20.848+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:20.859 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-25T12:35:20.810154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600155,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":235,"num_write_kb":1677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":189,"ondisk_log_size":189,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218900,"kb_used_data":2720,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755820,"statfs":{"total":773094113280,"available":772869959680,"internally_reserved":0,"allocated":2785280,"data_stored":2255722,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2077,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001495"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.314450+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:35:15.314450+0000","last_peered":"2024-04-25T12:35:15.314450+0000","last_clean":"2024-04-25T12:35:15.314450+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:35:15.314450+0000","last_undegraded":"2024-04-25T12:35:15.314450+0000","last_fullsized":"2024-04-25T12:35:15.314450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327331+0000","last_change":"2024-04-25T12:34:13.937842+0000","last_active":"2024-04-25T12:35:16.327331+0000","last_peered":"2024-04-25T12:35:16.327331+0000","last_clean":"2024-04-25T12:35:16.327331+0000","last_became_active":"2024-04-25T12:34:13.937503+0000","last_became_peered":"2024-04-25T12:34:13.937503+0000","last_unstale":"2024-04-25T12:35:16.327331+0000","last_undegraded":"2024-04-25T12:35:16.327331+0000","last_fullsized":"2024-04-25T12:35:16.327331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:23.021790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497045+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:35:16.497045+0000","last_peered":"2024-04-25T12:35:16.497045+0000","last_clean":"2024-04-25T12:35:16.497045+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:35:16.497045+0000","last_undegraded":"2024-04-25T12:35:16.497045+0000","last_fullsized":"2024-04-25T12:35:16.497045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327420+0000","last_change":"2024-04-25T12:34:12.904117+0000","last_active":"2024-04-25T12:35:16.327420+0000","last_peered":"2024-04-25T12:35:16.327420+0000","last_clean":"2024-04-25T12:35:16.327420+0000","last_became_active":"2024-04-25T12:34:12.903476+0000","last_became_peered":"2024-04-25T12:34:12.903476+0000","last_unstale":"2024-04-25T12:35:16.327420+0000","last_undegraded":"2024-04-25T12:35:16.327420+0000","last_fullsized":"2024-04-25T12:35:16.327420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326657+0000","last_change":"2024-04-25T12:34:13.951539+0000","last_active":"2024-04-25T12:35:16.326657+0000","last_peered":"2024-04-25T12:35:16.326657+0000","last_clean":"2024-04-25T12:35:16.326657+0000","last_became_active":"2024-04-25T12:34:13.951211+0000","last_became_peered":"2024-04-25T12:34:13.951211+0000","last_unstale":"2024-04-25T12:35:16.326657+0000","last_undegraded":"2024-04-25T12:35:16.326657+0000","last_fullsized":"2024-04-25T12:35:16.326657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.494767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.321897+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:35:16.321897+0000","last_peered":"2024-04-25T12:35:16.321897+0000","last_clean":"2024-04-25T12:35:16.321897+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:35:16.321897+0000","last_undegraded":"2024-04-25T12:35:16.321897+0000","last_fullsized":"2024-04-25T12:35:16.321897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.468281+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:35:16.468281+0000","last_peered":"2024-04-25T12:35:16.468281+0000","last_clean":"2024-04-25T12:35:16.468281+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:35:16.468281+0000","last_undegraded":"2024-04-25T12:35:16.468281+0000","last_fullsized":"2024-04-25T12:35:16.468281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.325283+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:35:16.325283+0000","last_peered":"2024-04-25T12:35:16.325283+0000","last_clean":"2024-04-25T12:35:16.325283+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:35:16.325283+0000","last_undegraded":"2024-04-25T12:35:16.325283+0000","last_fullsized":"2024-04-25T12:35:16.325283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324256+0000","last_change":"2024-04-25T12:34:18.008750+0000","last_active":"2024-04-25T12:35:16.324256+0000","last_peered":"2024-04-25T12:35:16.324256+0000","last_clean":"2024-04-25T12:35:16.324256+0000","last_became_active":"2024-04-25T12:34:18.006955+0000","last_became_peered":"2024-04-25T12:34:18.006955+0000","last_unstale":"2024-04-25T12:35:16.324256+0000","last_undegraded":"2024-04-25T12:35:16.324256+0000","last_fullsized":"2024-04-25T12:35:16.324256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467332+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:35:16.467332+0000","last_peered":"2024-04-25T12:35:16.467332+0000","last_clean":"2024-04-25T12:35:16.467332+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:35:16.467332+0000","last_undegraded":"2024-04-25T12:35:16.467332+0000","last_fullsized":"2024-04-25T12:35:16.467332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.321477+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:35:16.321477+0000","last_peered":"2024-04-25T12:35:16.321477+0000","last_clean":"2024-04-25T12:35:16.321477+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:35:16.321477+0000","last_undegraded":"2024-04-25T12:35:16.321477+0000","last_fullsized":"2024-04-25T12:35:16.321477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.323610+0000","last_change":"2024-04-25T12:34:18.007053+0000","last_active":"2024-04-25T12:35:16.323610+0000","last_peered":"2024-04-25T12:35:16.323610+0000","last_clean":"2024-04-25T12:35:16.323610+0000","last_became_active":"2024-04-25T12:34:18.004052+0000","last_became_peered":"2024-04-25T12:34:18.004052+0000","last_unstale":"2024-04-25T12:35:16.323610+0000","last_undegraded":"2024-04-25T12:35:16.323610+0000","last_fullsized":"2024-04-25T12:35:16.323610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.319551+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:35:16.319551+0000","last_peered":"2024-04-25T12:35:16.319551+0000","last_clean":"2024-04-25T12:35:16.319551+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:35:16.319551+0000","last_undegraded":"2024-04-25T12:35:16.319551+0000","last_fullsized":"2024-04-25T12:35:16.319551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.320318+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:35:16.320318+0000","last_peered":"2024-04-25T12:35:16.320318+0000","last_clean":"2024-04-25T12:35:16.320318+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:35:16.320318+0000","last_undegraded":"2024-04-25T12:35:16.320318+0000","last_fullsized":"2024-04-25T12:35:16.320318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467426+0000","last_change":"2024-04-25T12:35:00.703950+0000","last_active":"2024-04-25T12:35:16.467426+0000","last_peered":"2024-04-25T12:35:16.467426+0000","last_clean":"2024-04-25T12:35:16.467426+0000","last_became_active":"2024-04-25T12:35:00.702548+0000","last_became_peered":"2024-04-25T12:35:00.702548+0000","last_unstale":"2024-04-25T12:35:16.467426+0000","last_undegraded":"2024-04-25T12:35:16.467426+0000","last_fullsized":"2024-04-25T12:35:16.467426+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497311+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:35:16.497311+0000","last_peered":"2024-04-25T12:35:16.497311+0000","last_clean":"2024-04-25T12:35:16.497311+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:35:16.497311+0000","last_undegraded":"2024-04-25T12:35:16.497311+0000","last_fullsized":"2024-04-25T12:35:16.497311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.393127+0000","last_change":"2024-04-25T12:34:12.910217+0000","last_active":"2024-04-25T12:35:14.393127+0000","last_peered":"2024-04-25T12:35:14.393127+0000","last_clean":"2024-04-25T12:35:14.393127+0000","last_became_active":"2024-04-25T12:34:12.909909+0000","last_became_peered":"2024-04-25T12:34:12.909909+0000","last_unstale":"2024-04-25T12:35:14.393127+0000","last_undegraded":"2024-04-25T12:35:14.393127+0000","last_fullsized":"2024-04-25T12:35:14.393127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.466788+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:35:16.466788+0000","last_peered":"2024-04-25T12:35:16.466788+0000","last_clean":"2024-04-25T12:35:16.466788+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:35:16.466788+0000","last_undegraded":"2024-04-25T12:35:16.466788+0000","last_fullsized":"2024-04-25T12:35:16.466788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313192+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:35:15.313192+0000","last_peered":"2024-04-25T12:35:15.313192+0000","last_clean":"2024-04-25T12:35:15.313192+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:35:15.313192+0000","last_undegraded":"2024-04-25T12:35:15.313192+0000","last_fullsized":"2024-04-25T12:35:15.313192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313834+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:35:15.313834+0000","last_peered":"2024-04-25T12:35:15.313834+0000","last_clean":"2024-04-25T12:35:15.313834+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:35:15.313834+0000","last_undegraded":"2024-04-25T12:35:15.313834+0000","last_fullsized":"2024-04-25T12:35:15.313834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497617+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:35:16.497617+0000","last_peered":"2024-04-25T12:35:16.497617+0000","last_clean":"2024-04-25T12:35:16.497617+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:35:16.497617+0000","last_undegraded":"2024-04-25T12:35:16.497617+0000","last_fullsized":"2024-04-25T12:35:16.497617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.318895+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:35:16.318895+0000","last_peered":"2024-04-25T12:35:16.318895+0000","last_clean":"2024-04-25T12:35:16.318895+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:35:16.318895+0000","last_undegraded":"2024-04-25T12:35:16.318895+0000","last_fullsized":"2024-04-25T12:35:16.318895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.466154+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:35:16.466154+0000","last_peered":"2024-04-25T12:35:16.466154+0000","last_clean":"2024-04-25T12:35:16.466154+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:35:16.466154+0000","last_undegraded":"2024-04-25T12:35:16.466154+0000","last_fullsized":"2024-04-25T12:35:16.466154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.325633+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:35:16.325633+0000","last_peered":"2024-04-25T12:35:16.325633+0000","last_clean":"2024-04-25T12:35:16.325633+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:35:16.325633+0000","last_undegraded":"2024-04-25T12:35:16.325633+0000","last_fullsized":"2024-04-25T12:35:16.325633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313948+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:35:15.313948+0000","last_peered":"2024-04-25T12:35:15.313948+0000","last_clean":"2024-04-25T12:35:15.313948+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:35:15.313948+0000","last_undegraded":"2024-04-25T12:35:15.313948+0000","last_fullsized":"2024-04-25T12:35:15.313948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.323952+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:35:16.323952+0000","last_peered":"2024-04-25T12:35:16.323952+0000","last_clean":"2024-04-25T12:35:16.323952+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:35:16.323952+0000","last_undegraded":"2024-04-25T12:35:16.323952+0000","last_fullsized":"2024-04-25T12:35:16.323952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.323226+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:35:16.323226+0000","last_peered":"2024-04-25T12:35:16.323226+0000","last_clean":"2024-04-25T12:35:16.323226+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:35:16.323226+0000","last_undegraded":"2024-04-25T12:35:16.323226+0000","last_fullsized":"2024-04-25T12:35:16.323226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313531+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:35:15.313531+0000","last_peered":"2024-04-25T12:35:15.313531+0000","last_clean":"2024-04-25T12:35:15.313531+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:35:15.313531+0000","last_undegraded":"2024-04-25T12:35:15.313531+0000","last_fullsized":"2024-04-25T12:35:15.313531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"27'4","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.314284+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:35:15.314284+0000","last_peered":"2024-04-25T12:35:15.314284+0000","last_clean":"2024-04-25T12:35:15.314284+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:35:15.314284+0000","last_undegraded":"2024-04-25T12:35:15.314284+0000","last_fullsized":"2024-04-25T12:35:15.314284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313058+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:35:15.313058+0000","last_peered":"2024-04-25T12:35:15.313058+0000","last_clean":"2024-04-25T12:35:15.313058+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:35:15.313058+0000","last_undegraded":"2024-04-25T12:35:15.313058+0000","last_fullsized":"2024-04-25T12:35:15.313058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.393461+0000","last_change":"2024-04-25T12:34:13.936045+0000","last_active":"2024-04-25T12:35:14.393461+0000","last_peered":"2024-04-25T12:35:14.393461+0000","last_clean":"2024-04-25T12:35:14.393461+0000","last_became_active":"2024-04-25T12:34:13.935733+0000","last_became_peered":"2024-04-25T12:34:13.935733+0000","last_unstale":"2024-04-25T12:35:14.393461+0000","last_undegraded":"2024-04-25T12:35:14.393461+0000","last_fullsized":"2024-04-25T12:35:14.393461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:48.538137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.323156+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:35:16.323156+0000","last_peered":"2024-04-25T12:35:16.323156+0000","last_clean":"2024-04-25T12:35:16.323156+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:35:16.323156+0000","last_undegraded":"2024-04-25T12:35:16.323156+0000","last_fullsized":"2024-04-25T12:35:16.323156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392318+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:35:14.392318+0000","last_peered":"2024-04-25T12:35:14.392318+0000","last_clean":"2024-04-25T12:35:14.392318+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:35:14.392318+0000","last_undegraded":"2024-04-25T12:35:14.392318+0000","last_fullsized":"2024-04-25T12:35:14.392318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324882+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:35:16.324882+0000","last_peered":"2024-04-25T12:35:16.324882+0000","last_clean":"2024-04-25T12:35:16.324882+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:35:16.324882+0000","last_undegraded":"2024-04-25T12:35:16.324882+0000","last_fullsized":"2024-04-25T12:35:16.324882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326679+0000","last_change":"2024-04-25T12:34:12.904885+0000","last_active":"2024-04-25T12:35:16.326679+0000","last_peered":"2024-04-25T12:35:16.326679+0000","last_clean":"2024-04-25T12:35:16.326679+0000","last_became_active":"2024-04-25T12:34:12.903426+0000","last_became_peered":"2024-04-25T12:34:12.903426+0000","last_unstale":"2024-04-25T12:35:16.326679+0000","last_undegraded":"2024-04-25T12:35:16.326679+0000","last_fullsized":"2024-04-25T12:35:16.326679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392335+0000","last_change":"2024-04-25T12:34:17.999253+0000","last_active":"2024-04-25T12:35:14.392335+0000","last_peered":"2024-04-25T12:35:14.392335+0000","last_clean":"2024-04-25T12:35:14.392335+0000","last_became_active":"2024-04-25T12:34:17.998629+0000","last_became_peered":"2024-04-25T12:34:17.998629+0000","last_unstale":"2024-04-25T12:35:14.392335+0000","last_undegraded":"2024-04-25T12:35:14.392335+0000","last_fullsized":"2024-04-25T12:35:14.392335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:12.691411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.466999+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:35:16.466999+0000","last_peered":"2024-04-25T12:35:16.466999+0000","last_clean":"2024-04-25T12:35:16.466999+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:35:16.466999+0000","last_undegraded":"2024-04-25T12:35:16.466999+0000","last_fullsized":"2024-04-25T12:35:16.466999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.320118+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:35:16.320118+0000","last_peered":"2024-04-25T12:35:16.320118+0000","last_clean":"2024-04-25T12:35:16.320118+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:35:16.320118+0000","last_undegraded":"2024-04-25T12:35:16.320118+0000","last_fullsized":"2024-04-25T12:35:16.320118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392627+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:35:14.392627+0000","last_peered":"2024-04-25T12:35:14.392627+0000","last_clean":"2024-04-25T12:35:14.392627+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:35:14.392627+0000","last_undegraded":"2024-04-25T12:35:14.392627+0000","last_fullsized":"2024-04-25T12:35:14.392627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326546+0000","last_change":"2024-04-25T12:34:13.934104+0000","last_active":"2024-04-25T12:35:16.326546+0000","last_peered":"2024-04-25T12:35:16.326546+0000","last_clean":"2024-04-25T12:35:16.326546+0000","last_became_active":"2024-04-25T12:34:13.933374+0000","last_became_peered":"2024-04-25T12:34:13.933374+0000","last_unstale":"2024-04-25T12:35:16.326546+0000","last_undegraded":"2024-04-25T12:35:16.326546+0000","last_fullsized":"2024-04-25T12:35:16.326546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497175+0000","last_change":"2024-04-25T12:34:12.903744+0000","last_active":"2024-04-25T12:35:16.497175+0000","last_peered":"2024-04-25T12:35:16.497175+0000","last_clean":"2024-04-25T12:35:16.497175+0000","last_became_active":"2024-04-25T12:34:12.903405+0000","last_became_peered":"2024-04-25T12:34:12.903405+0000","last_unstale":"2024-04-25T12:35:16.497175+0000","last_undegraded":"2024-04-25T12:35:16.497175+0000","last_fullsized":"2024-04-25T12:35:16.497175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392386+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:35:14.392386+0000","last_peered":"2024-04-25T12:35:14.392386+0000","last_clean":"2024-04-25T12:35:14.392386+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:35:14.392386+0000","last_undegraded":"2024-04-25T12:35:14.392386+0000","last_fullsized":"2024-04-25T12:35:14.392386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392847+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:35:14.392847+0000","last_peered":"2024-04-25T12:35:14.392847+0000","last_clean":"2024-04-25T12:35:14.392847+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:35:14.392847+0000","last_undegraded":"2024-04-25T12:35:14.392847+0000","last_fullsized":"2024-04-25T12:35:14.392847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.318368+0000","last_change":"2024-04-25T12:34:12.915437+0000","last_active":"2024-04-25T12:35:16.318368+0000","last_peered":"2024-04-25T12:35:16.318368+0000","last_clean":"2024-04-25T12:35:16.318368+0000","last_became_active":"2024-04-25T12:34:12.911611+0000","last_became_peered":"2024-04-25T12:34:12.911611+0000","last_unstale":"2024-04-25T12:35:16.318368+0000","last_undegraded":"2024-04-25T12:35:16.318368+0000","last_fullsized":"2024-04-25T12:35:16.318368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327058+0000","last_change":"2024-04-25T12:34:18.006131+0000","last_active":"2024-04-25T12:35:16.327058+0000","last_peered":"2024-04-25T12:35:16.327058+0000","last_clean":"2024-04-25T12:35:16.327058+0000","last_became_active":"2024-04-25T12:34:18.005003+0000","last_became_peered":"2024-04-25T12:34:18.005003+0000","last_unstale":"2024-04-25T12:35:16.327058+0000","last_undegraded":"2024-04-25T12:35:16.327058+0000","last_fullsized":"2024-04-25T12:35:16.327058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.765352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313182+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:35:15.313182+0000","last_peered":"2024-04-25T12:35:15.313182+0000","last_clean":"2024-04-25T12:35:15.313182+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:35:15.313182+0000","last_undegraded":"2024-04-25T12:35:15.313182+0000","last_fullsized":"2024-04-25T12:35:15.313182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.320317+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:35:16.320317+0000","last_peered":"2024-04-25T12:35:16.320317+0000","last_clean":"2024-04-25T12:35:16.320317+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:35:16.320317+0000","last_undegraded":"2024-04-25T12:35:16.320317+0000","last_fullsized":"2024-04-25T12:35:16.320317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":127,"reported_epoch":30,"state":"activating","last_fresh":"2024-04-25T12:35:16.322823+0000","last_change":"2024-04-25T12:35:16.323546+0000","last_active":"2024-04-25T12:35:15.316662+0000","last_peered":"2024-04-25T12:35:14.021587+0000","last_clean":"2024-04-25T12:35:14.021587+0000","last_became_active":"2024-04-25T12:34:08.559310+0000","last_became_peered":"2024-04-25T12:34:08.559310+0000","last_unstale":"2024-04-25T12:35:16.322823+0000","last_undegraded":"2024-04-25T12:35:16.322823+0000","last_fullsized":"2024-04-25T12:35:16.322823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","5"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.322043+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:35:16.322043+0000","last_peered":"2024-04-25T12:35:16.322043+0000","last_clean":"2024-04-25T12:35:16.322043+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:35:16.322043+0000","last_undegraded":"2024-04-25T12:35:16.322043+0000","last_fullsized":"2024-04-25T12:35:16.322043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.321764+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:35:16.321764+0000","last_peered":"2024-04-25T12:35:16.321764+0000","last_clean":"2024-04-25T12:35:16.321764+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:35:16.321764+0000","last_undegraded":"2024-04-25T12:35:16.321764+0000","last_fullsized":"2024-04-25T12:35:16.321764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.393485+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:35:14.393485+0000","last_peered":"2024-04-25T12:35:14.393485+0000","last_clean":"2024-04-25T12:35:14.393485+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:35:14.393485+0000","last_undegraded":"2024-04-25T12:35:14.393485+0000","last_fullsized":"2024-04-25T12:35:14.393485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.323859+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:35:16.323859+0000","last_peered":"2024-04-25T12:35:16.323859+0000","last_clean":"2024-04-25T12:35:16.323859+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:35:16.323859+0000","last_undegraded":"2024-04-25T12:35:16.323859+0000","last_fullsized":"2024-04-25T12:35:16.323859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467097+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:35:16.467097+0000","last_peered":"2024-04-25T12:35:16.467097+0000","last_clean":"2024-04-25T12:35:16.467097+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:35:16.467097+0000","last_undegraded":"2024-04-25T12:35:16.467097+0000","last_fullsized":"2024-04-25T12:35:16.467097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327193+0000","last_change":"2024-04-25T12:34:12.924924+0000","last_active":"2024-04-25T12:35:16.327193+0000","last_peered":"2024-04-25T12:35:16.327193+0000","last_clean":"2024-04-25T12:35:16.327193+0000","last_became_active":"2024-04-25T12:34:12.924628+0000","last_became_peered":"2024-04-25T12:34:12.924628+0000","last_unstale":"2024-04-25T12:35:16.327193+0000","last_undegraded":"2024-04-25T12:35:16.327193+0000","last_fullsized":"2024-04-25T12:35:16.327193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327107+0000","last_change":"2024-04-25T12:34:18.017776+0000","last_active":"2024-04-25T12:35:16.327107+0000","last_peered":"2024-04-25T12:35:16.327107+0000","last_clean":"2024-04-25T12:35:16.327107+0000","last_became_active":"2024-04-25T12:34:18.017283+0000","last_became_peered":"2024-04-25T12:34:18.017283+0000","last_unstale":"2024-04-25T12:35:16.327107+0000","last_undegraded":"2024-04-25T12:35:16.327107+0000","last_fullsized":"2024-04-25T12:35:16.327107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.017317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467928+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:35:16.467928+0000","last_peered":"2024-04-25T12:35:16.467928+0000","last_clean":"2024-04-25T12:35:16.467928+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:35:16.467928+0000","last_undegraded":"2024-04-25T12:35:16.467928+0000","last_fullsized":"2024-04-25T12:35:16.467928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.314086+0000","last_change":"2024-04-25T12:34:12.904478+0000","last_active":"2024-04-25T12:35:15.314086+0000","last_peered":"2024-04-25T12:35:15.314086+0000","last_clean":"2024-04-25T12:35:15.314086+0000","last_became_active":"2024-04-25T12:34:12.904144+0000","last_became_peered":"2024-04-25T12:34:12.904144+0000","last_unstale":"2024-04-25T12:35:15.314086+0000","last_undegraded":"2024-04-25T12:35:15.314086+0000","last_fullsized":"2024-04-25T12:35:15.314086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:43.315936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.318057+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:35:16.318057+0000","last_peered":"2024-04-25T12:35:16.318057+0000","last_clean":"2024-04-25T12:35:16.318057+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:35:16.318057+0000","last_undegraded":"2024-04-25T12:35:16.318057+0000","last_fullsized":"2024-04-25T12:35:16.318057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.325901+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:35:16.325901+0000","last_peered":"2024-04-25T12:35:16.325901+0000","last_clean":"2024-04-25T12:35:16.325901+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:35:16.325901+0000","last_undegraded":"2024-04-25T12:35:16.325901+0000","last_fullsized":"2024-04-25T12:35:16.325901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.468366+0000","last_change":"2024-04-25T12:34:12.917254+0000","last_active":"2024-04-25T12:35:16.468366+0000","last_peered":"2024-04-25T12:35:16.468366+0000","last_clean":"2024-04-25T12:35:16.468366+0000","last_became_active":"2024-04-25T12:34:12.916875+0000","last_became_peered":"2024-04-25T12:34:12.916875+0000","last_unstale":"2024-04-25T12:35:16.468366+0000","last_undegraded":"2024-04-25T12:35:16.468366+0000","last_fullsized":"2024-04-25T12:35:16.468366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:57.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324431+0000","last_change":"2024-04-25T12:34:18.008742+0000","last_active":"2024-04-25T12:35:16.324431+0000","last_peered":"2024-04-25T12:35:16.324431+0000","last_clean":"2024-04-25T12:35:16.324431+0000","last_became_active":"2024-04-25T12:34:18.008275+0000","last_became_peered":"2024-04-25T12:34:18.008275+0000","last_unstale":"2024-04-25T12:35:16.324431+0000","last_undegraded":"2024-04-25T12:35:16.324431+0000","last_fullsized":"2024-04-25T12:35:16.324431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.612547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.319041+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:35:16.319041+0000","last_peered":"2024-04-25T12:35:16.319041+0000","last_clean":"2024-04-25T12:35:16.319041+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:35:16.319041+0000","last_undegraded":"2024-04-25T12:35:16.319041+0000","last_fullsized":"2024-04-25T12:35:16.319041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313345+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:35:15.313345+0000","last_peered":"2024-04-25T12:35:15.313345+0000","last_clean":"2024-04-25T12:35:15.313345+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:35:15.313345+0000","last_undegraded":"2024-04-25T12:35:15.313345+0000","last_fullsized":"2024-04-25T12:35:15.313345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327196+0000","last_change":"2024-04-25T12:34:18.005742+0000","last_active":"2024-04-25T12:35:16.327196+0000","last_peered":"2024-04-25T12:35:16.327196+0000","last_clean":"2024-04-25T12:35:16.327196+0000","last_became_active":"2024-04-25T12:34:18.005106+0000","last_became_peered":"2024-04-25T12:34:18.005106+0000","last_unstale":"2024-04-25T12:35:16.327196+0000","last_undegraded":"2024-04-25T12:35:16.327196+0000","last_fullsized":"2024-04-25T12:35:16.327196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:05.689760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.322577+0000","last_change":"2024-04-25T12:34:13.950427+0000","last_active":"2024-04-25T12:35:16.322577+0000","last_peered":"2024-04-25T12:35:16.322577+0000","last_clean":"2024-04-25T12:35:16.322577+0000","last_became_active":"2024-04-25T12:34:13.946066+0000","last_became_peered":"2024-04-25T12:34:13.946066+0000","last_unstale":"2024-04-25T12:35:16.322577+0000","last_undegraded":"2024-04-25T12:35:16.322577+0000","last_fullsized":"2024-04-25T12:35:16.322577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.466667+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:35:16.466667+0000","last_peered":"2024-04-25T12:35:16.466667+0000","last_clean":"2024-04-25T12:35:16.466667+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:35:16.466667+0000","last_undegraded":"2024-04-25T12:35:16.466667+0000","last_fullsized":"2024-04-25T12:35:16.466667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326625+0000","last_change":"2024-04-25T12:34:18.018911+0000","last_active":"2024-04-25T12:35:16.326625+0000","last_peered":"2024-04-25T12:35:16.326625+0000","last_clean":"2024-04-25T12:35:16.326625+0000","last_became_active":"2024-04-25T12:34:18.018263+0000","last_became_peered":"2024-04-25T12:34:18.018263+0000","last_unstale":"2024-04-25T12:35:16.326625+0000","last_undegraded":"2024-04-25T12:35:16.326625+0000","last_fullsized":"2024-04-25T12:35:16.326625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:57.422375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.319917+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:35:16.319917+0000","last_peered":"2024-04-25T12:35:16.319917+0000","last_clean":"2024-04-25T12:35:16.319917+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:35:16.319917+0000","last_undegraded":"2024-04-25T12:35:16.319917+0000","last_fullsized":"2024-04-25T12:35:16.319917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.319949+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:35:16.319949+0000","last_peered":"2024-04-25T12:35:16.319949+0000","last_clean":"2024-04-25T12:35:16.319949+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:35:16.319949+0000","last_undegraded":"2024-04-25T12:35:16.319949+0000","last_fullsized":"2024-04-25T12:35:16.319949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392577+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:35:14.392577+0000","last_peered":"2024-04-25T12:35:14.392577+0000","last_clean":"2024-04-25T12:35:14.392577+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:35:14.392577+0000","last_undegraded":"2024-04-25T12:35:14.392577+0000","last_fullsized":"2024-04-25T12:35:14.392577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392973+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:35:14.392973+0000","last_peered":"2024-04-25T12:35:14.392973+0000","last_clean":"2024-04-25T12:35:14.392973+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:35:14.392973+0000","last_undegraded":"2024-04-25T12:35:14.392973+0000","last_fullsized":"2024-04-25T12:35:14.392973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497459+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:35:16.497459+0000","last_peered":"2024-04-25T12:35:16.497459+0000","last_clean":"2024-04-25T12:35:16.497459+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:35:16.497459+0000","last_undegraded":"2024-04-25T12:35:16.497459+0000","last_fullsized":"2024-04-25T12:35:16.497459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.321155+0000","last_change":"2024-04-25T12:34:18.011897+0000","last_active":"2024-04-25T12:35:16.321155+0000","last_peered":"2024-04-25T12:35:16.321155+0000","last_clean":"2024-04-25T12:35:16.321155+0000","last_became_active":"2024-04-25T12:34:18.010833+0000","last_became_peered":"2024-04-25T12:34:18.010833+0000","last_unstale":"2024-04-25T12:35:16.321155+0000","last_undegraded":"2024-04-25T12:35:16.321155+0000","last_fullsized":"2024-04-25T12:35:16.321155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:02.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497427+0000","last_change":"2024-04-25T12:35:00.239744+0000","last_active":"2024-04-25T12:35:16.497427+0000","last_peered":"2024-04-25T12:35:16.497427+0000","last_clean":"2024-04-25T12:35:16.497427+0000","last_became_active":"2024-04-25T12:35:00.239414+0000","last_became_peered":"2024-04-25T12:35:00.239414+0000","last_unstale":"2024-04-25T12:35:16.497427+0000","last_undegraded":"2024-04-25T12:35:16.497427+0000","last_fullsized":"2024-04-25T12:35:16.497427+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:17.015606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'15","reported_seq":45,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324568+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:35:16.324568+0000","last_peered":"2024-04-25T12:35:16.324568+0000","last_clean":"2024-04-25T12:35:16.324568+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:35:16.324568+0000","last_undegraded":"2024-04-25T12:35:16.324568+0000","last_fullsized":"2024-04-25T12:35:16.324568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6177,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.312902+0000","last_change":"2024-04-25T12:34:18.006677+0000","last_active":"2024-04-25T12:35:15.312902+0000","last_peered":"2024-04-25T12:35:15.312902+0000","last_clean":"2024-04-25T12:35:15.312902+0000","last_became_active":"2024-04-25T12:34:18.006227+0000","last_became_peered":"2024-04-25T12:34:18.006227+0000","last_unstale":"2024-04-25T12:35:15.312902+0000","last_undegraded":"2024-04-25T12:35:15.312902+0000","last_fullsized":"2024-04-25T12:35:15.312902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:21.090016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.319756+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:35:16.319756+0000","last_peered":"2024-04-25T12:35:16.319756+0000","last_clean":"2024-04-25T12:35:16.319756+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:35:16.319756+0000","last_undegraded":"2024-04-25T12:35:16.319756+0000","last_fullsized":"2024-04-25T12:35:16.319756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497676+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:35:16.497676+0000","last_peered":"2024-04-25T12:35:16.497676+0000","last_clean":"2024-04-25T12:35:16.497676+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:35:16.497676+0000","last_undegraded":"2024-04-25T12:35:16.497676+0000","last_fullsized":"2024-04-25T12:35:16.497676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324630+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:35:16.324630+0000","last_peered":"2024-04-25T12:35:16.324630+0000","last_clean":"2024-04-25T12:35:16.324630+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:35:16.324630+0000","last_undegraded":"2024-04-25T12:35:16.324630+0000","last_fullsized":"2024-04-25T12:35:16.324630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326820+0000","last_change":"2024-04-25T12:34:13.951029+0000","last_active":"2024-04-25T12:35:16.326820+0000","last_peered":"2024-04-25T12:35:16.326820+0000","last_clean":"2024-04-25T12:35:16.326820+0000","last_became_active":"2024-04-25T12:34:13.950648+0000","last_became_peered":"2024-04-25T12:34:13.950648+0000","last_unstale":"2024-04-25T12:35:16.326820+0000","last_undegraded":"2024-04-25T12:35:16.326820+0000","last_fullsized":"2024-04-25T12:35:16.326820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:58.195667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.468131+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:35:16.468131+0000","last_peered":"2024-04-25T12:35:16.468131+0000","last_clean":"2024-04-25T12:35:16.468131+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:35:16.468131+0000","last_undegraded":"2024-04-25T12:35:16.468131+0000","last_fullsized":"2024-04-25T12:35:16.468131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.496983+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:35:16.496983+0000","last_peered":"2024-04-25T12:35:16.496983+0000","last_clean":"2024-04-25T12:35:16.496983+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:35:16.496983+0000","last_undegraded":"2024-04-25T12:35:16.496983+0000","last_fullsized":"2024-04-25T12:35:16.496983+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.325901+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:35:16.325901+0000","last_peered":"2024-04-25T12:35:16.325901+0000","last_clean":"2024-04-25T12:35:16.325901+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:35:16.325901+0000","last_undegraded":"2024-04-25T12:35:16.325901+0000","last_fullsized":"2024-04-25T12:35:16.325901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324952+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:35:16.324952+0000","last_peered":"2024-04-25T12:35:16.324952+0000","last_clean":"2024-04-25T12:35:16.324952+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:35:16.324952+0000","last_undegraded":"2024-04-25T12:35:16.324952+0000","last_fullsized":"2024-04-25T12:35:16.324952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313432+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:35:15.313432+0000","last_peered":"2024-04-25T12:35:15.313432+0000","last_clean":"2024-04-25T12:35:15.313432+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:35:15.313432+0000","last_undegraded":"2024-04-25T12:35:15.313432+0000","last_fullsized":"2024-04-25T12:35:15.313432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.314389+0000","last_change":"2024-04-25T12:34:18.005192+0000","last_active":"2024-04-25T12:35:15.314389+0000","last_peered":"2024-04-25T12:35:15.314389+0000","last_clean":"2024-04-25T12:35:15.314389+0000","last_became_active":"2024-04-25T12:34:18.004696+0000","last_became_peered":"2024-04-25T12:34:18.004696+0000","last_unstale":"2024-04-25T12:35:15.314389+0000","last_undegraded":"2024-04-25T12:35:15.314389+0000","last_fullsized":"2024-04-25T12:35:15.314389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:38.296490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313705+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:35:15.313705+0000","last_peered":"2024-04-25T12:35:15.313705+0000","last_clean":"2024-04-25T12:35:15.313705+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:35:15.313705+0000","last_undegraded":"2024-04-25T12:35:15.313705+0000","last_fullsized":"2024-04-25T12:35:15.313705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.318367+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:35:16.318367+0000","last_peered":"2024-04-25T12:35:16.318367+0000","last_clean":"2024-04-25T12:35:16.318367+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:35:16.318367+0000","last_undegraded":"2024-04-25T12:35:16.318367+0000","last_fullsized":"2024-04-25T12:35:16.318367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.323759+0000","last_change":"2024-04-25T12:34:18.011597+0000","last_active":"2024-04-25T12:35:16.323759+0000","last_peered":"2024-04-25T12:35:16.323759+0000","last_clean":"2024-04-25T12:35:16.323759+0000","last_became_active":"2024-04-25T12:34:18.011122+0000","last_became_peered":"2024-04-25T12:34:18.011122+0000","last_unstale":"2024-04-25T12:35:16.323759+0000","last_undegraded":"2024-04-25T12:35:16.323759+0000","last_fullsized":"2024-04-25T12:35:16.323759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.182138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326684+0000","last_change":"2024-04-25T12:34:13.934383+0000","last_active":"2024-04-25T12:35:16.326684+0000","last_peered":"2024-04-25T12:35:16.326684+0000","last_clean":"2024-04-25T12:35:16.326684+0000","last_became_active":"2024-04-25T12:34:13.933787+0000","last_became_peered":"2024-04-25T12:34:13.933787+0000","last_unstale":"2024-04-25T12:35:16.326684+0000","last_undegraded":"2024-04-25T12:35:16.326684+0000","last_fullsized":"2024-04-25T12:35:16.326684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392881+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:35:14.392881+0000","last_peered":"2024-04-25T12:35:14.392881+0000","last_clean":"2024-04-25T12:35:14.392881+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:35:14.392881+0000","last_undegraded":"2024-04-25T12:35:14.392881+0000","last_fullsized":"2024-04-25T12:35:14.392881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.319787+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:35:16.319787+0000","last_peered":"2024-04-25T12:35:16.319787+0000","last_clean":"2024-04-25T12:35:16.319787+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:35:16.319787+0000","last_undegraded":"2024-04-25T12:35:16.319787+0000","last_fullsized":"2024-04-25T12:35:16.319787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313497+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:35:15.313497+0000","last_peered":"2024-04-25T12:35:15.313497+0000","last_clean":"2024-04-25T12:35:15.313497+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:35:15.313497+0000","last_undegraded":"2024-04-25T12:35:15.313497+0000","last_fullsized":"2024-04-25T12:35:15.313497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313518+0000","last_change":"2024-04-25T12:34:18.000022+0000","last_active":"2024-04-25T12:35:15.313518+0000","last_peered":"2024-04-25T12:35:15.313518+0000","last_clean":"2024-04-25T12:35:15.313518+0000","last_became_active":"2024-04-25T12:34:17.999339+0000","last_became_peered":"2024-04-25T12:34:17.999339+0000","last_unstale":"2024-04-25T12:35:15.313518+0000","last_undegraded":"2024-04-25T12:35:15.313518+0000","last_fullsized":"2024-04-25T12:35:15.313518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:02.270078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327234+0000","last_change":"2024-04-25T12:34:12.904697+0000","last_active":"2024-04-25T12:35:16.327234+0000","last_peered":"2024-04-25T12:35:16.327234+0000","last_clean":"2024-04-25T12:35:16.327234+0000","last_became_active":"2024-04-25T12:34:12.903395+0000","last_became_peered":"2024-04-25T12:34:12.903395+0000","last_unstale":"2024-04-25T12:35:16.327234+0000","last_undegraded":"2024-04-25T12:35:16.327234+0000","last_fullsized":"2024-04-25T12:35:16.327234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.322197+0000","last_change":"2024-04-25T12:34:13.948425+0000","last_active":"2024-04-25T12:35:16.322197+0000","last_peered":"2024-04-25T12:35:16.322197+0000","last_clean":"2024-04-25T12:35:16.322197+0000","last_became_active":"2024-04-25T12:34:13.948082+0000","last_became_peered":"2024-04-25T12:34:13.948082+0000","last_unstale":"2024-04-25T12:35:16.322197+0000","last_undegraded":"2024-04-25T12:35:16.322197+0000","last_fullsized":"2024-04-25T12:35:16.322197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497436+0000","last_change":"2024-04-25T12:34:18.018943+0000","last_active":"2024-04-25T12:35:16.497436+0000","last_peered":"2024-04-25T12:35:16.497436+0000","last_clean":"2024-04-25T12:35:16.497436+0000","last_became_active":"2024-04-25T12:34:18.018366+0000","last_became_peered":"2024-04-25T12:34:18.018366+0000","last_unstale":"2024-04-25T12:35:16.497436+0000","last_undegraded":"2024-04-25T12:35:16.497436+0000","last_fullsized":"2024-04-25T12:35:16.497436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.322247+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:35:16.322247+0000","last_peered":"2024-04-25T12:35:16.322247+0000","last_clean":"2024-04-25T12:35:16.322247+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:35:16.322247+0000","last_undegraded":"2024-04-25T12:35:16.322247+0000","last_fullsized":"2024-04-25T12:35:16.322247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497119+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:35:16.497119+0000","last_peered":"2024-04-25T12:35:16.497119+0000","last_clean":"2024-04-25T12:35:16.497119+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:35:16.497119+0000","last_undegraded":"2024-04-25T12:35:16.497119+0000","last_fullsized":"2024-04-25T12:35:16.497119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.319364+0000","last_change":"2024-04-25T12:34:13.971886+0000","last_active":"2024-04-25T12:35:16.319364+0000","last_peered":"2024-04-25T12:35:16.319364+0000","last_clean":"2024-04-25T12:35:16.319364+0000","last_became_active":"2024-04-25T12:34:13.969458+0000","last_became_peered":"2024-04-25T12:34:13.969458+0000","last_unstale":"2024-04-25T12:35:16.319364+0000","last_undegraded":"2024-04-25T12:35:16.319364+0000","last_fullsized":"2024-04-25T12:35:16.319364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.232206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327215+0000","last_change":"2024-04-25T12:34:13.931590+0000","last_active":"2024-04-25T12:35:16.327215+0000","last_peered":"2024-04-25T12:35:16.327215+0000","last_clean":"2024-04-25T12:35:16.327215+0000","last_became_active":"2024-04-25T12:34:13.931222+0000","last_became_peered":"2024-04-25T12:34:13.931222+0000","last_unstale":"2024-04-25T12:35:16.327215+0000","last_undegraded":"2024-04-25T12:35:16.327215+0000","last_fullsized":"2024-04-25T12:35:16.327215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:08.617001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.393289+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:35:14.393289+0000","last_peered":"2024-04-25T12:35:14.393289+0000","last_clean":"2024-04-25T12:35:14.393289+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:35:14.393289+0000","last_undegraded":"2024-04-25T12:35:14.393289+0000","last_fullsized":"2024-04-25T12:35:14.393289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.393350+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:35:14.393350+0000","last_peered":"2024-04-25T12:35:14.393350+0000","last_clean":"2024-04-25T12:35:14.393350+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:35:14.393350+0000","last_undegraded":"2024-04-25T12:35:14.393350+0000","last_fullsized":"2024-04-25T12:35:14.393350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.325635+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:35:16.325635+0000","last_peered":"2024-04-25T12:35:16.325635+0000","last_clean":"2024-04-25T12:35:16.325635+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:35:16.325635+0000","last_undegraded":"2024-04-25T12:35:16.325635+0000","last_fullsized":"2024-04-25T12:35:16.325635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326521+0000","last_change":"2024-04-25T12:34:13.951745+0000","last_active":"2024-04-25T12:35:16.326521+0000","last_peered":"2024-04-25T12:35:16.326521+0000","last_clean":"2024-04-25T12:35:16.326521+0000","last_became_active":"2024-04-25T12:34:13.951405+0000","last_became_peered":"2024-04-25T12:34:13.951405+0000","last_unstale":"2024-04-25T12:35:16.326521+0000","last_undegraded":"2024-04-25T12:35:16.326521+0000","last_fullsized":"2024-04-25T12:35:16.326521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.398820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.318230+0000","last_change":"2024-04-25T12:34:13.967379+0000","last_active":"2024-04-25T12:35:16.318230+0000","last_peered":"2024-04-25T12:35:16.318230+0000","last_clean":"2024-04-25T12:35:16.318230+0000","last_became_active":"2024-04-25T12:34:13.967026+0000","last_became_peered":"2024-04-25T12:34:13.967026+0000","last_unstale":"2024-04-25T12:35:16.318230+0000","last_undegraded":"2024-04-25T12:35:16.318230+0000","last_fullsized":"2024-04-25T12:35:16.318230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326123+0000","last_change":"2024-04-25T12:34:13.938584+0000","last_active":"2024-04-25T12:35:16.326123+0000","last_peered":"2024-04-25T12:35:16.326123+0000","last_clean":"2024-04-25T12:35:16.326123+0000","last_became_active":"2024-04-25T12:34:13.938300+0000","last_became_peered":"2024-04-25T12:34:13.938300+0000","last_unstale":"2024-04-25T12:35:16.326123+0000","last_undegraded":"2024-04-25T12:35:16.326123+0000","last_fullsized":"2024-04-25T12:35:16.326123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.322042+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:35:16.322042+0000","last_peered":"2024-04-25T12:35:16.322042+0000","last_clean":"2024-04-25T12:35:16.322042+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:35:16.322042+0000","last_undegraded":"2024-04-25T12:35:16.322042+0000","last_fullsized":"2024-04-25T12:35:16.322042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T12:35:15.313370+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:35:15.313370+0000","last_peered":"2024-04-25T12:35:15.313370+0000","last_clean":"2024-04-25T12:35:15.313370+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:35:15.313370+0000","last_undegraded":"2024-04-25T12:35:15.313370+0000","last_fullsized":"2024-04-25T12:35:15.313370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467245+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:35:16.467245+0000","last_peered":"2024-04-25T12:35:16.467245+0000","last_clean":"2024-04-25T12:35:16.467245+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:35:16.467245+0000","last_undegraded":"2024-04-25T12:35:16.467245+0000","last_fullsized":"2024-04-25T12:35:16.467245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.322314+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:35:16.322314+0000","last_peered":"2024-04-25T12:35:16.322314+0000","last_clean":"2024-04-25T12:35:16.322314+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:35:16.322314+0000","last_undegraded":"2024-04-25T12:35:16.322314+0000","last_fullsized":"2024-04-25T12:35:16.322314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.497715+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:35:16.497715+0000","last_peered":"2024-04-25T12:35:16.497715+0000","last_clean":"2024-04-25T12:35:16.497715+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:35:16.497715+0000","last_undegraded":"2024-04-25T12:35:16.497715+0000","last_fullsized":"2024-04-25T12:35:16.497715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324111+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:35:16.324111+0000","last_peered":"2024-04-25T12:35:16.324111+0000","last_clean":"2024-04-25T12:35:16.324111+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:35:16.324111+0000","last_undegraded":"2024-04-25T12:35:16.324111+0000","last_fullsized":"2024-04-25T12:35:16.324111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.318198+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:35:16.318198+0000","last_peered":"2024-04-25T12:35:16.318198+0000","last_clean":"2024-04-25T12:35:16.318198+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:35:16.318198+0000","last_undegraded":"2024-04-25T12:35:16.318198+0000","last_fullsized":"2024-04-25T12:35:16.318198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326023+0000","last_change":"2024-04-25T12:34:13.962845+0000","last_active":"2024-04-25T12:35:16.326023+0000","last_peered":"2024-04-25T12:35:16.326023+0000","last_clean":"2024-04-25T12:35:16.326023+0000","last_became_active":"2024-04-25T12:34:13.962542+0000","last_became_peered":"2024-04-25T12:34:13.962542+0000","last_unstale":"2024-04-25T12:35:16.326023+0000","last_undegraded":"2024-04-25T12:35:16.326023+0000","last_fullsized":"2024-04-25T12:35:16.326023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.392965+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:35:14.392965+0000","last_peered":"2024-04-25T12:35:14.392965+0000","last_clean":"2024-04-25T12:35:14.392965+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:35:14.392965+0000","last_undegraded":"2024-04-25T12:35:14.392965+0000","last_fullsized":"2024-04-25T12:35:14.392965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T12:35:14.393169+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:35:14.393169+0000","last_peered":"2024-04-25T12:35:14.393169+0000","last_clean":"2024-04-25T12:35:14.393169+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:35:14.393169+0000","last_undegraded":"2024-04-25T12:35:14.393169+0000","last_fullsized":"2024-04-25T12:35:14.393169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.466972+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:35:16.466972+0000","last_peered":"2024-04-25T12:35:16.466972+0000","last_clean":"2024-04-25T12:35:16.466972+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:35:16.466972+0000","last_undegraded":"2024-04-25T12:35:16.466972+0000","last_fullsized":"2024-04-25T12:35:16.466972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467986+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:35:16.467986+0000","last_peered":"2024-04-25T12:35:16.467986+0000","last_clean":"2024-04-25T12:35:16.467986+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:35:16.467986+0000","last_undegraded":"2024-04-25T12:35:16.467986+0000","last_fullsized":"2024-04-25T12:35:16.467986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467862+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:35:16.467862+0000","last_peered":"2024-04-25T12:35:16.467862+0000","last_clean":"2024-04-25T12:35:16.467862+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:35:16.467862+0000","last_undegraded":"2024-04-25T12:35:16.467862+0000","last_fullsized":"2024-04-25T12:35:16.467862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324819+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:35:16.324819+0000","last_peered":"2024-04-25T12:35:16.324819+0000","last_clean":"2024-04-25T12:35:16.324819+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:35:16.324819+0000","last_undegraded":"2024-04-25T12:35:16.324819+0000","last_fullsized":"2024-04-25T12:35:16.324819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326219+0000","last_change":"2024-04-25T12:34:13.958790+0000","last_active":"2024-04-25T12:35:16.326219+0000","last_peered":"2024-04-25T12:35:16.326219+0000","last_clean":"2024-04-25T12:35:16.326219+0000","last_became_active":"2024-04-25T12:34:13.958467+0000","last_became_peered":"2024-04-25T12:34:13.958467+0000","last_unstale":"2024-04-25T12:35:16.326219+0000","last_undegraded":"2024-04-25T12:35:16.326219+0000","last_fullsized":"2024-04-25T12:35:16.326219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.466430+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:35:16.466430+0000","last_peered":"2024-04-25T12:35:16.466430+0000","last_clean":"2024-04-25T12:35:16.466430+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:35:16.466430+0000","last_undegraded":"2024-04-25T12:35:16.466430+0000","last_fullsized":"2024-04-25T12:35:16.466430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.326774+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:35:16.326774+0000","last_peered":"2024-04-25T12:35:16.326774+0000","last_clean":"2024-04-25T12:35:16.326774+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:35:16.326774+0000","last_undegraded":"2024-04-25T12:35:16.326774+0000","last_fullsized":"2024-04-25T12:35:16.326774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467032+0000","last_change":"2024-04-25T12:34:13.968944+0000","last_active":"2024-04-25T12:35:16.467032+0000","last_peered":"2024-04-25T12:35:16.467032+0000","last_clean":"2024-04-25T12:35:16.467032+0000","last_became_active":"2024-04-25T12:34:13.967757+0000","last_became_peered":"2024-04-25T12:34:13.967757+0000","last_unstale":"2024-04-25T12:35:16.467032+0000","last_undegraded":"2024-04-25T12:35:16.467032+0000","last_fullsized":"2024-04-25T12:35:16.467032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.324366+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:35:16.324366+0000","last_peered":"2024-04-25T12:35:16.324366+0000","last_clean":"2024-04-25T12:35:16.324366+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:35:16.324366+0000","last_undegraded":"2024-04-25T12:35:16.324366+0000","last_fullsized":"2024-04-25T12:35:16.324366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327038+0000","last_change":"2024-04-25T12:34:13.930185+0000","last_active":"2024-04-25T12:35:16.327038+0000","last_peered":"2024-04-25T12:35:16.327038+0000","last_clean":"2024-04-25T12:35:16.327038+0000","last_became_active":"2024-04-25T12:34:13.929865+0000","last_became_peered":"2024-04-25T12:34:13.929865+0000","last_unstale":"2024-04-25T12:35:16.327038+0000","last_undegraded":"2024-04-25T12:35:16.327038+0000","last_fullsized":"2024-04-25T12:35:16.327038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:59.003966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.327188+0000","last_change":"2024-04-25T12:34:13.934508+0000","last_active":"2024-04-25T12:35:16.327188+0000","last_peered":"2024-04-25T12:35:16.327188+0000","last_clean":"2024-04-25T12:35:16.327188+0000","last_became_active":"2024-04-25T12:34:13.933947+0000","last_became_peered":"2024-04-25T12:34:13.933947+0000","last_unstale":"2024-04-25T12:35:16.327188+0000","last_undegraded":"2024-04-25T12:35:16.327188+0000","last_fullsized":"2024-04-25T12:35:16.327188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:50.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-25T12:35:16.467932+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:35:16.467932+0000","last_peered":"2024-04-25T12:35:16.467932+0000","last_clean":"2024-04-25T12:35:16.467932+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:35:16.467932+0000","last_undegraded":"2024-04-25T12:35:16.467932+0000","last_fullsized":"2024-04-25T12:35:16.467932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9787,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":19574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65108,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":138278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":138368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":319488,"data_stored":124285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":124581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":728224,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":137970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":143263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":720753,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:35:20.948 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:20.949+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:21.023 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:21.023 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:21.023 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:21.023 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:21.023 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:21.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:21.049 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:21.050+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:21.150 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:21.151+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:21.251 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:21.251+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:21.351 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:21.352+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:21.365 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.410909162999999, 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:21.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:35:21.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.370 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.410797219999999, 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:35:21.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:35:21.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:35:21.375 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.410748419999997, 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:35:21.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.41067726, 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.381 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:35:21.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:21.386 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.410628848000002, 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:35:21.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.402432920000003, 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:35:21.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 45.296188921999999, 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:35:21.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 45.296006921, 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:35:21.403 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.243972933999999, 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.243896977999999, 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:35:21.414 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.239468238000001, 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:21.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:35:21.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.418 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.239286175, 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:35:21.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:35:21.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.239181811000002, 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.237906253999999, 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:35:21.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.236852030000001, 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.431 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:35:21.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:22 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.233485684000001, 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026067460000000001, 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128290+0000", 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.474e-06 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128302+0000", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2001000000000001e-05 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128323+0000", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0613999999999999e-05 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128339+0000", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5713999999999999e-05 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128423+0000", 2024-04-25T12:35:21.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4154999999999996e-05 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128451+0000", 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8473999999999999e-05 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130796+0000", 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002344245 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130896+0000", 2024-04-25T12:35:21.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010006899999999999 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.207986389999999, 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:35:21.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:44 2.14 2:2e2fa760:::200.00000001:head [write 3170~258 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.486701572999999, 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026976719999999999, 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875077+0000", 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0749999999999999e-06 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875102+0000", 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5102000000000001e-05 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875144+0000", 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1647999999999998e-05 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875170+0000", 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5876000000000001e-05 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875363+0000", 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019320700000000001 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875551+0000", 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000188037 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875700+0000", 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148837 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877284+0000", 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001584483 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877626+0000", 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034161599999999999 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877700+0000", 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4467000000000004e-05 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877771+0000", 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0324e-05 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.486474895000001, 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.4439143429999994, 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:21.453 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:35:21.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:35:21.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:35:21.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:35:21.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:21.458 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:21.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:35:21.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:21.461 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:21.461+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:21.561 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:21.562+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:21.662 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:21.663+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.708+0000 7f8fbbc00640 1 Processor -- start 2024-04-25T12:35:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.708+0000 7f8fbbc00640 1 -- start start 2024-04-25T12:35:21.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.710+0000 7f8fbbc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8fb40c8250 0x7f8fb40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:21.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.710+0000 7f8fbbc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb4156e60 con 0x7f8fb4159820 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.710+0000 7f8fbbc00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb40c7130 con 0x7f8fb40c9b80 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.710+0000 7f8fbbc00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb40749c0 con 0x7f8fb40c8250 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb3e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8fb40c8250 0x7f8fb40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb9c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8fb4159820 0x7f8fb4159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43592/0 (socket says 172.21.15.123:43592) 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb9c00640 1 -- 172.21.15.123:0/3978434684 learned_addr learned my addr 172.21.15.123:0/3978434684 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb9200640 1 --1- 172.21.15.123:0/3978434684 >> v1:172.21.15.153:6789/0 conn(0x7f8fb40c9b80 0x7f8fb40742b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:58950/0 (socket says 172.21.15.123:58950) 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb3400640 1 -- 172.21.15.123:0/3978434684 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160698166 0 0) 0x7f8fb40c7130 con 0x7f8fb40c9b80 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb3400640 1 -- 172.21.15.123:0/3978434684 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f9c003520 con 0x7f8fb40c9b80 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb3e00640 1 -- 172.21.15.123:0/3978434684 >> v1:172.21.15.153:6789/0 conn(0x7f8fb40c9b80 legacy=0x7f8fb40742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb3400640 1 -- 172.21.15.123:0/3978434684 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947225775 0 0) 0x7f8fb4156e60 con 0x7f8fb4159820 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb3e00640 1 -- 172.21.15.123:0/3978434684 >> v1:172.21.15.123:6789/0 conn(0x7f8fb4159820 legacy=0x7f8fb4159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:21.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.711+0000 7f8fb3e00640 1 -- 172.21.15.123:0/3978434684 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb415ca70 con 0x7f8fb40c8250 2024-04-25T12:35:21.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.712+0000 7f8fb3e00640 1 --2- 172.21.15.123:0/3978434684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 0x7f8fb40c8660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00097a0 tx=0x7f8fa002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=664bcf935d23eaaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:21.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.712+0000 7f8fb3400640 1 -- 172.21.15.123:0/3978434684 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0037070 con 0x7f8fb40c8250 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.713+0000 7f8fb3400640 1 -- 172.21.15.123:0/3978434684 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa0032440 con 0x7f8fb40c8250 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.713+0000 7f8fb3400640 1 -- 172.21.15.123:0/3978434684 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0032810 con 0x7f8fb40c8250 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.713+0000 7f8fbbc00640 1 -- 172.21.15.123:0/3978434684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 msgr2=0x7f8fb40c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.713+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/3978434684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 0x7f8fb40c8660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00097a0 tx=0x7f8fa002a230 comp rx=0 tx=0).stop 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.713+0000 7f8fbbc00640 1 -- 172.21.15.123:0/3978434684 shutdown_connections 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.713+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/3978434684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 0x7f8fb40c8660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.713+0000 7f8fbbc00640 1 -- 172.21.15.123:0/3978434684 >> 172.21.15.123:0/3978434684 conn(0x7f8fb406c750 msgr2=0x7f8fb406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.714+0000 7f8fbbc00640 1 -- 172.21.15.123:0/3978434684 shutdown_connections 2024-04-25T12:35:21.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.714+0000 7f8fbbc00640 1 -- 172.21.15.123:0/3978434684 wait complete. 2024-04-25T12:35:21.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.716+0000 7f8fbbc00640 1 Processor -- start 2024-04-25T12:35:21.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.716+0000 7f8fbbc00640 1 -- start start 2024-04-25T12:35:21.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fbbc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 0x7f8fb41539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fbbc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb40c9b80 0x7f8fb41f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fbbc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8fb4159820 0x7f8fb4154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fbbc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb415d030 con 0x7f8fb40c9b80 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fbbc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40c8ba0 con 0x7f8fb40c8250 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fbbc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb415d310 con 0x7f8fb4159820 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:21.717 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fb3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb40c9b80 0x7f8fb41f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fb3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb40c9b80 0x7f8fb41f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52088/0 (socket says 172.21.15.123:52088) 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.717+0000 7f8fb3e00640 1 -- 172.21.15.123:0/1984396567 learned_addr learned my addr 172.21.15.123:0/1984396567 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb9200640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 0x7f8fb41539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb9c00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8fb4159820 0x7f8fb4154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb3e00640 1 -- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8fb4159820 msgr2=0x7f8fb4154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb3e00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8fb4159820 0x7f8fb4154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb3e00640 1 -- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 msgr2=0x7f8fb41539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb3e00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 0x7f8fb41539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb3e00640 1 -- 172.21.15.123:0/1984396567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb41f7120 con 0x7f8fb40c9b80 2024-04-25T12:35:21.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.718+0000 7f8fb3e00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb40c9b80 0x7f8fb41f2cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0035040 tx=0x7f8fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.719+0000 7f8fb0c00640 1 -- 172.21.15.123:0/1984396567 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0041020 con 0x7f8fb40c9b80 2024-04-25T12:35:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.719+0000 7f8fb0c00640 1 -- 172.21.15.123:0/1984396567 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa003d6e0 con 0x7f8fb40c9b80 2024-04-25T12:35:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.719+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb41f6360 con 0x7f8fb40c9b80 2024-04-25T12:35:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.719+0000 7f8fb0c00640 1 -- 172.21.15.123:0/1984396567 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0037040 con 0x7f8fb40c9b80 2024-04-25T12:35:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.719+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb41f6920 con 0x7f8fb40c9b80 2024-04-25T12:35:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.720+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb4159470 con 0x7f8fb40c9b80 2024-04-25T12:35:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.721+0000 7f8fb0c00640 1 -- 172.21.15.123:0/1984396567 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8fa0044020 con 0x7f8fb40c9b80 2024-04-25T12:35:21.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.723+0000 7f8fb0c00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f940744e0 0x7f8f940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:21.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.723+0000 7f8fb0c00640 1 -- 172.21.15.123:0/1984396567 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6619+0+0 (secure 0 0 0) 0x7f8fa00b5c90 con 0x7f8fb40c9b80 2024-04-25T12:35:21.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.723+0000 7f8fb9200640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f940744e0 0x7f8f940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:21.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.724+0000 7f8fb9200640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f940744e0 0x7f8f940769f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000f60 tx=0x7f8fa8007cf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:21.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.727+0000 7f8fb0c00640 1 -- 172.21.15.123:0/1984396567 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fb4159470 con 0x7f8fb40c9b80 2024-04-25T12:35:21.737 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:21.762 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:21.763+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:21.863 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:21.864+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:21.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:21.932+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f8fb4076700 con 0x7f8fb40c9b80 2024-04-25T12:35:21.964 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:21.965+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:22.044 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:22.044 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:22.044 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:22.044 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:22.045 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:22.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:22.066 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:22.065+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:22.166 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:22.167+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:22.267 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:22.268+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:22.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.349+0000 7f8fb0c00640 1 -- 172.21.15.123:0/1984396567 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v32) ==== 101+0+0 (secure 0 0 0) 0x7f8fa004b030 con 0x7f8fb40c9b80 2024-04-25T12:35:22.349 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:35:22.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f940744e0 msgr2=0x7f8f940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:22.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f940744e0 0x7f8f940769f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000f60 tx=0x7f8fa8007cf0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb40c9b80 msgr2=0x7f8fb41f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb40c9b80 0x7f8fb41f2cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0035040 tx=0x7f8fa00046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 shutdown_connections 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f940744e0 0x7f8f940769f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8fb4159820 0x7f8fb4154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb40c9b80 0x7f8fb41f2cc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 --2- 172.21.15.123:0/1984396567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fb40c8250 0x7f8fb41539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 >> 172.21.15.123:0/1984396567 conn(0x7f8fb406c750 msgr2=0x7f8fb40ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:22.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.352+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 shutdown_connections 2024-04-25T12:35:22.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.353+0000 7f8fbbc00640 1 -- 172.21.15.123:0/1984396567 wait complete. 2024-04-25T12:35:22.368 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:22.369+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:22.372 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:22.372 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.421858862999997, 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:22.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:22.377 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.421174680999997, 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:22.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.381 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.41972414, 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:35:22.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:35:22.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:35:22.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:22.387 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.419463325000002, 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:35:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:35:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:35:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.392 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.415612938000002, 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:22.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:35:22.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:35:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:35:22.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:35:22.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.41539057, 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.399 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.415036780999998, 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:22.404 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:35:22.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:35:22.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.254417832999998, 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:35:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.254210340999997, 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:35:22.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:35:22.418 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.254065851, 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.419 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:22.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:35:22.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:35:22.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:35:22.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253934057000002, 2024-04-25T12:35:22.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:35:22.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:35:22.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:35:22.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253859761999998, 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.436 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253760020999998, 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:35:22.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:35:22.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:35:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253690558999999, 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.442 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:35:22.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:35:22.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:35:22.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253590445, 2024-04-25T12:35:22.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.452 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253493354, 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:22.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:35:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253402948999998, 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:35:22.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253254976000001, 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:22.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:35:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:35:22.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.467 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.253147571, 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:22.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.218896272999999, 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:35:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:22.476 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:22.478 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:22.479 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:22.477+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:22.578 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:22.578+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:22.605 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:22.605 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:22.605 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:22.605 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:22.614 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:22.679 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:22.680+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:22.692 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:22.766 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:22.766 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:22.766 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:22.767 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:22.767 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:22.775 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:22.780 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:22.781+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:22.881 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:22.880+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:22.889 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.939436344000001, 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:35:22.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:22.894 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.939348635000002, 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:35:22.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.899 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.939222706999999, 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.904 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.939161048000003, 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.907 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.939104284999999, 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.910 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.911 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.912 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:35:22.914 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.939040814999998, 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.915 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:35:22.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:35:22.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.938988866000003, 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:35:22.920 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.921 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.924 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.938936916000003, 2024-04-25T12:35:22.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.929 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:22.930 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.938819807000002, 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:35:22.932 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:35:22.933 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.934 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:22.935 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 63.93878136, 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.936 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:35:22.937 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:35:22.938 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 32.773646307, 2024-04-25T12:35:22.939 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.940 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.941 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:35:22.942 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:35:22.943 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.944 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 32.773588255999996, 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.945 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.946 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.947 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.948 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.949 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 32.773528562999999, 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:22.950 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.951 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.952 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.953 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.954 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.736608481000001, 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.955 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:35:22.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:35:22.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:35:22.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:35:22.959 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.736519515000001, 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.960 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:35:22.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:22.963 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:22.965 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.736475674000001, 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:22.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.968 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:35:22.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.734961769999998, 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:35:22.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:35:22.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:35:22.975 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.734042553999998, 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.976 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:22.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.733587200999999, 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:35:22.984 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 27.733558222999999, 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:22.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:22.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:35:22.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:22.988 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:22.989 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:22.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.971+0000 7ffa14e00640 1 Processor -- start 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.971+0000 7ffa14e00640 1 -- start start 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.973+0000 7ffa14e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffa10154510 0x7ffa100c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.973+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa1015e3a0 0x7ffa100c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa14e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa10074230 con 0x7ffa1015df90 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa14e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ffa100c84b0 con 0x7ffa10154510 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa14e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ffa10154920 con 0x7ffa1015e3a0 2024-04-25T12:35:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0f400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffa10154510 0x7ffa100c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa1015e3a0 0x7ffa100c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0ea00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ffa1015df90 0x7ffa10072ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34874/0 (socket says 172.21.15.123:34874) 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0ea00640 1 -- 172.21.15.123:0/1941151197 learned_addr learned my addr 172.21.15.123:0/1941151197 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0f400640 1 -- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa1015e3a0 msgr2=0x7ffa100c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0f400640 1 --2- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa1015e3a0 0x7ffa100c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0f400640 1 -- 172.21.15.123:0/1941151197 >> v1:172.21.15.153:6789/0 conn(0x7ffa1015df90 legacy=0x7ffa10072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0f400640 1 -- 172.21.15.123:0/1941151197 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa10074510 con 0x7ffa10154510 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0e000640 1 -- 172.21.15.123:0/1941151197 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513437000 0 0) 0x7ffa10074230 con 0x7ffa1015df90 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.974+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa1015e3a0 0x7ffa100c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.975+0000 7ffa0f400640 1 --2- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa10154510 0x7ffa100c7f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa000097a0 tx=0x7ffa0002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b169b9e6e2b3b4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.976+0000 7ffa0e000640 1 -- 172.21.15.123:0/1941151197 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00037070 con 0x7ffa10154510 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.976+0000 7ffa0e000640 1 -- 172.21.15.123:0/1941151197 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa00032440 con 0x7ffa10154510 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.976+0000 7ffa0e000640 1 -- 172.21.15.123:0/1941151197 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00032810 con 0x7ffa10154510 2024-04-25T12:35:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 -- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa10154510 msgr2=0x7ffa100c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 --2- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa10154510 0x7ffa100c7f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa000097a0 tx=0x7ffa0002a210 comp rx=0 tx=0).stop 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 -- 172.21.15.123:0/1941151197 shutdown_connections 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 --2- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa1015e3a0 0x7ffa100c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 --2- 172.21.15.123:0/1941151197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa10154510 0x7ffa100c7f70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 -- 172.21.15.123:0/1941151197 >> 172.21.15.123:0/1941151197 conn(0x7ffa10070b90 msgr2=0x7ffa10070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 -- 172.21.15.123:0/1941151197 shutdown_connections 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.977+0000 7ffa14e00640 1 -- 172.21.15.123:0/1941151197 wait complete. 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.980+0000 7ffa14e00640 1 Processor -- start 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.980+0000 7ffa14e00640 1 -- start start 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.981+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa10154510 0x7ffa101e6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa1015df90 0x7ffa1015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1015e3a0 0x7ffa10157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa14e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100733f0 con 0x7ffa1015e3a0 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa14e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1015d790 con 0x7ffa1015df90 2024-04-25T12:35:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa14e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ffa10074ad0 con 0x7ffa10154510 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa1015df90 0x7ffa1015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa10154510 0x7ffa101e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa10154510 0x7ffa101e6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35368/0 (socket says 172.21.15.123:35368) 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0f400640 1 -- 172.21.15.123:0/2324449613 learned_addr learned my addr 172.21.15.123:0/2324449613 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1015e3a0 0x7ffa10157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0ea00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa10154510 msgr2=0x7ffa101e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0ea00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa10154510 0x7ffa101e6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0ea00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1015e3a0 msgr2=0x7ffa10157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0ea00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1015e3a0 0x7ffa10157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0ea00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1015c7a0 con 0x7ffa1015df90 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1015e3a0 0x7ffa10157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.982+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffa10154510 0x7ffa101e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.983+0000 7ffa0ea00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa1015df90 0x7ffa1015c260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c440 tx=0x7ff9fc00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.983+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0074e0 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.983+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0047e0 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.983+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc004bd0 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.983+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa100c4d00 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.983+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1015d030 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.984+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ffa100c4a60 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.985+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9fc011070 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.986+0000 7ffa07400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9f00744b0 0x7ff9f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.987+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6619+0+0 (secure 0 0 0) 0x7ff9fc095350 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.987+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9f00744b0 0x7ff9f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.987+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v32) ==== 54+0+16 (secure 0 0 0) 0x7ffa100c4a60 con 0x7ffa1015df90 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.987+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ffa1015b2a0 0x7ffa10149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.987+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa100736d0 con 0x7ffa1015b2a0 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.987+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ffa1015b2a0 0x7ffa10149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.988+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9f00744b0 0x7ff9f00769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002a250 tx=0x7ffa0002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.988+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ffa1015b2a0 0x7ffa10149e70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:22.989+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa100736d0 con 0x7ffa1015b2a0 2024-04-25T12:35:23.000 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:22.991+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:23.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.005+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa1006a350 con 0x7ffa1015b2a0 2024-04-25T12:35:23.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.009+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa1006a350 con 0x7ffa1015b2a0 2024-04-25T12:35:23.008 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.009+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ffa100c6400 0x7ffa101f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.009+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ffa10071320 con 0x7ffa100c6400 2024-04-25T12:35:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.010+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ffa100c6400 0x7ffa101f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:23.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.010+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ffa100c6400 0x7ffa101f5ca0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.011+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa10071320 con 0x7ffa100c6400 2024-04-25T12:35:23.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.028+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa10071040 con 0x7ffa100c6400 2024-04-25T12:35:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.030+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa10071040 con 0x7ffa100c6400 2024-04-25T12:35:23.030 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.031+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ffa101f66f0 0x7ffa10062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.031+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ffa101f6b00 con 0x7ffa101f66f0 2024-04-25T12:35:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.031+0000 7ffa0ea00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ffa101f66f0 0x7ffa10062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.032+0000 7ffa0ea00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ffa101f66f0 0x7ffa10062fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.033+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa101f6b00 con 0x7ffa101f66f0 2024-04-25T12:35:23.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.048+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa10063520 con 0x7ffa101f66f0 2024-04-25T12:35:23.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.052+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa10063520 con 0x7ffa101f66f0 2024-04-25T12:35:23.051 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:23.052 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:23.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.052+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ffa1014b410 0x7ffa1014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:23.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.052+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ffa1014bd60 con 0x7ffa1014b410 2024-04-25T12:35:23.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.053+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ffa1014b410 0x7ffa1014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:23.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.054+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ffa1014b410 0x7ffa1014b820 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.056+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa1014bd60 con 0x7ffa1014b410 2024-04-25T12:35:23.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.071+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa1014c240 con 0x7ffa1014b410 2024-04-25T12:35:23.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.074+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa1014c240 con 0x7ffa1014b410 2024-04-25T12:35:23.074 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:23.074 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:23.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.075+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ffa1002cee0 0x7ffa1002d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:23.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.075+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ffa1002d830 con 0x7ffa1002cee0 2024-04-25T12:35:23.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.075+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ffa1002cee0 0x7ffa1002d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:23.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.076+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ffa1002cee0 0x7ffa1002d2f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.077+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa1002d830 con 0x7ffa1002cee0 2024-04-25T12:35:23.091 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:23.092+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:23.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.093+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa1002d830 con 0x7ffa1002cee0 2024-04-25T12:35:23.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.095+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa1002d830 con 0x7ffa1002cee0 2024-04-25T12:35:23.094 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.096+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ffa10207020 0x7ffa10209440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.096+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ffa1020c210 con 0x7ffa10207020 2024-04-25T12:35:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.096+0000 7ffa0ea00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ffa10207020 0x7ffa10209440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:23.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.097+0000 7ffa0ea00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ffa10207020 0x7ffa10209440 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.098+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa1020c210 con 0x7ffa10207020 2024-04-25T12:35:23.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.113+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa10209980 con 0x7ffa10207020 2024-04-25T12:35:23.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.115+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa10209980 con 0x7ffa10207020 2024-04-25T12:35:23.114 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:23.115 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:23.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.115+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ffa10209e90 0x7ffa1020c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:23.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.115+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ffa1020f080 con 0x7ffa10209e90 2024-04-25T12:35:23.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.116+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ffa10209e90 0x7ffa1020c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:23.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.117+0000 7ffa0fe00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ffa10209e90 0x7ffa1020c2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.118+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa1020f080 con 0x7ffa10209e90 2024-04-25T12:35:23.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.133+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa1020c7f0 con 0x7ffa10209e90 2024-04-25T12:35:23.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.135+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa1020c7f0 con 0x7ffa10209e90 2024-04-25T12:35:23.134 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:23.135 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:23.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.135+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ffa1020ce70 0x7ffa1020f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:23.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.135+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ffa10212060 con 0x7ffa1020ce70 2024-04-25T12:35:23.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.136+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ffa1020ce70 0x7ffa1020f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:23.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.137+0000 7ffa0f400640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ffa1020ce70 0x7ffa1020f290 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:23.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.138+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa10212060 con 0x7ffa1020ce70 2024-04-25T12:35:23.143 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:23.143 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:23.152 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:35:23.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.153+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ffa1020f7d0 con 0x7ffa1020ce70 2024-04-25T12:35:23.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.155+0000 7ffa07400640 1 -- 172.21.15.123:0/2324449613 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa1020f7d0 con 0x7ffa1020ce70 2024-04-25T12:35:23.155 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ffa1015b2a0 msgr2=0x7ffa10149e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ffa1015b2a0 0x7ffa10149e70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ffa100c6400 msgr2=0x7ffa101f5ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ffa100c6400 0x7ffa101f5ca0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ffa101f66f0 msgr2=0x7ffa10062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ffa101f66f0 0x7ffa10062fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ffa1014b410 msgr2=0x7ffa1014b820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ffa1014b410 0x7ffa1014b820 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ffa1002cee0 msgr2=0x7ffa1002d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ffa1002cee0 0x7ffa1002d2f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ffa10207020 msgr2=0x7ffa10209440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa0fe00640 1 -- 172.21.15.123:0/2324449613 reap_dead start 2024-04-25T12:35:23.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ffa10207020 0x7ffa10209440 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa0fe00640 1 -- 172.21.15.123:0/2324449613 reap_dead start 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ffa10209e90 msgr2=0x7ffa1020c2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ffa10209e90 0x7ffa1020c2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ffa1020ce70 msgr2=0x7ffa1020f290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ffa1020ce70 0x7ffa1020f290 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.156+0000 7ffa0fe00640 1 -- 172.21.15.123:0/2324449613 reap_dead start 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9f00744b0 msgr2=0x7ff9f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9f00744b0 0x7ff9f00769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002a250 tx=0x7ffa0002e000 comp rx=0 tx=0).stop 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa1015df90 msgr2=0x7ffa1015c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa1015df90 0x7ffa1015c260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c440 tx=0x7ff9fc00c900 comp rx=0 tx=0).stop 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 shutdown_connections 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9f00744b0 0x7ff9f00769c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 --2- 172.21.15.123:0/2324449613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa1015df90 0x7ffa1015c260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 >> 172.21.15.123:0/2324449613 conn(0x7ffa10070b90 msgr2=0x7ffa100c3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:23.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 shutdown_connections 2024-04-25T12:35:23.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:23.157+0000 7ffa14e00640 1 -- 172.21.15.123:0/2324449613 wait complete. 2024-04-25T12:35:23.222 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:23.223+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:23.265 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:23.265 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:23.272 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:35:23.293 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:23.294+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:23.385 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:23.385 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:23.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:23.395 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:23.395+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:23.494 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:23.495+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:23.506 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:23.506 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:23.515 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:23.595 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:23.594+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:23.627 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:23.627 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:23.637 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:23.696 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:23.697+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:23.788 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:23.788 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:23.797 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:23.796+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:23.798 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:23.898 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:23.899+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:23.948 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:23.949 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:23.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:24.001 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:24.001+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:24.069 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:24.070 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:24.078 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:24.100 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:24.101+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:24.201 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:24.202+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:24.230 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:24.230 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:24.240 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:24.302 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:24.302+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:24.403 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:24.404+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:24.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:35:24.504 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:24.504+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:24.518 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:24.518 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:24.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:24.604 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:24.605+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:24.705 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:24.705+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:24.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.744+0000 7f0745200640 1 Processor -- start 2024-04-25T12:35:24.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.744+0000 7f0745200640 1 -- start start 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.746+0000 7f0745200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07400c8250 0x7f074015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.746+0000 7f0745200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f07400cda90 0x7f07400cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.746+0000 7f0745200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07400c2b80 con 0x7f0740074690 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.746+0000 7f0745200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f07400c4400 con 0x7f07400cda90 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.746+0000 7f0745200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f07400c46e0 con 0x7f07400c8250 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f0737e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07400c8250 0x7f074015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f073e800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f07400cda90 0x7f07400cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f0737e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f07400c8250 0x7f074015ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41008/0 (socket says 172.21.15.123:41008) 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f073de00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0740074690 0x7f07401566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34876/0 (socket says 172.21.15.123:34876) 2024-04-25T12:35:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f0737e00640 1 -- 172.21.15.123:0/3183030404 learned_addr learned my addr 172.21.15.123:0/3183030404 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f0737e00640 1 -- 172.21.15.123:0/3183030404 >> v2:172.21.15.123:3300/0 conn(0x7f07400cda90 msgr2=0x7f07400cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f0737e00640 1 --2- 172.21.15.123:0/3183030404 >> v2:172.21.15.123:3300/0 conn(0x7f07400cda90 0x7f07400cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f0737e00640 1 -- 172.21.15.123:0/3183030404 >> v1:172.21.15.153:6789/0 conn(0x7f0740074690 legacy=0x7f07401566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f0737e00640 1 -- 172.21.15.123:0/3183030404 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074015c610 con 0x7f07400c8250 2024-04-25T12:35:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.747+0000 7f073d400640 1 -- 172.21.15.123:0/3183030404 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124240562 0 0) 0x7f07400c2b80 con 0x7f0740074690 2024-04-25T12:35:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.748+0000 7f0737e00640 1 --2- 172.21.15.123:0/3183030404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400c8250 0x7f074015ce10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f07400ce540 tx=0x7f072802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26034740aa69446f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.749+0000 7f073d400640 1 -- 172.21.15.123:0/3183030404 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728037070 con 0x7f07400c8250 2024-04-25T12:35:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.749+0000 7f073d400640 1 -- 172.21.15.123:0/3183030404 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0728032440 con 0x7f07400c8250 2024-04-25T12:35:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.749+0000 7f073d400640 1 -- 172.21.15.123:0/3183030404 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728032810 con 0x7f07400c8250 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.749+0000 7f0745200640 1 -- 172.21.15.123:0/3183030404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400c8250 msgr2=0x7f074015ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.749+0000 7f0745200640 1 --2- 172.21.15.123:0/3183030404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400c8250 0x7f074015ce10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f07400ce540 tx=0x7f072802a270 comp rx=0 tx=0).stop 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.750+0000 7f0745200640 1 -- 172.21.15.123:0/3183030404 shutdown_connections 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.750+0000 7f0745200640 1 --2- 172.21.15.123:0/3183030404 >> v2:172.21.15.123:3300/0 conn(0x7f07400cda90 0x7f07400cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.750+0000 7f0745200640 1 --2- 172.21.15.123:0/3183030404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400c8250 0x7f074015ce10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.750+0000 7f0745200640 1 -- 172.21.15.123:0/3183030404 >> 172.21.15.123:0/3183030404 conn(0x7f074006c750 msgr2=0x7f074006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.750+0000 7f0745200640 1 -- 172.21.15.123:0/3183030404 shutdown_connections 2024-04-25T12:35:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.750+0000 7f0745200640 1 -- 172.21.15.123:0/3183030404 wait complete. 2024-04-25T12:35:24.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.752+0000 7f0745200640 1 Processor -- start 2024-04-25T12:35:24.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.752+0000 7f0745200640 1 -- start start 2024-04-25T12:35:24.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0745200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0740074690 0x7f0740153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:24.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0745200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07400c8250 0x7f074015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0745200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400cda90 0x7f074015ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0745200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0740074aa0 con 0x7f07400c8250 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0745200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07400c49c0 con 0x7f07400cda90 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0745200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0740074c20 con 0x7f0740074690 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0737e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07400c8250 0x7f074015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0737e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07400c8250 0x7f074015ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54732/0 (socket says 172.21.15.123:54732) 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f0737e00640 1 -- 172.21.15.123:0/2032619794 learned_addr learned my addr 172.21.15.123:0/2032619794 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f073e800640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400cda90 0x7f074015ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.754+0000 7f073de00640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0740074690 0x7f0740153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f0737e00640 1 -- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0740074690 msgr2=0x7f0740153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f0737e00640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0740074690 0x7f0740153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f0737e00640 1 -- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400cda90 msgr2=0x7f074015ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f0737e00640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400cda90 0x7f074015ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f0737e00640 1 -- 172.21.15.123:0/2032619794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0740154410 con 0x7f07400c8250 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f073e800640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400cda90 0x7f074015ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f0737e00640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07400c8250 0x7f074015ea20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0728000c00 tx=0x7f07280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.755+0000 7f0736000640 1 -- 172.21.15.123:0/2032619794 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728041020 con 0x7f07400c8250 2024-04-25T12:35:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.756+0000 7f0736000640 1 -- 172.21.15.123:0/2032619794 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0728030640 con 0x7f07400c8250 2024-04-25T12:35:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.756+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07401e6a00 con 0x7f07400c8250 2024-04-25T12:35:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.756+0000 7f0736000640 1 -- 172.21.15.123:0/2032619794 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728037040 con 0x7f07400c8250 2024-04-25T12:35:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.756+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07401e6f80 con 0x7f07400c8250 2024-04-25T12:35:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.757+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07400c3170 con 0x7f07400c8250 2024-04-25T12:35:24.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.758+0000 7f0736000640 1 -- 172.21.15.123:0/2032619794 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0728044020 con 0x7f07400c8250 2024-04-25T12:35:24.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.759+0000 7f0736000640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f071c0744b0 0x7f071c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:24.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.759+0000 7f0736000640 1 -- 172.21.15.123:0/2032619794 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6619+0+0 (secure 0 0 0) 0x7f07280b5c20 con 0x7f07400c8250 2024-04-25T12:35:24.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.760+0000 7f073de00640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f071c0744b0 0x7f071c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:24.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.760+0000 7f073de00640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f071c0744b0 0x7f071c0769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0730007fd0 tx=0x7f0730007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:24.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.769+0000 7f0736000640 1 -- 172.21.15.123:0/2032619794 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07400c3170 con 0x7f07400c8250 2024-04-25T12:35:24.806 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:24.806+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:24.842 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:24.842 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:24.865 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:24.912 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:24.913+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:24.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:24.989+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f0740076700 con 0x7f07400c8250 2024-04-25T12:35:25.008 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:25.009+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:25.109 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:25.109+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:25.132 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:25.132 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:25.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:25.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:25.249 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:25.250+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:25.311 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:25.312+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:25.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.383+0000 7f0736000640 1 -- 172.21.15.123:0/2032619794 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v33) ==== 87+0+0 (secure 0 0 0) 0x7f072804b030 con 0x7f07400c8250 2024-04-25T12:35:25.383 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:35:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.386+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f071c0744b0 msgr2=0x7f071c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.386+0000 7f0745200640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f071c0744b0 0x7f071c0769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0730007fd0 tx=0x7f0730007de0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.386+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07400c8250 msgr2=0x7f074015ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.386+0000 7f0745200640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07400c8250 0x7f074015ea20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0728000c00 tx=0x7f07280046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 shutdown_connections 2024-04-25T12:35:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f071c0744b0 0x7f071c0769c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07400cda90 0x7f074015ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07400c8250 0x7f074015ea20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 --2- 172.21.15.123:0/2032619794 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0740074690 0x7f0740153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 >> 172.21.15.123:0/2032619794 conn(0x7f074006c750 msgr2=0x7f07400c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:25.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 shutdown_connections 2024-04-25T12:35:25.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.387+0000 7f0745200640 1 -- 172.21.15.123:0/2032619794 wait complete. 2024-04-25T12:35:25.412 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:25.412+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:25.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.461+0000 7f76d8e00640 1 Processor -- start 2024-04-25T12:35:25.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.462+0000 7f76d8e00640 1 -- start start 2024-04-25T12:35:25.461 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:25.461 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.463+0000 7f76d8e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f76d4154650 0x7f76d415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.463+0000 7f76d8e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f76d40c9f00 0x7f76d40c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.463+0000 7f76d8e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d415b380 con 0x7f76d40730e0 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.463+0000 7f76d8e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f76d4074b20 con 0x7f76d40c9f00 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.463+0000 7f76d8e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f76d40ca780 con 0x7f76d4154650 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f76d40c9f00 0x7f76d40c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f76d4154650 0x7f76d415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f76d40c9f00 0x7f76d40c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54746/0 (socket says 172.21.15.123:54746) 2024-04-25T12:35:25.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d2800640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f76d40730e0 0x7f76d40708f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34886/0 (socket says 172.21.15.123:34886) 2024-04-25T12:35:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1400640 1 -- 172.21.15.123:0/1183740570 learned_addr learned my addr 172.21.15.123:0/1183740570 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1400640 1 -- 172.21.15.123:0/1183740570 >> v2:172.21.15.153:3300/0 conn(0x7f76d4154650 msgr2=0x7f76d415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1400640 1 --2- 172.21.15.123:0/1183740570 >> v2:172.21.15.153:3300/0 conn(0x7f76d4154650 0x7f76d415ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1400640 1 -- 172.21.15.123:0/1183740570 >> v1:172.21.15.153:6789/0 conn(0x7f76d40730e0 legacy=0x7f76d40708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1400640 1 -- 172.21.15.123:0/1183740570 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d40caa60 con 0x7f76d40c9f00 2024-04-25T12:35:25.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.464+0000 7f76d1e00640 1 --2- 172.21.15.123:0/1183740570 >> v2:172.21.15.153:3300/0 conn(0x7f76d4154650 0x7f76d415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:25.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.465+0000 7f76d1400640 1 --2- 172.21.15.123:0/1183740570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40c9f00 0x7f76d40c8170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76d4071a20 tx=0x7f76c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19077d7a9160503c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.466+0000 7f76d0a00640 1 -- 172.21.15.123:0/1183740570 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037070 con 0x7f76d40c9f00 2024-04-25T12:35:25.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.466+0000 7f76d0a00640 1 -- 172.21.15.123:0/1183740570 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c0032440 con 0x7f76d40c9f00 2024-04-25T12:35:25.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.466+0000 7f76d0a00640 1 -- 172.21.15.123:0/1183740570 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0032810 con 0x7f76d40c9f00 2024-04-25T12:35:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 -- 172.21.15.123:0/1183740570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40c9f00 msgr2=0x7f76d40c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 --2- 172.21.15.123:0/1183740570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40c9f00 0x7f76d40c8170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76d4071a20 tx=0x7f76c002a270 comp rx=0 tx=0).stop 2024-04-25T12:35:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 -- 172.21.15.123:0/1183740570 shutdown_connections 2024-04-25T12:35:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 --2- 172.21.15.123:0/1183740570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40c9f00 0x7f76d40c8170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 --2- 172.21.15.123:0/1183740570 >> v2:172.21.15.153:3300/0 conn(0x7f76d4154650 0x7f76d415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 -- 172.21.15.123:0/1183740570 >> 172.21.15.123:0/1183740570 conn(0x7f76d406f440 msgr2=0x7f76d406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 -- 172.21.15.123:0/1183740570 shutdown_connections 2024-04-25T12:35:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.467+0000 7f76d8e00640 1 -- 172.21.15.123:0/1183740570 wait complete. 2024-04-25T12:35:25.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.469+0000 7f76d8e00640 1 Processor -- start 2024-04-25T12:35:25.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.470+0000 7f76d8e00640 1 -- start start 2024-04-25T12:35:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.471+0000 7f76d8e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76d40730e0 0x7f76d4158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.471+0000 7f76d8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76d40c9f00 0x7f76d4152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.471+0000 7f76d8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d4154650 0x7f76d41f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.471+0000 7f76d8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76d415e010 con 0x7f76d4154650 2024-04-25T12:35:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.471+0000 7f76d8e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76d415dd30 con 0x7f76d40c9f00 2024-04-25T12:35:25.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.471+0000 7f76d8e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f76d415e2f0 con 0x7f76d40730e0 2024-04-25T12:35:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.471+0000 7f76d1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76d40c9f00 0x7f76d4152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76d40c9f00 0x7f76d4152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41036/0 (socket says 172.21.15.123:41036) 2024-04-25T12:35:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 -- 172.21.15.123:0/2632969679 learned_addr learned my addr 172.21.15.123:0/2632969679 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76d40730e0 0x7f76d4158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d2800640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d4154650 0x7f76d41f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76d40730e0 msgr2=0x7f76d4158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76d40730e0 0x7f76d4158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d4154650 msgr2=0x7f76d41f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d4154650 0x7f76d41f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d41588b0 con 0x7f76d40c9f00 2024-04-25T12:35:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.472+0000 7f76d1400640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76d40c9f00 0x7f76d4152e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.473+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037040 con 0x7f76d40c9f00 2024-04-25T12:35:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.473+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c0042070 con 0x7f76d40c9f00 2024-04-25T12:35:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.473+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0030880 con 0x7f76d40c9f00 2024-04-25T12:35:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.473+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41f0ad0 con 0x7f76d40c9f00 2024-04-25T12:35:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.473+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41f0d90 con 0x7f76d40c9f00 2024-04-25T12:35:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.474+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f76d4076700 con 0x7f76d40c9f00 2024-04-25T12:35:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.475+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f76c0030a20 con 0x7f76d40c9f00 2024-04-25T12:35:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.476+0000 7f76ca000640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76b40744b0 0x7f76b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.476+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6619+0+0 (secure 0 0 0) 0x7f76c00b5e30 con 0x7f76d40c9f00 2024-04-25T12:35:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.477+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76b40744b0 0x7f76b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.477+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v33) ==== 54+0+16 (secure 0 0 0) 0x7f76c00b83d0 con 0x7f76d40c9f00 2024-04-25T12:35:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.477+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f76d40c6b00 0x7f76d41f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.477+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76d40744e0 con 0x7f76d40c6b00 2024-04-25T12:35:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.477+0000 7f76d2800640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f76d40c6b00 0x7f76d41f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.477+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76b40744b0 0x7f76b40769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009560 tx=0x7f76c4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.478+0000 7f76d2800640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f76d40c6b00 0x7f76d41f1070 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.479+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d40744e0 con 0x7f76d40c6b00 2024-04-25T12:35:25.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:25.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.495+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d4073c20 con 0x7f76d40c6b00 2024-04-25T12:35:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.497+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d4073c20 con 0x7f76d40c6b00 2024-04-25T12:35:25.497 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.498+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f76d41f47c0 0x7f76d4146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.498+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f76d41f4bd0 con 0x7f76d41f47c0 2024-04-25T12:35:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.498+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f76d41f47c0 0x7f76d4146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.498+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f76d41f47c0 0x7f76d4146af0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.499+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d41f4bd0 con 0x7f76d41f47c0 2024-04-25T12:35:25.512 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:25.512+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:25.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.516+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d40c8d60 con 0x7f76d41f47c0 2024-04-25T12:35:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.518+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d40c8d60 con 0x7f76d41f47c0 2024-04-25T12:35:25.518 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.518+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f76d4147540 0x7f76d4062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.518+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f76d4147950 con 0x7f76d4147540 2024-04-25T12:35:25.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.518+0000 7f76d1400640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f76d4147540 0x7f76d4062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.519+0000 7f76d1400640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f76d4147540 0x7f76d4062fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.520+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d4147950 con 0x7f76d4147540 2024-04-25T12:35:25.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.547+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d41533d0 con 0x7f76d4147540 2024-04-25T12:35:25.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.550+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d41533d0 con 0x7f76d4147540 2024-04-25T12:35:25.550 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.551+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f76d4063a30 0x7f76d40cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.551+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f76d4063e40 con 0x7f76d4063a30 2024-04-25T12:35:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.551+0000 7f76d2800640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f76d4063a30 0x7f76d40cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.552+0000 7f76d2800640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f76d4063a30 0x7f76d40cba00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.552+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d4063e40 con 0x7f76d4063a30 2024-04-25T12:35:25.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.567+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d4063e40 con 0x7f76d4063a30 2024-04-25T12:35:25.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.570+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d4063e40 con 0x7f76d4063a30 2024-04-25T12:35:25.569 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.570+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f76d40cc450 0x7f76d402c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.570+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f76d40cc860 con 0x7f76d40cc450 2024-04-25T12:35:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.571+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f76d40cc450 0x7f76d402c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.571+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f76d40cc450 0x7f76d402c9d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.573+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d40cc860 con 0x7f76d40cc450 2024-04-25T12:35:25.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.588+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d40cc860 con 0x7f76d40cc450 2024-04-25T12:35:25.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.590+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d40cc860 con 0x7f76d40cc450 2024-04-25T12:35:25.589 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:25.590 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:25.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.590+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f76d402d420 0x7f76d402d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.591+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f76d402de70 con 0x7f76d402d420 2024-04-25T12:35:25.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.591+0000 7f76d1400640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f76d402d420 0x7f76d402d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.592+0000 7f76d1400640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f76d402d420 0x7f76d402d830 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.593+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d402de70 con 0x7f76d402d420 2024-04-25T12:35:25.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.608+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d402de40 con 0x7f76d402d420 2024-04-25T12:35:25.610 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.610+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d402de40 con 0x7f76d402d420 2024-04-25T12:35:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.611+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f76d4207820 0x7f76d4207c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.611+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f76d420cd60 con 0x7f76d4207820 2024-04-25T12:35:25.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.611+0000 7f76d2800640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f76d4207820 0x7f76d4207c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.612+0000 7f76d2800640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f76d4207820 0x7f76d4207c30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.613+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d420cd60 con 0x7f76d4207820 2024-04-25T12:35:25.613 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:25.614+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:25.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.628+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d4208280 con 0x7f76d4207820 2024-04-25T12:35:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.631+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d4208280 con 0x7f76d4207820 2024-04-25T12:35:25.630 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.631+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f76d42089a0 0x7f76d4208db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.631+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f76d42094a0 con 0x7f76d42089a0 2024-04-25T12:35:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.632+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f76d42089a0 0x7f76d4208db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.632+0000 7f76d1e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f76d42089a0 0x7f76d4208db0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:25.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.633+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76d42094a0 con 0x7f76d42089a0 2024-04-25T12:35:25.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.648+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76d406dc70 con 0x7f76d42089a0 2024-04-25T12:35:25.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.650+0000 7f76ca000640 1 -- 172.21.15.123:0/2632969679 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76d406dc70 con 0x7f76d42089a0 2024-04-25T12:35:25.650 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:25.651 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:25.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f76d40c6b00 msgr2=0x7f76d41f1070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f76d40c6b00 0x7f76d41f1070 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f76d41f47c0 msgr2=0x7f76d4146af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f76d41f47c0 0x7f76d4146af0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f76d4147540 msgr2=0x7f76d4062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f76d4147540 0x7f76d4062fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f76d4063a30 msgr2=0x7f76d40cba00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f76d4063a30 0x7f76d40cba00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f76d40cc450 msgr2=0x7f76d402c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f76d40cc450 0x7f76d402c9d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f76d402d420 msgr2=0x7f76d402d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f76d402d420 0x7f76d402d830 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f76d4207820 msgr2=0x7f76d4207c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f76d4207820 0x7f76d4207c30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f76d42089a0 msgr2=0x7f76d4208db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f76d42089a0 0x7f76d4208db0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d2800640 1 -- 172.21.15.123:0/2632969679 reap_dead start 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d2800640 1 -- 172.21.15.123:0/2632969679 reap_dead start 2024-04-25T12:35:25.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d2800640 1 -- 172.21.15.123:0/2632969679 reap_dead start 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d2800640 1 -- 172.21.15.123:0/2632969679 reap_dead start 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d2800640 1 -- 172.21.15.123:0/2632969679 reap_dead start 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76b40744b0 msgr2=0x7f76b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76b40744b0 0x7f76b40769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009560 tx=0x7f76c4009190 comp rx=0 tx=0).stop 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76d40c9f00 msgr2=0x7f76d4152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.651+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76d40c9f00 0x7f76d4152e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c00046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.652+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 shutdown_connections 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.652+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76b40744b0 0x7f76b40769c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.652+0000 7f76d8e00640 1 --2- 172.21.15.123:0/2632969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76d40c9f00 0x7f76d4152e90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.652+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 >> 172.21.15.123:0/2632969679 conn(0x7f76d406f440 msgr2=0x7f76d40c63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.652+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 shutdown_connections 2024-04-25T12:35:25.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:25.652+0000 7f76d8e00640 1 -- 172.21.15.123:0/2632969679 wait complete. 2024-04-25T12:35:25.714 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:25.713+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:25.733 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:25.733 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:25.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:25.814 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:25.815+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:25.864 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:35:25.864 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:35:25.864 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-04-25T12:35:25.864 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-25T12:35:25.864 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-25T12:35:25.865 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-04-25T12:35:25.915 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:25.914+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:25.994 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:25.994 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-25T12:35:25.994 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.991098625000006, 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53759151699999996, 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.020 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:26.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024334+0000", 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7179999999999999e-06 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024345+0000", 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0608000000000001e-05 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024373+0000", 2024-04-25T12:35:26.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7966e-05 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024392+0000", 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024490+0000", 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7823000000000002e-05 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for peered", 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024520+0000", 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0056000000000002e-05 2024-04-25T12:35:26.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.559770+0000", 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53525036000000004 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560001+0000", 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023139100000000001 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560330+0000", 2024-04-25T12:35:26.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032824500000000001 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561700+0000", 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013704730000000001 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561742+0000", 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1826000000000002e-05 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561855+0000", 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011287599999999999 2024-04-25T12:35:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561924+0000", 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9484000000000001e-05 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.428175405999994, 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095393000000000006, 2024-04-25T12:35:26.026 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587257+0000", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9100000000000003e-07 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587490+0000", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000233186 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587502+0000", 2024-04-25T12:35:26.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2499e-05 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587512+0000", 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7699999999999996e-06 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.591891+0000", 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043783770000000001 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592036+0000", 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014543999999999999 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592306+0000", 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026964399999999998 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595185+0000", 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028791730000000001 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596722+0000", 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001537131 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596762+0000", 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0491000000000002e-05 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596795+0000", 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2798e-05 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.427902486999997, 2024-04-25T12:35:26.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012300247, 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 24 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587530+0000", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3799999999999996e-07 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587688+0000", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015835999999999999 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587699+0000", 2024-04-25T12:35:26.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0727000000000001e-05 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587710+0000", 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0987e-05 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593399+0000", 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005688935 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593685+0000", 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028662199999999999 2024-04-25T12:35:26.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593893+0000", 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020758500000000001 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595599+0000", 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017058209999999999 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599668+0000", 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040696439999999999 2024-04-25T12:35:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599769+0000", 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100752 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599829+0000", 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0075999999999999e-05 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.036 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.427680429000006, 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012351434, 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 25 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587752+0000", 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587758+0000", 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2199999999999997e-06 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587770+0000", 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2129e-05 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587778+0000", 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594340+0000", 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065618270000000001 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594415+0000", 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4535000000000004e-05 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594510+0000", 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5100999999999996e-05 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595607+0000", 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010968239999999999 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599926+0000", 2024-04-25T12:35:26.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043194469999999997 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600027+0000", 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100961 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600102+0000", 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5339000000000005e-05 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.222731238999998, 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091591999999999993, 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792701+0000", 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.17e-06 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792911+0000", 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000210163 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792932+0000", 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0472e-05 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792952+0000", 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0553000000000001e-05 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799157+0000", 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062046030000000004 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799246+0000", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8505999999999996e-05 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799389+0000", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014319100000000001 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801232+0000", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018427599999999999 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801753+0000", 2024-04-25T12:35:26.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052153699999999998 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801814+0000", 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1014000000000002e-05 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801859+0000", 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5231000000000001e-05 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.046 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.222292890999995, 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092910930000000003, 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 52 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793140+0000", 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-06 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793151+0000", 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0966e-05 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793168+0000", 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793184+0000", 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6200000000000001e-05 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801271+0000", 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080863189999999998 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801360+0000", 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9248999999999999e-05 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801515+0000", 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000154883 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802197+0000", 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068243299999999998 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802339+0000", 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014159 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802393+0000", 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4215000000000002e-05 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802430+0000", 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6434999999999999e-05 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.053 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.764289621000003, 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012798129, 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 70 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.054 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251143+0000", 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2729999999999999e-06 2024-04-25T12:35:26.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251153+0000", 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8439999999999993e-06 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251172+0000", 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8906000000000002e-05 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251183+0000", 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1653e-05 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257332+0000", 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061486459999999998 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257490+0000", 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000157947 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257724+0000", 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023408200000000001 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259695+0000", 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001971169 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263731+0000", 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035683 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263842+0000", 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110985 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263940+0000", 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7941000000000003e-05 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.764232711999995, 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013164844, 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 71 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251200+0000", 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2999999999999999e-07 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251206+0000", 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9399999999999999e-06 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251215+0000", 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251223+0000", 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2260000000000005e-06 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259955+0000", 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087315629999999995 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260077+0000", 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122263 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260310+0000", 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232702 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263699+0000", 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033888080000000001 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264218+0000", 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051973199999999996 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264304+0000", 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5433999999999996e-05 2024-04-25T12:35:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264363+0000", 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9743e-05 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.764191103000002, 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017184050999999999, 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 72 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251241+0000", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.23e-07 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251477+0000", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023584499999999999 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251504+0000", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7633000000000001e-05 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251516+0000", 2024-04-25T12:35:26.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.164e-05 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260993+0000", 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094771119999999993 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261122+0000", 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012831400000000001 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261319+0000", 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019727199999999999 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264078+0000", 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027589120000000001 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268258+0000", 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041806569999999999 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268357+0000", 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8721000000000006e-05 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268424+0000", 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7222000000000001e-05 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.763896160000002, 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017105597, 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.069 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 73 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251536+0000", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4300000000000001e-07 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251543+0000", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3840000000000002e-06 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251552+0000", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6860000000000001e-06 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251560+0000", 2024-04-25T12:35:26.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0499999999999992e-06 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262615+0000", 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011054494 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262757+0000", 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142521 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262999+0000", 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024147400000000001 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264190+0000", 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011911840000000001 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268472+0000", 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042823940000000001 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268571+0000", 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8218000000000001e-05 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268641+0000", 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0248999999999998e-05 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.763859174000004, 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017309201999999999, 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.074 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 74 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251573+0000", 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9999999999999997e-07 2024-04-25T12:35:26.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251579+0000", 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6330000000000001e-06 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251592+0000", 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3869999999999999e-05 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251621+0000", 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8966e-05 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264505+0000", 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012883307 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264627+0000", 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012204 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264832+0000", 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020571899999999999 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268227+0000", 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003394308 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268663+0000", 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043630699999999998 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268761+0000", 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7774000000000003e-05 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268881+0000", 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120578 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.763789064999997, 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01751983, 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 75 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251643+0000", 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5300000000000003e-07 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251849+0000", 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000206345 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251863+0000", 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3077e-05 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251873+0000", 2024-04-25T12:35:26.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.08e-05 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265525+0000", 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013651876 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265667+0000", 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014156699999999999 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265939+0000", 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027219700000000001 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268931+0000", 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029921990000000001 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269008+0000", 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6619000000000006e-05 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269082+0000", 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4401999999999997e-05 2024-04-25T12:35:26.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269162+0000", 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9895000000000007e-05 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.763541396999997, 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018357248, 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.084 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 76 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251891+0000", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7300000000000005e-07 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251905+0000", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4279e-05 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251915+0000", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251935+0000", 2024-04-25T12:35:26.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.961e-05 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267152+0000", 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015217341000000001 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267276+0000", 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012396 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267509+0000", 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023329500000000001 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268988+0000", 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014781709999999999 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270041+0000", 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001053806 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270161+0000", 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119494 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270247+0000", 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6382000000000001e-05 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.744316888, 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099439890000000003, 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 80 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271115+0000", 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271221+0000", 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105713 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271236+0000", 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5415999999999999e-05 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271240+0000", 2024-04-25T12:35:26.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3449999999999999e-06 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275441+0000", 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042009090000000001 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275579+0000", 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013740700000000001 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275795+0000", 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021634900000000001 2024-04-25T12:35:26.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.277926+0000", 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021311220000000001 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.280958+0000", 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030320490000000002 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281016+0000", 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8232000000000001e-05 2024-04-25T12:35:26.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281058+0000", 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1993999999999998e-05 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.389300523000003, 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094693499999999996, 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.094 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 92 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626132+0000", 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4399999999999999e-07 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626137+0000", 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4929999999999998e-06 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626147+0000", 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5300000000000002e-06 2024-04-25T12:35:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626155+0000", 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4130000000000006e-06 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629532+0000", 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033774830000000001 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629626+0000", 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3394999999999996e-05 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629792+0000", 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016611100000000001 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633343+0000", 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00355055 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635469+0000", 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021262009999999999 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635548+0000", 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9167999999999998e-05 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635600+0000", 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2262000000000002e-05 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.389264186999995, 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096152330000000008, 2024-04-25T12:35:26.099 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 93 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626168+0000", 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9699999999999995e-07 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626290+0000", 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121941 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626304+0000", 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3729e-05 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626321+0000", 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7062999999999999e-05 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630147+0000", 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003826 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630229+0000", 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1953000000000006e-05 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630372+0000", 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014345199999999999 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633348+0000", 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029755789999999999 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635672+0000", 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023242829999999999 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635733+0000", 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0899000000000001e-05 2024-04-25T12:35:26.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635782+0000", 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9636999999999999e-05 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.389090267, 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097157200000000006, 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.104 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 94 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626342+0000", 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4300000000000002e-07 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626348+0000", 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0059999999999998e-06 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626358+0000", 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626367+0000", 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8939999999999999e-06 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631146+0000", 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047795429999999998 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631307+0000", 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016041699999999999 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631539+0000", 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232258 2024-04-25T12:35:26.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635089+0000", 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035503980000000002 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635880+0000", 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079070200000000001 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635984+0000", 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104221 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.636057+0000", 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2597999999999998e-05 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 35.897380714999997, 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010749583, 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118054+0000", 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118075+0000", 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118119+0000", 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4372000000000001e-05 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118163+0000", 2024-04-25T12:35:26.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3766000000000001e-05 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119484+0000", 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001320494 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119803+0000", 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031975599999999998 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120121+0000", 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031758999999999997 2024-04-25T12:35:26.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128491+0000", 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083702940000000003 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128605+0000", 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113158 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128733+0000", 2024-04-25T12:35:26.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012861099999999999 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128800+0000", 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7086000000000001e-05 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 35.897181729000003, 2024-04-25T12:35:26.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011301215999999999, 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118251+0000", 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6160000000000001e-06 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118263+0000", 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2243e-05 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118305+0000", 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1838999999999997e-05 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118327+0000", 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2082999999999998e-05 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120981+0000", 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002653201 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121221+0000", 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000240058 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121655+0000", 2024-04-25T12:35:26.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043405400000000002 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128501+0000", 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068465519999999997 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128984+0000", 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048240400000000003 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129487+0000", 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050289499999999995 2024-04-25T12:35:26.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129551+0000", 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4270999999999995e-05 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 35.897074298, 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012520414000000001, 2024-04-25T12:35:26.119 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:26.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118358+0000", 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3230000000000001e-06 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118368+0000", 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3899999999999999e-06 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2481999999999999e-05 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118410+0000", 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9304e-05 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126108+0000", 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076979409999999998 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:26.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126379+0000", 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027136700000000001 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126653+0000", 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274238 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129707+0000", 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030541800000000001 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:26.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130686+0000", 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097908900000000004 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130791+0000", 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104824 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130877+0000", 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6275999999999999e-05 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.124 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:26.125 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:26.125 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:26.125 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:26.044+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:26.126 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:26.127 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:26.126+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:26.227 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:26.226+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:26.327 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:26.328+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:26.428 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:26.428+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:26.443 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:26.443 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:26.443 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:26.443 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:26.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:26.529 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:26.530+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:26.630 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:26.631+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:26.731 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:26.732+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:26.731 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:26.731 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:26.731 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:26.731 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:26.731 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:26.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:26.831 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:26.831+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:26.932 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:26.933+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:27.032 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:27.033+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.110961395999993, 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:35:27.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:35:27.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.069 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.110849453, 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:35:27.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.110800652999998, 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.110729492999994, 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:35:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.110681080999996, 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:35:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.102485153000003, 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:35:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:35:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 50.996241155, 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:35:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 50.996059154000001, 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:35:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:35:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.944025166999999, 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:35:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.943949211000003, 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.939520471000002, 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:35:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:35:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:35:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.939338407999998, 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:35:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:35:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.939234044000003, 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:35:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.937958487000003, 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:35:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:35:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.936904263000002, 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:22 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.933537917000002, 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026067460000000001, 2024-04-25T12:35:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128290+0000", 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.474e-06 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128302+0000", 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2001000000000001e-05 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128323+0000", 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0613999999999999e-05 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128339+0000", 2024-04-25T12:35:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5713999999999999e-05 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128423+0000", 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4154999999999996e-05 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128451+0000", 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8473999999999999e-05 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130796+0000", 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002344245 2024-04-25T12:35:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130896+0000", 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010006899999999999 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.908038622999999, 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:35:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:35:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:44 2.14 2:2e2fa760:::200.00000001:head [write 3170~258 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.186753805999999, 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026976719999999999, 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875077+0000", 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0749999999999999e-06 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875102+0000", 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5102000000000001e-05 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875144+0000", 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1647999999999998e-05 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875170+0000", 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5876000000000001e-05 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875363+0000", 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019320700000000001 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875551+0000", 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000188037 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875700+0000", 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148837 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877284+0000", 2024-04-25T12:35:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001584483 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877626+0000", 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034161599999999999 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877700+0000", 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4467000000000004e-05 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877771+0000", 2024-04-25T12:35:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0324e-05 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.186527128000002, 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:35:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:35:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.143966576, 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:35:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:35:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:35:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:35:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:35:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:35:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:27.154 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:27.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:27.156 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:27.155+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:27.256 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:27.257+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:27.356 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:27.356+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:27.420 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:35:27.457 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:27.457+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:27.474 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:27.474 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:27.474 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:27.474 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:27.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:27.558 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:27.559+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:27.658 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:27.658+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:27.674 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:27.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.689+0000 7fb124400640 1 Processor -- start 2024-04-25T12:35:27.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.689+0000 7fb124400640 1 -- start start 2024-04-25T12:35:27.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.692+0000 7fb124400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c0c9b80 0x7fb11c0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.692+0000 7fb124400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb11c159f30 0x7fb11c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.692+0000 7fb124400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb11c156e60 con 0x7fb11c0c8250 2024-04-25T12:35:27.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.692+0000 7fb124400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb11c0c8ba0 con 0x7fb11c159f30 2024-04-25T12:35:27.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.692+0000 7fb124400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c15ca70 con 0x7fb11c0c9b80 2024-04-25T12:35:27.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb122400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb11c159f30 0x7fb11c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb122400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb11c159f30 0x7fb11c0c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54776/0 (socket says 172.21.15.123:54776) 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb122400640 1 -- 172.21.15.123:0/1276573276 learned_addr learned my addr 172.21.15.123:0/1276573276 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb121000640 1 --1- 172.21.15.123:0/1276573276 >> v1:172.21.15.153:6789/0 conn(0x7fb11c0c8250 0x7fb11c159820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34900/0 (socket says 172.21.15.123:34900) 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb121a00640 1 --2- 172.21.15.123:0/1276573276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c0c9b80 0x7fb11c0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb122400640 1 -- 172.21.15.123:0/1276573276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c0c9b80 msgr2=0x7fb11c0742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb122400640 1 --2- 172.21.15.123:0/1276573276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c0c9b80 0x7fb11c0742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb11be00640 1 -- 172.21.15.123:0/1276573276 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358156767 0 0) 0x7fb11c156e60 con 0x7fb11c0c8250 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb122400640 1 -- 172.21.15.123:0/1276573276 >> v1:172.21.15.153:6789/0 conn(0x7fb11c0c8250 legacy=0x7fb11c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.693+0000 7fb122400640 1 -- 172.21.15.123:0/1276573276 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c0c7130 con 0x7fb11c159f30 2024-04-25T12:35:27.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.694+0000 7fb122400640 1 --2- 172.21.15.123:0/1276573276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c159f30 0x7fb11c0c8660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb114009e50 tx=0x7fb11402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8209a9df88baca09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.695+0000 7fb11be00640 1 -- 172.21.15.123:0/1276573276 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114036070 con 0x7fb11c159f30 2024-04-25T12:35:27.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.695+0000 7fb11be00640 1 -- 172.21.15.123:0/1276573276 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb114031600 con 0x7fb11c159f30 2024-04-25T12:35:27.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.695+0000 7fb11be00640 1 -- 172.21.15.123:0/1276573276 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1140319d0 con 0x7fb11c159f30 2024-04-25T12:35:27.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 -- 172.21.15.123:0/1276573276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c159f30 msgr2=0x7fb11c0c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 --2- 172.21.15.123:0/1276573276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c159f30 0x7fb11c0c8660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb114009e50 tx=0x7fb11402a710 comp rx=0 tx=0).stop 2024-04-25T12:35:27.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 -- 172.21.15.123:0/1276573276 shutdown_connections 2024-04-25T12:35:27.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 --2- 172.21.15.123:0/1276573276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c159f30 0x7fb11c0c8660 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 --2- 172.21.15.123:0/1276573276 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c0c9b80 0x7fb11c0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 -- 172.21.15.123:0/1276573276 >> 172.21.15.123:0/1276573276 conn(0x7fb11c06c750 msgr2=0x7fb11c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:27.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 -- 172.21.15.123:0/1276573276 shutdown_connections 2024-04-25T12:35:27.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.696+0000 7fb124400640 1 -- 172.21.15.123:0/1276573276 wait complete. 2024-04-25T12:35:27.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.699+0000 7fb124400640 1 Processor -- start 2024-04-25T12:35:27.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.699+0000 7fb124400640 1 -- start start 2024-04-25T12:35:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb124400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c0c8250 0x7fb11c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb124400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb11c0c9b80 0x7fb11c0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb124400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c159f30 0x7fb11c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb124400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c15d310 con 0x7fb11c0c8250 2024-04-25T12:35:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb124400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c15d030 con 0x7fb11c0c9b80 2024-04-25T12:35:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb124400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c15d5f0 con 0x7fb11c159f30 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb121a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c0c8250 0x7fb11c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb121000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb11c0c9b80 0x7fb11c0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb122400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c159f30 0x7fb11c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb121000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb11c0c9b80 0x7fb11c0709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41048/0 (socket says 172.21.15.123:41048) 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.701+0000 7fb121000640 1 -- 172.21.15.123:0/538445259 learned_addr learned my addr 172.21.15.123:0/538445259 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb121a00640 1 -- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c159f30 msgr2=0x7fb11c1542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb121a00640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c159f30 0x7fb11c1542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb121a00640 1 -- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb11c0c9b80 msgr2=0x7fb11c0709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb121a00640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb11c0c9b80 0x7fb11c0709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb121a00640 1 -- 172.21.15.123:0/538445259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c157660 con 0x7fb11c0c8250 2024-04-25T12:35:27.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb122400640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c159f30 0x7fb11c1542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:27.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb121000640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb11c0c9b80 0x7fb11c0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:27.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.702+0000 7fb121a00640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c0c8250 0x7fb11c0b1a60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb11000e8a0 tx=0x7fb11000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.703+0000 7fb119600640 1 -- 172.21.15.123:0/538445259 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110017070 con 0x7fb11c0c8250 2024-04-25T12:35:27.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.703+0000 7fb119600640 1 -- 172.21.15.123:0/538445259 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb11000c940 con 0x7fb11c0c8250 2024-04-25T12:35:27.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.703+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c0af990 con 0x7fb11c0c8250 2024-04-25T12:35:27.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.703+0000 7fb119600640 1 -- 172.21.15.123:0/538445259 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110009e60 con 0x7fb11c0c8250 2024-04-25T12:35:27.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.703+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c1f2ad0 con 0x7fb11c0c8250 2024-04-25T12:35:27.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.704+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c076700 con 0x7fb11c0c8250 2024-04-25T12:35:27.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.705+0000 7fb119600640 1 -- 172.21.15.123:0/538445259 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb11001a070 con 0x7fb11c0c8250 2024-04-25T12:35:27.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.707+0000 7fb119600640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0fc0744e0 0x7fb0fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.707+0000 7fb119600640 1 -- 172.21.15.123:0/538445259 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6619+0+0 (secure 0 0 0) 0x7fb1100956a0 con 0x7fb11c0c8250 2024-04-25T12:35:27.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.707+0000 7fb121000640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0fc0744e0 0x7fb0fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.708+0000 7fb121000640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0fc0744e0 0x7fb0fc0769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb108000f60 tx=0x7fb108009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.712+0000 7fb119600640 1 -- 172.21.15.123:0/538445259 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb11005f440 con 0x7fb11c0c8250 2024-04-25T12:35:27.747 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:27.747 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:27.747 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:27.747 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:27.747 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:27.759 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:27.760+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:27.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:27.861 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:27.861+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.918+0000 7f1af3800640 1 Processor -- start 2024-04-25T12:35:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.919+0000 7f1af3800640 1 -- start start 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.920+0000 7f1af3800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1aec0c9f00 0x7f1aec0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.920+0000 7f1af3800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aec074b20 con 0x7f1aec0730e0 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1af3800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aec0ca780 con 0x7f1aec154650 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1af3800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1aec0caa60 con 0x7f1aec0c9f00 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1aebe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1aec0c9f00 0x7f1aec0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1af0e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1aec154650 0x7f1aec15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34912/0 (socket says 172.21.15.123:34912) 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1af0e00640 1 -- 172.21.15.123:0/1023446758 learned_addr learned my addr 172.21.15.123:0/1023446758 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1af1800640 1 --1- 172.21.15.123:0/1023446758 >> v1:172.21.15.123:6789/0 conn(0x7f1aec0730e0 0x7f1aec164d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:52784/0 (socket says 172.21.15.123:52784) 2024-04-25T12:35:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1aebe00640 1 -- 172.21.15.123:0/1023446758 >> v1:172.21.15.153:6789/0 conn(0x7f1aec154650 legacy=0x7f1aec15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1aebe00640 1 -- 172.21.15.123:0/1023446758 >> v1:172.21.15.123:6789/0 conn(0x7f1aec0730e0 legacy=0x7f1aec164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.921+0000 7f1aeb400640 1 -- 172.21.15.123:0/1023446758 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2969737064 0 0) 0x7f1aec0ca780 con 0x7f1aec154650 2024-04-25T12:35:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.922+0000 7f1aebe00640 1 -- 172.21.15.123:0/1023446758 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec15da50 con 0x7f1aec0c9f00 2024-04-25T12:35:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.923+0000 7f1aebe00640 1 --2- 172.21.15.123:0/1023446758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0c9f00 0x7f1aec0c8170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097a0 tx=0x7f1ad802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c5fa74d6644ccb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.923+0000 7f1aeb400640 1 -- 172.21.15.123:0/1023446758 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8037070 con 0x7f1aec0c9f00 2024-04-25T12:35:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.923+0000 7f1aeb400640 1 -- 172.21.15.123:0/1023446758 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad802adc0 con 0x7f1aec0c9f00 2024-04-25T12:35:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.923+0000 7f1aeb400640 1 -- 172.21.15.123:0/1023446758 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8032660 con 0x7f1aec0c9f00 2024-04-25T12:35:27.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.924+0000 7f1af3800640 1 -- 172.21.15.123:0/1023446758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0c9f00 msgr2=0x7f1aec0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.924+0000 7f1af3800640 1 --2- 172.21.15.123:0/1023446758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0c9f00 0x7f1aec0c8170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097a0 tx=0x7f1ad802a130 comp rx=0 tx=0).stop 2024-04-25T12:35:27.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.924+0000 7f1af3800640 1 -- 172.21.15.123:0/1023446758 shutdown_connections 2024-04-25T12:35:27.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.924+0000 7f1af3800640 1 --2- 172.21.15.123:0/1023446758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0c9f00 0x7f1aec0c8170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.924+0000 7f1af3800640 1 -- 172.21.15.123:0/1023446758 >> 172.21.15.123:0/1023446758 conn(0x7f1aec06f440 msgr2=0x7f1aec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:27.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.924+0000 7f1af3800640 1 -- 172.21.15.123:0/1023446758 shutdown_connections 2024-04-25T12:35:27.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.924+0000 7f1af3800640 1 -- 172.21.15.123:0/1023446758 wait complete. 2024-04-25T12:35:27.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.926+0000 7f1af3800640 1 Processor -- start 2024-04-25T12:35:27.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.926+0000 7f1af3800640 1 -- start start 2024-04-25T12:35:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af3800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0730e0 0x7f1aec1e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af3800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1aec0c9f00 0x7f1aec15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af3800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aec154650 0x7f1aec146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af3800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec15e2f0 con 0x7f1aec154650 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af3800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec15e010 con 0x7f1aec0730e0 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af3800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1aec154a60 con 0x7f1aec0c9f00 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1aebe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1aec0c9f00 0x7f1aec15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af0e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0730e0 0x7f1aec1e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1aebe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1aec0c9f00 0x7f1aec15c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35414/0 (socket says 172.21.15.123:35414) 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1aebe00640 1 -- 172.21.15.123:0/2228360651 learned_addr learned my addr 172.21.15.123:0/2228360651 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.928+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aec154650 0x7f1aec146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1aebe00640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0730e0 msgr2=0x7f1aec1e71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1aebe00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1aec0730e0 0x7f1aec1e71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1aebe00640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aec154650 msgr2=0x7f1aec146af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1aebe00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aec154650 0x7f1aec146af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1aebe00640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec1e7730 con 0x7f1aec0c9f00 2024-04-25T12:35:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aec154650 0x7f1aec146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1aebe00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1aec0c9f00 0x7f1aec15c480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097a0 tx=0x7f1ad80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8037040 con 0x7f1aec0c9f00 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.929+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad8004030 con 0x7f1aec0c9f00 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.930+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8004400 con 0x7f1aec0c9f00 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.930+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aec0c36f0 con 0x7f1aec0c9f00 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.930+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aec0c39b0 con 0x7f1aec0c9f00 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.931+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f1aec15e8a0 con 0x7f1aec0c9f00 2024-04-25T12:35:27.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.931+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fb11c0c9590 con 0x7fb11c0c8250 2024-04-25T12:35:27.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.932+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ad803a070 con 0x7f1aec0c9f00 2024-04-25T12:35:27.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.933+0000 7f1ae8c00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1acc0744e0 0x7f1acc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.934+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 6619+0+0 (secure 0 0 0) 0x7f1ad80b57c0 con 0x7f1aec0c9f00 2024-04-25T12:35:27.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.934+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v33) ==== 54+0+16 (secure 0 0 0) 0x7f1aec15e8a0 con 0x7f1aec0c9f00 2024-04-25T12:35:27.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.934+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1acc0744e0 0x7f1acc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.934+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f1aec0c3b80 0x7f1aec147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.934+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1aec0b2250 con 0x7f1aec0c3b80 2024-04-25T12:35:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.935+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1acc0744e0 0x7f1acc0769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0000f60 tx=0x7f1ae0007e40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.935+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f1aec0c3b80 0x7f1aec147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.935+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f1aec0c3b80 0x7f1aec147030 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.937+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec0b2250 con 0x7f1aec0c3b80 2024-04-25T12:35:27.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.953+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec076700 con 0x7f1aec0c3b80 2024-04-25T12:35:27.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.956+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec076700 con 0x7f1aec0c3b80 2024-04-25T12:35:27.955 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.956+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f1aec062fe0 0x7f1aec0633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.956+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1aec158010 con 0x7f1aec062fe0 2024-04-25T12:35:27.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.957+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f1aec062fe0 0x7f1aec0633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.958+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f1aec062fe0 0x7f1aec0633f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.959+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec158010 con 0x7f1aec062fe0 2024-04-25T12:35:27.961 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:27.962+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:27.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.975+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec0744e0 con 0x7f1aec062fe0 2024-04-25T12:35:27.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.979+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec0744e0 con 0x7f1aec062fe0 2024-04-25T12:35:27.978 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:27.979 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:27.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.979+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f1aec0cb9e0 0x7f1aec0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.979+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f1aec071550 con 0x7f1aec0cb9e0 2024-04-25T12:35:27.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.980+0000 7f1aebe00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f1aec0cb9e0 0x7f1aec0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.980+0000 7f1aebe00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f1aec0cb9e0 0x7f1aec0cbdf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:27.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.981+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec071550 con 0x7f1aec0cb9e0 2024-04-25T12:35:27.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.996+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec071270 con 0x7f1aec0cb9e0 2024-04-25T12:35:27.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.998+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec071270 con 0x7f1aec0cb9e0 2024-04-25T12:35:27.998 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:27.998 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:27.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.999+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f1aec158660 0x7f1aec158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:27.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.999+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f1aec0c4780 con 0x7f1aec158660 2024-04-25T12:35:27.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:27.999+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f1aec158660 0x7f1aec158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:27.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.000+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f1aec158660 0x7f1aec158a70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:28.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.001+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec0c4780 con 0x7f1aec158660 2024-04-25T12:35:28.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.016+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec0c4780 con 0x7f1aec158660 2024-04-25T12:35:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.019+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec0c4780 con 0x7f1aec158660 2024-04-25T12:35:28.018 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:28.019 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:28.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.019+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f1aec1594c0 0x7f1aec02c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:28.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.019+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f1aec15c9c0 con 0x7f1aec1594c0 2024-04-25T12:35:28.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.020+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f1aec1594c0 0x7f1aec02c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.020+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f1aec1594c0 0x7f1aec02c9d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.021+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec15c9c0 con 0x7f1aec1594c0 2024-04-25T12:35:28.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.036+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec15c9c0 con 0x7f1aec1594c0 2024-04-25T12:35:28.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.038+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec15c9c0 con 0x7f1aec1594c0 2024-04-25T12:35:28.038 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:28.038 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:28.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.039+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f1aec02d420 0x7f1aec02d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:28.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.039+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f1aec02dee0 con 0x7f1aec02d420 2024-04-25T12:35:28.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.039+0000 7f1aebe00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f1aec02d420 0x7f1aec02d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:28.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.040+0000 7f1aebe00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f1aec02d420 0x7f1aec02d830 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:28.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.041+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec02dee0 con 0x7f1aec02d420 2024-04-25T12:35:28.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.055+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec02deb0 con 0x7f1aec02d420 2024-04-25T12:35:28.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.058+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec02deb0 con 0x7f1aec02d420 2024-04-25T12:35:28.057 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:28.057 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:28.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.058+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f1aec209cf0 0x7f1aec20c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:28.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.058+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f1aec20eee0 con 0x7f1aec209cf0 2024-04-25T12:35:28.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.059+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f1aec209cf0 0x7f1aec20c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:28.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.060+0000 7f1af1800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f1aec209cf0 0x7f1aec20c110 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.061+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec20eee0 con 0x7f1aec209cf0 2024-04-25T12:35:28.062 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:28.063+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:28.074 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:28.074 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:28.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.075+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec20c650 con 0x7f1aec209cf0 2024-04-25T12:35:28.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.078+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec20c650 con 0x7f1aec209cf0 2024-04-25T12:35:28.077 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:28.078 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:28.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.078+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f1aec20ccd0 0x7f1aec20d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:28.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.078+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f1aec2122d0 con 0x7f1aec20ccd0 2024-04-25T12:35:28.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.079+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f1aec20ccd0 0x7f1aec20d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:28.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.079+0000 7f1af0e00640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f1aec20ccd0 0x7f1aec20d140 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:28.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.080+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aec2122d0 con 0x7f1aec20ccd0 2024-04-25T12:35:28.094 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:28.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.096+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1aec20d790 con 0x7f1aec20ccd0 2024-04-25T12:35:28.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.098+0000 7f1ae8c00640 1 -- 172.21.15.123:0/2228360651 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1aec20d790 con 0x7f1aec20ccd0 2024-04-25T12:35:28.097 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:28.099 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:28.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f1aec0c3b80 msgr2=0x7f1aec147030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f1aec0c3b80 0x7f1aec147030 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f1aec062fe0 msgr2=0x7f1aec0633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f1aec062fe0 0x7f1aec0633f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f1aec0cb9e0 msgr2=0x7f1aec0cbdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f1aec0cb9e0 0x7f1aec0cbdf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f1aec158660 msgr2=0x7f1aec158a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f1aec158660 0x7f1aec158a70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af1800640 1 -- 172.21.15.123:0/2228360651 reap_dead start 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f1aec1594c0 msgr2=0x7f1aec02c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f1aec1594c0 0x7f1aec02c9d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f1aec02d420 msgr2=0x7f1aec02d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f1aec02d420 0x7f1aec02d830 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f1aec209cf0 msgr2=0x7f1aec20c110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f1aec209cf0 0x7f1aec20c110 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f1aec20ccd0 msgr2=0x7f1aec20d140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f1aec20ccd0 0x7f1aec20d140 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1acc0744e0 msgr2=0x7f1acc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1acc0744e0 0x7f1acc0769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0000f60 tx=0x7f1ae0007e40 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1aec0c9f00 msgr2=0x7f1aec15c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 --2- 172.21.15.123:0/2228360651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1aec0c9f00 0x7f1aec15c480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097a0 tx=0x7f1ad80046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af1800640 1 -- 172.21.15.123:0/2228360651 reap_dead start 2024-04-25T12:35:28.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af1800640 1 -- 172.21.15.123:0/2228360651 reap_dead start 2024-04-25T12:35:28.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 shutdown_connections 2024-04-25T12:35:28.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.099+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 >> 172.21.15.123:0/2228360651 conn(0x7f1aec06f440 msgr2=0x7f1aec15a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:28.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.100+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 shutdown_connections 2024-04-25T12:35:28.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.100+0000 7f1af3800640 1 -- 172.21.15.123:0/2228360651 wait complete. 2024-04-25T12:35:28.163 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:28.162+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:28.220 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:28.220 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:28.229 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:28.271 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:28.270+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:28.365 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:28.366+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:28.382 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:28.382 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:28.392 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:28.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.395+0000 7fb119600640 1 -- 172.21.15.123:0/538445259 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v34) ==== 97+0+0 (secure 0 0 0) 0x7fb110063740 con 0x7fb11c0c8250 2024-04-25T12:35:28.394 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:35:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.399+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0fc0744e0 msgr2=0x7fb0fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.399+0000 7fb124400640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0fc0744e0 0x7fb0fc0769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb108000f60 tx=0x7fb108009040 comp rx=0 tx=0).stop 2024-04-25T12:35:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.399+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c0c8250 msgr2=0x7fb11c0b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.399+0000 7fb124400640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c0c8250 0x7fb11c0b1a60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb11000e8a0 tx=0x7fb11000ed60 comp rx=0 tx=0).stop 2024-04-25T12:35:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 shutdown_connections 2024-04-25T12:35:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb0fc0744e0 0x7fb0fc0769f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb11c159f30 0x7fb11c1542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb11c0c9b80 0x7fb11c0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 --2- 172.21.15.123:0/538445259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb11c0c8250 0x7fb11c0b1a60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 >> 172.21.15.123:0/538445259 conn(0x7fb11c06c750 msgr2=0x7fb11c156540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 shutdown_connections 2024-04-25T12:35:28.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:28.401+0000 7fb124400640 1 -- 172.21.15.123:0/538445259 wait complete. 2024-04-25T12:35:28.466 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:28.465+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:28.504 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:28.504 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:28.513 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:35:28.567 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:28.567+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:28.665 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:28.665 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:28.667 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:28.667+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:28.674 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:35:28.769 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:28.769+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:28.827 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:28.827 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:28.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:35:28.871 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:28.870+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:28.949 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:28.949 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:28.958 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:29.004 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:29.003+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:29.069 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:29.101 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:29.102 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:29.102 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:29.102+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:29.202 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:29.202+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:29.256 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:29.256 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:29.265 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:29.304 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:29.304+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:29.404 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:29.404+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:29.422 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:29.422 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:29.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:29.516 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:29.516+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:29.585 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:29.585 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:29.594 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:29.635 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:29.635+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:29.708 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:29.707+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:29.745 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:29.745 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:29.754 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:29.837 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:29.837+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:29.906 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:29.906 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:29.909 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:29.909+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:29.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:30.010 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:30.010+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:30.112 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:30.112+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:30.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:30.202 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:30.202 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:30.212 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:30.212+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:30.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:30.313 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:30.313+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:30.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.390+0000 7f7d09c00640 1 Processor -- start 2024-04-25T12:35:30.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.390+0000 7f7d09c00640 1 -- start start 2024-04-25T12:35:30.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.392+0000 7f7d09c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d04073460 0x7f7d0415d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.392+0000 7f7d09c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d040ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.392+0000 7f7d09c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d04073c10 con 0x7f7d040b2a10 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.392+0000 7f7d09c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0415d7b0 con 0x7f7d04073460 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.392+0000 7f7d09c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7d04156fb0 con 0x7f7d040ca060 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d02a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d04073460 0x7f7d0415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d03400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d040ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d02000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7d040b2a10 0x7f7d040c3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34920/0 (socket says 172.21.15.123:34920) 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d02000640 1 -- 172.21.15.123:0/3238722922 learned_addr learned my addr 172.21.15.123:0/3238722922 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d01600640 1 -- 172.21.15.123:0/3238722922 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884482155 0 0) 0x7f7d04073c10 con 0x7f7d040b2a10 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d01600640 1 -- 172.21.15.123:0/3238722922 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ce8003520 con 0x7f7d040b2a10 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d03400640 1 -- 172.21.15.123:0/3238722922 >> v2:172.21.15.153:3300/0 conn(0x7f7d04073460 msgr2=0x7f7d0415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d03400640 1 --2- 172.21.15.123:0/3238722922 >> v2:172.21.15.153:3300/0 conn(0x7f7d04073460 0x7f7d0415d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d03400640 1 -- 172.21.15.123:0/3238722922 >> v1:172.21.15.153:6789/0 conn(0x7f7d040b2a10 legacy=0x7f7d040c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d03400640 1 -- 172.21.15.123:0/3238722922 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04074100 con 0x7f7d040ca060 2024-04-25T12:35:30.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.393+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3238722922 >> v2:172.21.15.153:3300/0 conn(0x7f7d04073460 0x7f7d0415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.394+0000 7f7d03400640 1 --2- 172.21.15.123:0/3238722922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d040ca470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009d90 tx=0x7f7cfc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad354660d46cb9e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.394+0000 7f7d01600640 1 -- 172.21.15.123:0/3238722922 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc007740 con 0x7f7d040ca060 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d01600640 1 -- 172.21.15.123:0/3238722922 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0078e0 con 0x7f7d040ca060 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d01600640 1 -- 172.21.15.123:0/3238722922 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc007cb0 con 0x7f7d040ca060 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d09c00640 1 -- 172.21.15.123:0/3238722922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 msgr2=0x7f7d040ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3238722922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d040ca470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009d90 tx=0x7f7cfc02a660 comp rx=0 tx=0).stop 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d09c00640 1 -- 172.21.15.123:0/3238722922 shutdown_connections 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3238722922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d040ca470 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3238722922 >> v2:172.21.15.153:3300/0 conn(0x7f7d04073460 0x7f7d0415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d09c00640 1 -- 172.21.15.123:0/3238722922 >> 172.21.15.123:0/3238722922 conn(0x7f7d0406f440 msgr2=0x7f7d0406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.395+0000 7f7d09c00640 1 -- 172.21.15.123:0/3238722922 shutdown_connections 2024-04-25T12:35:30.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.396+0000 7f7d09c00640 1 -- 172.21.15.123:0/3238722922 wait complete. 2024-04-25T12:35:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.398+0000 7f7d09c00640 1 Processor -- start 2024-04-25T12:35:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.398+0000 7f7d09c00640 1 -- start start 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.399+0000 7f7d09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d04073460 0x7f7d041f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.399+0000 7f7d09c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d040b2a10 0x7f7d0415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.399+0000 7f7d09c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d041e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.399+0000 7f7d09c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04070a20 con 0x7f7d04073460 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.399+0000 7f7d09c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04071310 con 0x7f7d040b2a10 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.399+0000 7f7d09c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7d040717a0 con 0x7f7d040ca060 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d02000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d040b2a10 0x7f7d0415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d041e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d041e3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:35428/0 (socket says 172.21.15.123:35428) 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 -- 172.21.15.123:0/3627126111 learned_addr learned my addr 172.21.15.123:0/3627126111 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:30.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d04073460 0x7f7d041f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d040b2a10 msgr2=0x7f7d0415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d040b2a10 0x7f7d0415bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d04073460 msgr2=0x7f7d041f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d04073460 0x7f7d041f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.400+0000 7f7d03400640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041f6ba0 con 0x7f7d040ca060 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.401+0000 7f7d03400640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d041e3c00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009eb0 tx=0x7f7cfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.401+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc03f070 con 0x7f7d040ca060 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.401+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc031bc0 con 0x7f7d040ca060 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.401+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc009450 con 0x7f7d040ca060 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.401+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041f3590 con 0x7f7d040ca060 2024-04-25T12:35:30.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.401+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041575a0 con 0x7f7d040ca060 2024-04-25T12:35:30.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.403+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7d040b3b10 con 0x7f7d040ca060 2024-04-25T12:35:30.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.403+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7cfc03b410 con 0x7f7d040ca060 2024-04-25T12:35:30.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.404+0000 7f7cfaa00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744e0 0x7f7ce40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.405+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 6619+0+0 (secure 0 0 0) 0x7f7cfc0b6bd0 con 0x7f7d040ca060 2024-04-25T12:35:30.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.405+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744e0 0x7f7ce40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.405+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v34) ==== 54+0+16 (secure 0 0 0) 0x7f7d040b3b10 con 0x7f7d040ca060 2024-04-25T12:35:30.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.405+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f7d041f3760 0x7f7d040c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.405+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d041f7280 con 0x7f7d041f3760 2024-04-25T12:35:30.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.406+0000 7f7d02000640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f7d041f3760 0x7f7d040c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.406+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744e0 0x7f7ce40769f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009560 tx=0x7f7cf4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.406+0000 7f7d02000640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f7d041f3760 0x7f7d040c8140 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.407+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7cf0001450 con 0x7f7d041f3760 2024-04-25T12:35:30.414 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:30.414+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:30.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.424+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d04076f10 con 0x7f7d041f3760 2024-04-25T12:35:30.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.426+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d04076f10 con 0x7f7d041f3760 2024-04-25T12:35:30.426 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:30.427 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:30.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.426+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f7d041541f0 0x7f7d041e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.426+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7d04076700 con 0x7f7d041541f0 2024-04-25T12:35:30.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.427+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f7d041541f0 0x7f7d041e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.427+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f7d041541f0 0x7f7d041e4650 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.428+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d04076700 con 0x7f7d041541f0 2024-04-25T12:35:30.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:35:30.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.445+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d040c3eb0 con 0x7f7d041541f0 2024-04-25T12:35:30.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.448+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d040c3eb0 con 0x7f7d041541f0 2024-04-25T12:35:30.448 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.448+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f7d04152930 0x7f7d04152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.448+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f7d04153280 con 0x7f7d04152930 2024-04-25T12:35:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.449+0000 7f7d02000640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f7d04152930 0x7f7d04152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.449+0000 7f7d02000640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f7d04152930 0x7f7d04152d40 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.450+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d04153280 con 0x7f7d04152930 2024-04-25T12:35:30.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.480+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d040c9b90 con 0x7f7d04152930 2024-04-25T12:35:30.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.482+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d040c9b90 con 0x7f7d04152930 2024-04-25T12:35:30.483 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:30.483 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:30.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.483+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f7d040634f0 0x7f7d04063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.483+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f7d040b02a0 con 0x7f7d040634f0 2024-04-25T12:35:30.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.483+0000 7f7d03400640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f7d040634f0 0x7f7d04063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.484+0000 7f7d03400640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f7d040634f0 0x7f7d04063900 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.485+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d040b02a0 con 0x7f7d040634f0 2024-04-25T12:35:30.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.515+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d040b02a0 con 0x7f7d040634f0 2024-04-25T12:35:30.515 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:30.515+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:30.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.517+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d040b02a0 con 0x7f7d040634f0 2024-04-25T12:35:30.517 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:30.518 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:30.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.517+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f7d0415a750 0x7f7d0415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.517+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f7d041e5980 con 0x7f7d0415a750 2024-04-25T12:35:30.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.517+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f7d0415a750 0x7f7d0415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.518+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f7d0415a750 0x7f7d0415ab60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.519+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d041e5980 con 0x7f7d0415a750 2024-04-25T12:35:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.534+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d041e5980 con 0x7f7d0415a750 2024-04-25T12:35:30.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.536+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d041e5980 con 0x7f7d0415a750 2024-04-25T12:35:30.536 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:30.537 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:30.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.536+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f7d0415b0a0 0x7f7d0402cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.536+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f7d0402d4a0 con 0x7f7d0415b0a0 2024-04-25T12:35:30.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.537+0000 7f7d02000640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f7d0415b0a0 0x7f7d0402cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.538+0000 7f7d02000640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f7d0415b0a0 0x7f7d0402cee0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.539+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d0402d4a0 con 0x7f7d0415b0a0 2024-04-25T12:35:30.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.554+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d0406dc70 con 0x7f7d0415b0a0 2024-04-25T12:35:30.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.556+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d0406dc70 con 0x7f7d0415b0a0 2024-04-25T12:35:30.557 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:30.557 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:30.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.557+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f7d0402dc40 0x7f7d0420c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.557+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f7d0420f030 con 0x7f7d0402dc40 2024-04-25T12:35:30.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.557+0000 7f7d03400640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f7d0402dc40 0x7f7d0420c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.558+0000 7f7d03400640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f7d0402dc40 0x7f7d0420c0b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.559+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d0420f030 con 0x7f7d0402dc40 2024-04-25T12:35:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:30.561 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:30.561 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:30.561 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:30.561 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:30.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.574+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d0402d4a0 con 0x7f7d0402dc40 2024-04-25T12:35:30.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.576+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d0402d4a0 con 0x7f7d0402dc40 2024-04-25T12:35:30.576 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:30.577 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:30.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.576+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f7d0420e330 0x7f7d0420e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.576+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f7d04212480 con 0x7f7d0420e330 2024-04-25T12:35:30.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.577+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f7d0420e330 0x7f7d0420e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.577+0000 7f7d02a00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f7d0420e330 0x7f7d0420e7a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.578+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d04212480 con 0x7f7d0420e330 2024-04-25T12:35:30.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:30.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.593+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7d04211300 con 0x7f7d0420e330 2024-04-25T12:35:30.599 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.595+0000 7f7cfaa00640 1 -- 172.21.15.123:0/3627126111 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7d04211300 con 0x7f7d0420e330 2024-04-25T12:35:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f7d041f3760 msgr2=0x7f7d040c8140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f7d041f3760 0x7f7d040c8140 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f7d041541f0 msgr2=0x7f7d041e4650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f7d041541f0 0x7f7d041e4650 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f7d04152930 msgr2=0x7f7d04152d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f7d04152930 0x7f7d04152d40 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f7d040634f0 msgr2=0x7f7d04063900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f7d040634f0 0x7f7d04063900 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f7d0415a750 msgr2=0x7f7d0415ab60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d03400640 1 -- 172.21.15.123:0/3627126111 reap_dead start 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f7d0415a750 0x7f7d0415ab60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d03400640 1 -- 172.21.15.123:0/3627126111 reap_dead start 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f7d0415b0a0 msgr2=0x7f7d0402cee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f7d0415b0a0 0x7f7d0402cee0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f7d0402dc40 msgr2=0x7f7d0420c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f7d0402dc40 0x7f7d0420c0b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f7d0420e330 msgr2=0x7f7d0420e7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f7d0420e330 0x7f7d0420e7a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744e0 msgr2=0x7f7ce40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744e0 0x7f7ce40769f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009560 tx=0x7f7cf4009190 comp rx=0 tx=0).stop 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 msgr2=0x7f7d041e3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d09c00640 1 --2- 172.21.15.123:0/3627126111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d040ca060 0x7f7d041e3c00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009eb0 tx=0x7f7cfc004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.596+0000 7f7d03400640 1 -- 172.21.15.123:0/3627126111 reap_dead start 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.597+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 shutdown_connections 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.597+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 >> 172.21.15.123:0/3627126111 conn(0x7f7d0406f440 msgr2=0x7f7d040af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.597+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 shutdown_connections 2024-04-25T12:35:30.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.597+0000 7f7d09c00640 1 -- 172.21.15.123:0/3627126111 wait complete. 2024-04-25T12:35:30.623 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:30.623+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:30.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.685+0000 7f4c72600640 1 Processor -- start 2024-04-25T12:35:30.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.685+0000 7f4c72600640 1 -- start start 2024-04-25T12:35:30.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.687+0000 7f4c72600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c6c0c9b80 0x7f4c6c0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.687+0000 7f4c72600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c0c8250 0x7f4c6c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.687+0000 7f4c72600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c6c156e60 con 0x7f4c6c159820 2024-04-25T12:35:30.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.687+0000 7f4c72600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4c6c0c8ba0 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.687+0000 7f4c72600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6c15ca70 con 0x7f4c6c0c8250 2024-04-25T12:35:30.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6b400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c6c0c9b80 0x7f4c6c0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c0c8250 0x7f4c6c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6be00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4c6c159820 0x7f4c6c159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34928/0 (socket says 172.21.15.123:34928) 2024-04-25T12:35:30.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6be00640 1 -- 172.21.15.123:0/458682823 learned_addr learned my addr 172.21.15.123:0/458682823 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:30.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6b400640 1 -- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c0c8250 msgr2=0x7f4c6c0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:30.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6b400640 1 --2- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c0c8250 0x7f4c6c0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6b400640 1 -- 172.21.15.123:0/458682823 >> v1:172.21.15.153:6789/0 conn(0x7f4c6c159820 legacy=0x7f4c6c159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6b400640 1 -- 172.21.15.123:0/458682823 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c0c7130 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6aa00640 1 --2- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c0c8250 0x7f4c6c0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:30.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.688+0000 7f4c6a000640 1 -- 172.21.15.123:0/458682823 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1946418077 0 0) 0x7f4c6c156e60 con 0x7f4c6c159820 2024-04-25T12:35:30.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.689+0000 7f4c6b400640 1 --2- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c0742b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0097a0 tx=0x7f4c5c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f7c5ff59f4654c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.689+0000 7f4c6a000640 1 -- 172.21.15.123:0/458682823 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c037070 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.689+0000 7f4c6a000640 1 -- 172.21.15.123:0/458682823 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5c032440 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.690+0000 7f4c6a000640 1 -- 172.21.15.123:0/458682823 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c032810 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.690+0000 7f4c72600640 1 -- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 msgr2=0x7f4c6c0742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.690+0000 7f4c72600640 1 --2- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c0742b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0097a0 tx=0x7f4c5c02a180 comp rx=0 tx=0).stop 2024-04-25T12:35:30.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.690+0000 7f4c72600640 1 -- 172.21.15.123:0/458682823 shutdown_connections 2024-04-25T12:35:30.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.691+0000 7f4c72600640 1 --2- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c0c8250 0x7f4c6c0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.691+0000 7f4c72600640 1 --2- 172.21.15.123:0/458682823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c0742b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.691+0000 7f4c72600640 1 -- 172.21.15.123:0/458682823 >> 172.21.15.123:0/458682823 conn(0x7f4c6c06c750 msgr2=0x7f4c6c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:30.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.691+0000 7f4c72600640 1 -- 172.21.15.123:0/458682823 shutdown_connections 2024-04-25T12:35:30.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.691+0000 7f4c72600640 1 -- 172.21.15.123:0/458682823 wait complete. 2024-04-25T12:35:30.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.693+0000 7f4c72600640 1 Processor -- start 2024-04-25T12:35:30.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.694+0000 7f4c72600640 1 -- start start 2024-04-25T12:35:30.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.695+0000 7f4c72600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6c0c8250 0x7f4c6c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.695+0000 7f4c72600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.695+0000 7f4c72600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c159820 0x7f4c6c14a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.695+0000 7f4c72600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c15d310 con 0x7f4c6c0c8250 2024-04-25T12:35:30.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.695+0000 7f4c72600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c15d030 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.695+0000 7f4c72600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6c15d5f0 con 0x7f4c6c159820 2024-04-25T12:35:30.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41110/0 (socket says 172.21.15.123:41110) 2024-04-25T12:35:30.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 -- 172.21.15.123:0/3607274070 learned_addr learned my addr 172.21.15.123:0/3607274070 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:30.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6be00640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c159820 0x7f4c6c14a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6b400640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6c0c8250 0x7f4c6c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 -- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c159820 msgr2=0x7f4c6c14a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c159820 0x7f4c6c14a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 -- 172.21.15.123:0/3607274070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6c0c8250 msgr2=0x7f4c6c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6c0c8250 0x7f4c6c0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.696+0000 7f4c6aa00640 1 -- 172.21.15.123:0/3607274070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c141560 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.697+0000 7f4c6aa00640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c141020 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800e8d0 tx=0x7f4c5800ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.697+0000 7f4c63400640 1 -- 172.21.15.123:0/3607274070 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58017070 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.697+0000 7f4c63400640 1 -- 172.21.15.123:0/3607274070 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5800c940 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.697+0000 7f4c63400640 1 -- 172.21.15.123:0/3607274070 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5800cd30 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.697+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6c142bd0 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.697+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6c143190 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.698+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6c076700 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.700+0000 7f4c63400640 1 -- 172.21.15.123:0/3607274070 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4c5801a070 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.701+0000 7f4c63400640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4c4c0744b0 0x7f4c4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:30.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.701+0000 7f4c63400640 1 -- 172.21.15.123:0/3607274070 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 6619+0+0 (secure 0 0 0) 0x7f4c58095360 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.701+0000 7f4c6b400640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4c4c0744b0 0x7f4c4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:30.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.702+0000 7f4c6b400640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4c4c0744b0 0x7f4c4c0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c034000 tx=0x7f4c5c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:30.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.704+0000 7f4c63400640 1 -- 172.21.15.123:0/3607274070 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c5805f180 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.717 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:30.717+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:30.818 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:30.817+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:30.857 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:30.857 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.902129446000004, 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012628781, 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951638+0000", 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951659+0000", 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1027999999999999e-05 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951691+0000", 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2267000000000001e-05 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951710+0000", 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9063999999999999e-05 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952026+0000", 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000315719 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952367+0000", 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034072600000000002 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955799+0000", 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034325969999999999 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962852+0000", 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070528889999999997 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964076+0000", 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012234119999999999 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964178+0000", 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102166 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964264+0000", 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6175999999999996e-05 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:30.863 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.901991615, 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012746265999999999, 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951774+0000", 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-06 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951782+0000", 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2090000000000005e-06 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951796+0000", 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3559e-05 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951808+0000", 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2183e-05 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956591+0000", 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047823550000000003 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956696+0000", 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105696 2024-04-25T12:35:30.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956907+0000", 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000211165 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962907+0000", 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059995650000000001 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964283+0000", 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013758189999999999 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964381+0000", 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7977000000000001e-05 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964519+0000", 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138675 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.868 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:31 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.900822891000004, 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076541370000000001, 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952943+0000", 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952999+0000", 2024-04-25T12:35:30.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5968999999999999e-05 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953032+0000", 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2598999999999997e-05 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953058+0000", 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6585999999999999e-05 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953207+0000", 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014848300000000001 2024-04-25T12:35:30.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953263+0000", 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6005999999999999e-05 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960434+0000", 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071716219999999999 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960596+0000", 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016173899999999999 2024-04-25T12:35:30.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.900640972000005, 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010940785, 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953125+0000", 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.22e-06 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953138+0000", 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2996000000000001e-05 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953158+0000", 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0173000000000002e-05 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953176+0000", 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7927e-05 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954316+0000", 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011402599999999999 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954375+0000", 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963968+0000", 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095929269999999994 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964065+0000", 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6628999999999999e-05 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.900542148, 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080627230000000008, 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953225+0000", 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3640000000000002e-06 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953259+0000", 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3955999999999998e-05 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953292+0000", 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3549e-05 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953320+0000", 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7892e-05 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953495+0000", 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017444800000000001 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953705+0000", 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021011000000000001 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953862+0000", 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015660399999999999 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959861+0000", 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059993959999999997 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961071+0000", 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012101900000000001 2024-04-25T12:35:30.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961192+0000", 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121279 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961285+0000", 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2935000000000004e-05 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.900365410999996, 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010652488999999999, 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953401+0000", 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.513e-06 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953412+0000", 2024-04-25T12:35:30.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209000000000001e-05 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953473+0000", 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0668999999999998e-05 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953496+0000", 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3246000000000001e-05 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956356+0000", 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00286002 2024-04-25T12:35:30.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956620+0000", 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026411099999999998 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.957005+0000", 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038473499999999998 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962757+0000", 2024-04-25T12:35:30.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057517080000000003 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963927+0000", 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00117023 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964002+0000", 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5483999999999998e-05 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964052+0000", 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9564000000000002e-05 2024-04-25T12:35:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.900236961000004, 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011827087999999999, 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953529+0000", 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953594+0000", 2024-04-25T12:35:30.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4807999999999993e-05 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953617+0000", 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3220000000000001e-05 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953683+0000", 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5328999999999997e-05 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954191+0000", 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000508428 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040280099999999999 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954922+0000", 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032838300000000002 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960432+0000", 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055099989999999998 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965226+0000", 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047938700000000004 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965297+0000", 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0992000000000001e-05 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965355+0000", 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7695000000000002e-05 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:19 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.737333161000002, 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010949354999999999, 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116434+0000", 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116475+0000", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1486000000000002e-05 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116505+0000", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9709e-05 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116529+0000", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3839999999999999e-05 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116651+0000", 2024-04-25T12:35:30.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122013 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116685+0000", 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4513e-05 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127273+0000", 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010587653000000001 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127381+0000", 2024-04-25T12:35:30.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108204 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:20 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.737178034000003, 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014061374, 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116588+0000", 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0499999999999999e-06 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116602+0000", 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3793e-05 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116621+0000", 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9545999999999999e-05 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116637+0000", 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5696e-05 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117243+0000", 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060668400000000002 2024-04-25T12:35:30.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117280+0000", 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6619000000000003e-05 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130559+0000", 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013278570999999999 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130648+0000", 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9414999999999997e-05 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.900 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:27 2.9 e25/18 2:91bd965c:::501.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.736751384000002, 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013688128000000001, 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117015+0000", 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3200000000000001e-06 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117121+0000", 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106451 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117145+0000", 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.404e-05 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117210+0000", 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4314999999999997e-05 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120564+0000", 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033548050000000002 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120624+0000", 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9234999999999998e-05 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130654+0000", 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010030437999999999 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130702+0000", 2024-04-25T12:35:30.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7524000000000001e-05 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:18 2.13 e25/18 2:ca4dd7f5:::611.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.736398084999998, 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010076254, 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117369+0000", 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.88e-06 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117385+0000", 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6670000000000001e-05 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117413+0000", 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7691999999999998e-05 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117435+0000", 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2416e-05 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117549+0000", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113436 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1509e-05 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127347+0000", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097767759999999992 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127443+0000", 2024-04-25T12:35:30.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5874999999999999e-05 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.735186411999997, 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012295627, 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118582+0000", 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000001e-06 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118599+0000", 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7176e-05 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2505999999999997e-05 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118659+0000", 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7559000000000002e-05 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120040+0000", 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001380956 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120305+0000", 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026497700000000001 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120707+0000", 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040189499999999999 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128035+0000", 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073284049999999996 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130711+0000", 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026752910000000002 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130787+0000", 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6384999999999995e-05 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130874+0000", 2024-04-25T12:35:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7020000000000004e-05 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.735030412, 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012319366, 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118747+0000", 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1286e-05 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118772+0000", 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5150000000000001e-05 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118793+0000", 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0778999999999999e-05 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121911+0000", 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031179409999999999 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122331+0000", 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041955999999999997 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124441+0000", 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021107629999999999 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130205+0000", 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057636140000000002 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130888+0000", 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068289100000000003 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130961+0000", 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3026000000000002e-05 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131054+0000", 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2855999999999993e-05 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:21 2.8 2:1470b2f8:::613.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.734941683999999, 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012409474, 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118824+0000", 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:35:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118834+0000", 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118855+0000", 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999999e-05 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118875+0000", 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9809999999999998e-05 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119612+0000", 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073729299999999998 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119912+0000", 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029931000000000001 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120202+0000", 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000289907 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130342+0000", 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010140188 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131091+0000", 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074893900000000001 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131166+0000", 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5272999999999996e-05 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131233+0000", 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6625000000000007e-05 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.734832937, 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012992702, 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118933+0000", 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3069999999999999e-06 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118943+0000", 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7200000000000001e-06 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118967+0000", 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3828999999999998e-05 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118986+0000", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9057000000000001e-05 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124788+0000", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00580271 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125092+0000", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030394999999999999 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125479+0000", 2024-04-25T12:35:30.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038660000000000002 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131299+0000", 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058200559999999997 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131823+0000", 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052382700000000004 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131888+0000", 2024-04-25T12:35:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4740999999999996e-05 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131925+0000", 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6905e-05 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:5 2.7 e25/18 2:e455cf45:::60b.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.734709488999997, 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011712752, 2024-04-25T12:35:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119057+0000", 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.296e-06 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119068+0000", 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.189e-05 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119091+0000", 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2802999999999998e-05 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119110+0000", 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8473000000000001e-05 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121085+0000", 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019750689999999999 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121149+0000", 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3979000000000005e-05 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130709+0000", 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009559817 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130768+0000", 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9425000000000001e-05 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:21 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 35.698920201, 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066623919999999996, 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154846+0000", 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.06e-06 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154856+0000", 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0713000000000001e-05 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154878+0000", 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1180000000000001e-05 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154896+0000", 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8343e-05 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155332+0000", 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043663000000000003 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155389+0000", 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6818999999999999e-05 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161381+0000", 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059914440000000003 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161507+0000", 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126203 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.937 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:22 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 35.698646429999997, 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064678330000000001, 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155120+0000", 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155131+0000", 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1793e-05 2024-04-25T12:35:30.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155155+0000", 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3306e-05 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155174+0000", 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9383000000000001e-05 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157101+0000", 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019271570000000001 2024-04-25T12:35:30.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157161+0000", 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9945000000000003e-05 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161512+0000", 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043506389999999999 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161586+0000", 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4250000000000002e-05 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.941 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:26 2.11 2:893cd530:::mds2_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 35.697709457000002, 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064127289999999998, 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156057+0000", 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8330000000000001e-06 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156090+0000", 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2719000000000003e-05 2024-04-25T12:35:30.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156116+0000", 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6531999999999999e-05 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156140+0000", 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3453999999999999e-05 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156350+0000", 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021060999999999999 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156467+0000", 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116823 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156594+0000", 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012698900000000001 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161255+0000", 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046609570000000003 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162360+0000", 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001105167 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162416+0000", 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6076999999999999e-05 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162468+0000", 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1567999999999998e-05 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.946 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 35.697558807999997, 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066007829999999998, 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:30.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156208+0000", 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5570000000000001e-06 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156216+0000", 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0900000000000005e-06 2024-04-25T12:35:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156237+0000", 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1452999999999999e-05 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156255+0000", 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8091999999999999e-05 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156841+0000", 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00058573200000000005 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156983+0000", 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000141676 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162807+0000", 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058241830000000001 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:30.950 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:30.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:30.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:30.918+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f4c6c0c9590 con 0x7f4c6c0c9b80 2024-04-25T12:35:30.953 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:30.952+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:31.004 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:35:31.004 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:35:31.004 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-25T12:35:31.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-25T12:35:31.052 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:31.052+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:31.154 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:31.154+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:31.207 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:31.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:31.207 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:31.207 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:31.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:31.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.243+0000 7f29e9000640 1 Processor -- start 2024-04-25T12:35:31.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.243+0000 7f29e9000640 1 -- start start 2024-04-25T12:35:31.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.245+0000 7f29e9000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f29e4159f30 0x7f29e40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.245+0000 7f29e9000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e4156e60 con 0x7f29e40c8250 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.245+0000 7f29e9000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e40c7130 con 0x7f29e40c9b80 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.245+0000 7f29e9000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f29e40749c0 con 0x7f29e4159f30 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.245+0000 7f29dbe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f29e40c8250 0x7f29e4159820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:52792/0 (socket says 172.21.15.123:52792) 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.245+0000 7f29dbe00640 1 -- 172.21.15.123:0/1020091487 learned_addr learned my addr 172.21.15.123:0/1020091487 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.245+0000 7f29e2800640 1 --2- 172.21.15.123:0/1020091487 >> v2:172.21.15.153:3300/0 conn(0x7f29e4159f30 0x7f29e40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.246+0000 7f29e1400640 1 -- 172.21.15.123:0/1020091487 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 920825308 0 0) 0x7f29e40c7130 con 0x7f29e40c9b80 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.246+0000 7f29e1400640 1 -- 172.21.15.123:0/1020091487 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29c8003520 con 0x7f29e40c9b80 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.246+0000 7f29e1400640 1 -- 172.21.15.123:0/1020091487 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2887006857 0 0) 0x7f29e4156e60 con 0x7f29e40c8250 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.246+0000 7f29e1400640 1 -- 172.21.15.123:0/1020091487 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29e40c7130 con 0x7f29e40c8250 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.246+0000 7f29e2800640 1 -- 172.21.15.123:0/1020091487 >> v1:172.21.15.153:6789/0 conn(0x7f29e40c9b80 legacy=0x7f29e40742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.246+0000 7f29e2800640 1 -- 172.21.15.123:0/1020091487 >> v1:172.21.15.123:6789/0 conn(0x7f29e40c8250 legacy=0x7f29e4159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.246+0000 7f29e2800640 1 -- 172.21.15.123:0/1020091487 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e415ca70 con 0x7f29e4159f30 2024-04-25T12:35:31.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.247+0000 7f29e2800640 1 --2- 172.21.15.123:0/1020091487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e4159f30 0x7f29e40c8660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009d90 tx=0x7f29dc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b98cf782c590d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.247+0000 7f29e1400640 1 -- 172.21.15.123:0/1020091487 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc007740 con 0x7f29e4159f30 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.247+0000 7f29e1400640 1 -- 172.21.15.123:0/1020091487 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc0078e0 con 0x7f29e4159f30 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.247+0000 7f29e1400640 1 -- 172.21.15.123:0/1020091487 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc007cb0 con 0x7f29e4159f30 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.248+0000 7f29e9000640 1 -- 172.21.15.123:0/1020091487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e4159f30 msgr2=0x7f29e40c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.248+0000 7f29e9000640 1 --2- 172.21.15.123:0/1020091487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e4159f30 0x7f29e40c8660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009d90 tx=0x7f29dc02a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.248+0000 7f29e9000640 1 -- 172.21.15.123:0/1020091487 shutdown_connections 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.248+0000 7f29e9000640 1 --2- 172.21.15.123:0/1020091487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e4159f30 0x7f29e40c8660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.248+0000 7f29e9000640 1 -- 172.21.15.123:0/1020091487 >> 172.21.15.123:0/1020091487 conn(0x7f29e406c750 msgr2=0x7f29e406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.248+0000 7f29e9000640 1 -- 172.21.15.123:0/1020091487 shutdown_connections 2024-04-25T12:35:31.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.248+0000 7f29e9000640 1 -- 172.21.15.123:0/1020091487 wait complete. 2024-04-25T12:35:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.251+0000 7f29e9000640 1 Processor -- start 2024-04-25T12:35:31.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.251+0000 7f29e9000640 1 -- start start 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.252+0000 7f29e9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e40c8250 0x7f29e41f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.252+0000 7f29e9000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e40c9b80 0x7f29e40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.252+0000 7f29e9000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29e4159f30 0x7f29e414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.252+0000 7f29e9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29e415d030 con 0x7f29e40c8250 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.252+0000 7f29e9000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40c8ba0 con 0x7f29e40c9b80 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.252+0000 7f29e9000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f29e415d310 con 0x7f29e4159f30 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.252+0000 7f29e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e40c8250 0x7f29e41f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e40c8250 0x7f29e41f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54828/0 (socket says 172.21.15.123:54828) 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 -- 172.21.15.123:0/3036130204 learned_addr learned my addr 172.21.15.123:0/3036130204 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e2800640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29e4159f30 0x7f29e414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:31.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29dbe00640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e40c9b80 0x7f29e40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:31.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 -- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29e4159f30 msgr2=0x7f29e414d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29e4159f30 0x7f29e414d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 -- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e40c9b80 msgr2=0x7f29e40b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e40c9b80 0x7f29e40b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 -- 172.21.15.123:0/3036130204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e41f3b60 con 0x7f29e40c8250 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.253+0000 7f29e1e00640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e40c8250 0x7f29e41f3620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c410 tx=0x7f29d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.254+0000 7f29da000640 1 -- 172.21.15.123:0/3036130204 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4017070 con 0x7f29e40c8250 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.254+0000 7f29da000640 1 -- 172.21.15.123:0/3036130204 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d40052d0 con 0x7f29e40c8250 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.254+0000 7f29da000640 1 -- 172.21.15.123:0/3036130204 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4003810 con 0x7f29e40c8250 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.254+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e4153a20 con 0x7f29e40c8250 2024-04-25T12:35:31.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.254+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e415a810 con 0x7f29e40c8250 2024-04-25T12:35:31.256 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:31.254+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:31.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.255+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e40c6d70 con 0x7f29e40c8250 2024-04-25T12:35:31.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.257+0000 7f29da000640 1 -- 172.21.15.123:0/3036130204 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f29d401a070 con 0x7f29e40c8250 2024-04-25T12:35:31.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.258+0000 7f29da000640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29c00744e0 0x7f29c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:31.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.258+0000 7f29da000640 1 -- 172.21.15.123:0/3036130204 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 6619+0+0 (secure 0 0 0) 0x7f29d4094910 con 0x7f29e40c8250 2024-04-25T12:35:31.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.258+0000 7f29dbe00640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29c00744e0 0x7f29c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:31.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.259+0000 7f29dbe00640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29c00744e0 0x7f29c00769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29e40709b0 tx=0x7f29cc007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:31.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.262+0000 7f29da000640 1 -- 172.21.15.123:0/3036130204 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29e40c6d70 con 0x7f29e40c8250 2024-04-25T12:35:31.356 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:31.356+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:31.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.418+0000 7f4c63400640 1 -- 172.21.15.123:0/3607274070 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v35) ==== 91+0+0 (secure 0 0 0) 0x7f4c58063480 con 0x7f4c6c0c9b80 2024-04-25T12:35:31.418 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:35:31.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4c4c0744b0 msgr2=0x7f4c4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4c4c0744b0 0x7f4c4c0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c034000 tx=0x7f4c5c02e000 comp rx=0 tx=0).stop 2024-04-25T12:35:31.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 msgr2=0x7f4c6c141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c141020 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800e8d0 tx=0x7f4c5800ed90 comp rx=0 tx=0).stop 2024-04-25T12:35:31.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 shutdown_connections 2024-04-25T12:35:31.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4c4c0744b0 0x7f4c4c0769c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4c6c159820 0x7f4c6c14a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c6c0c9b80 0x7f4c6c141020 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 --2- 172.21.15.123:0/3607274070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6c0c8250 0x7f4c6c0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 >> 172.21.15.123:0/3607274070 conn(0x7f4c6c06c750 msgr2=0x7f4c6c0c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:31.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 shutdown_connections 2024-04-25T12:35:31.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.421+0000 7f4c72600640 1 -- 172.21.15.123:0/3607274070 wait complete. 2024-04-25T12:35:31.456 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:31.457+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:31.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.460+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f29e4076f10 con 0x7f29e40c8250 2024-04-25T12:35:31.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.463+0000 7f29da000640 1 -- 172.21.15.123:0/3036130204 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v35) ==== 81+0+37963 (secure 0 0 0) 0x7f29e4076f10 con 0x7f29e40c8250 2024-04-25T12:35:31.463 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:35:31.463 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "epoch": 35, 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "created": "2024-04-25T12:33:54.601604+0000", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "modified": "2024-04-25T12:35:31.408253+0000", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "last_up_change": "2024-04-25T12:34:07.013466+0000", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "last_in_change": "2024-04-25T12:33:58.290536+0000", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_num": 5804032, 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_set": [ 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "nodeep-scrub", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "pglog_hardlimit", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "purged_snapdirs", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_deletes", 2024-04-25T12:35:31.464 INFO:teuthology.orchestra.run.smithi123.stdout: "sortbitwise" 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_version": 6, 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_snapshot": "", 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_max": 4, 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd": 8, 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "require_min_compat_client": "luminous", 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "min_compat_client": "luminous", 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "require_osd_release": "squid", 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_crimson": false, 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: "pools": [ 2024-04-25T12:35:31.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 1, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": ".mgr", 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:06.827674+0000", 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:31.466 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 1, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 1, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 1, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 1, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 1, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "17", 2024-04-25T12:35:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:31.468 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:31.469 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_max": 32, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 1 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": {} 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:35:31.470 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 8, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 8, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 0.25, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 2, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_metadata", 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:11.030660+0000", 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:35:31.471 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:35:31.472 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "20", 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:31.473 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:31.474 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:31.475 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_bias": 4, 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 16, 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_priority": 5 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": "cephfs" 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:35:31.476 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 2.25, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 2.25, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2.25, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2.25, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 3, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data", 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:12.395253+0000", 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:35:31.477 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 64, 2024-04-25T12:35:31.478 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 64, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 64, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 64, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 64, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "21", 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:31.479 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:31.480 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:31.481 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.05000000074505806 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 1.5, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 1.5, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 1.5, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 1.5, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.483 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 4, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data_ec", 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:15.975923+0000", 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 5, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 3, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 4, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 3, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 1, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:35:31.484 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "25", 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:35:31.485 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:35:31.486 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:35:31.487 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 8192, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.488 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "osds": [ 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "2d9ee216-abfa-45db-a0ac-ef834981be6f", 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 26, 2024-04-25T12:35:31.489 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6816", 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6817", 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6818", 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6819", 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.491 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6822", 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6823", 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6820", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6821", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6817/1405705428", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6819/1405705428", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6823/1405705428", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6821/1405705428", 2024-04-25T12:35:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "a60a3a6f-376a-4fb2-baf1-e13fc615f6e5", 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.494 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 29, 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6800", 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6801", 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.495 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6802", 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6803", 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6806", 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6807", 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.497 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6804", 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6805", 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6801/1234958824", 2024-04-25T12:35:31.498 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6803/1234958824", 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6807/1234958824", 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6805/1234958824", 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "cfb7cbbd-82cd-46fe-b89b-15775df1fe93", 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.499 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6808", 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6809", 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6810", 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6811", 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6814", 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6815", 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6812", 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6813", 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6809/2222173849", 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6811/2222173849", 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6815/2222173849", 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6813/2222173849", 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9e8ccd97-ab68-4f03-bbb3-5820eccf4568", 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6824", 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6825", 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6826", 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6827", 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.507 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6830", 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6831", 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6828", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6829", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6825/1491071423", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6827/1491071423", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6831/1491071423", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6829/1491071423", 2024-04-25T12:35:31.509 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "7d5bfd6e-173a-413d-abdb-df6d82c941d3", 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.510 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6802", 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6803", 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.511 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6804", 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6805", 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.512 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6808", 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6809", 2024-04-25T12:35:31.513 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6806", 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.514 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6807", 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6803/3135982807", 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6805/3135982807", 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6809/3135982807", 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6807/3135982807", 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9141fa91-c547-4711-910a-9ef123017fb5", 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.516 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6826", 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6827", 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.517 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6828", 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6829", 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6832", 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6833", 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.519 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6830", 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6831", 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2327013656 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6827/2327013656", 2024-04-25T12:35:31.520 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6829/2327013656", 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6833/2327013656", 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6831/2327013656", 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "66fbc935-086e-4b03-aa03-96e0bc8b1d27", 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.521 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 23, 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6810", 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.522 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6811", 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6812", 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.523 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6813", 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6816", 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6817", 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.524 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6814", 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6815", 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.525 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6811/3626282289", 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6813/3626282289", 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6817/3626282289", 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6815/3626282289", 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "de59a619-b84a-4623-ac89-447178815d65", 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:35:31.526 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 26, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6818", 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6819", 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6820", 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6821", 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6824", 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6825", 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6822", 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6823", 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6819/3086604284", 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6821/3086604284", 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6825/3086604284", 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6823/3086604284", 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:31.531 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_xinfo": [ 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.805055+0000", 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.532 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.746266+0000", 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.706883+0000", 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.826277+0000", 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.534 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.402228+0000", 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.535 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.536018+0000", 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.375016+0000", 2024-04-25T12:35:31.536 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.454835+0000", 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:31.537 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap": [], 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_items": [ 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "1.0", 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 5, 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 4 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "3.15", 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:35:31.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 0, 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 7 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "4.d", 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 3, 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 0 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_primaries": [], 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_temp": [], 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_temp": [], 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklist": {}, 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "range_blocklist": {}, 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profiles": { 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_data_ec": { 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-device-class": "", 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-num-failure-domains": "0", 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-osds-per-failure-domain": "0", 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-root": "default", 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "jerasure-per-chunk-alignment": "false", 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:35:31.540 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "2", 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van", 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "w": "8" 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "default": { 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "1", 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van" 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.541 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps_queue": [], 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "new_removed_snaps": [], 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "new_purged_snaps": [], 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_node_flags": {}, 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "device_class_flags": {}, 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode": { 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_enabled": false, 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_bucket_count": 0, 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "degraded_stretch_mode": 0, 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "recovering_stretch_mode": 0, 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_bucket": 0 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.542 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29c00744e0 msgr2=0x7f29c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29c00744e0 0x7f29c00769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29e40709b0 tx=0x7f29cc007d50 comp rx=0 tx=0).stop 2024-04-25T12:35:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e40c8250 msgr2=0x7f29e41f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e40c8250 0x7f29e41f3620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c410 tx=0x7f29d400c8d0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 shutdown_connections 2024-04-25T12:35:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29c00744e0 0x7f29c00769f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29e4159f30 0x7f29e414d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29e40c9b80 0x7f29e40b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 --2- 172.21.15.123:0/3036130204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e40c8250 0x7f29e41f3620 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 >> 172.21.15.123:0/3036130204 conn(0x7f29e406c750 msgr2=0x7f29e40ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 shutdown_connections 2024-04-25T12:35:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:31.465+0000 7f29e9000640 1 -- 172.21.15.123:0/3036130204 wait complete. 2024-04-25T12:35:31.546 INFO:tasks.thrashosds.thrasher:j is {'epoch': 35, 'fsid': '2239057a-dc25-4ee3-8341-8c80e403f296', 'created': '2024-04-25T12:33:54.601604+0000', 'modified': '2024-04-25T12:35:31.408253+0000', 'last_up_change': '2024-04-25T12:34:07.013466+0000', 'last_in_change': '2024-04-25T12:33:58.290536+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': '2d9ee216-abfa-45db-a0ac-ef834981be6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 1405705428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 1405705428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 1405705428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 1405705428}]}, 'public_addr': '172.21.15.123:6817/1405705428', 'cluster_addr': '172.21.15.123:6819/1405705428', 'heartbeat_back_addr': '172.21.15.123:6823/1405705428', 'heartbeat_front_addr': '172.21.15.123:6821/1405705428', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a60a3a6f-376a-4fb2-baf1-e13fc615f6e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 1234958824}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 1234958824}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 1234958824}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 1234958824}]}, 'public_addr': '172.21.15.123:6801/1234958824', 'cluster_addr': '172.21.15.123:6803/1234958824', 'heartbeat_back_addr': '172.21.15.123:6807/1234958824', 'heartbeat_front_addr': '172.21.15.123:6805/1234958824', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'cfb7cbbd-82cd-46fe-b89b-15775df1fe93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 2222173849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 2222173849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 2222173849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 2222173849}]}, 'public_addr': '172.21.15.123:6809/2222173849', 'cluster_addr': '172.21.15.123:6811/2222173849', 'heartbeat_back_addr': '172.21.15.123:6815/2222173849', 'heartbeat_front_addr': '172.21.15.123:6813/2222173849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9e8ccd97-ab68-4f03-bbb3-5820eccf4568', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 1491071423}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 1491071423}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 1491071423}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 1491071423}]}, 'public_addr': '172.21.15.123:6825/1491071423', 'cluster_addr': '172.21.15.123:6827/1491071423', 'heartbeat_back_addr': '172.21.15.123:6831/1491071423', 'heartbeat_front_addr': '172.21.15.123:6829/1491071423', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d5bfd6e-173a-413d-abdb-df6d82c941d3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6802', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6803', 'nonce': 3135982807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6804', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6805', 'nonce': 3135982807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6808', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6809', 'nonce': 3135982807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6806', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6807', 'nonce': 3135982807}]}, 'public_addr': '172.21.15.153:6803/3135982807', 'cluster_addr': '172.21.15.153:6805/3135982807', 'heartbeat_back_addr': '172.21.15.153:6809/3135982807', 'heartbeat_front_addr': '172.21.15.153:6807/3135982807', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9141fa91-c547-4711-910a-9ef123017fb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6826', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6827', 'nonce': 2327013656}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6828', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6829', 'nonce': 2327013656}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 2327013656}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6830', 'nonce': 2327013656}, {'type': 'v1', 'addr': '172.21.15.153:6831', 'nonce': 2327013656}]}, 'public_addr': '172.21.15.153:6827/2327013656', 'cluster_addr': '172.21.15.153:6829/2327013656', 'heartbeat_back_addr': '172.21.15.153:6833/2327013656', 'heartbeat_front_addr': '172.21.15.153:6831/2327013656', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '66fbc935-086e-4b03-aa03-96e0bc8b1d27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6810', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6811', 'nonce': 3626282289}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6812', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6813', 'nonce': 3626282289}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6816', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6817', 'nonce': 3626282289}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6814', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6815', 'nonce': 3626282289}]}, 'public_addr': '172.21.15.153:6811/3626282289', 'cluster_addr': '172.21.15.153:6813/3626282289', 'heartbeat_back_addr': '172.21.15.153:6817/3626282289', 'heartbeat_front_addr': '172.21.15.153:6815/3626282289', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'de59a619-b84a-4623-ac89-447178815d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6818', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6819', 'nonce': 3086604284}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6820', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6821', 'nonce': 3086604284}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6824', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6825', 'nonce': 3086604284}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6822', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6823', 'nonce': 3086604284}]}, 'public_addr': '172.21.15.153:6819/3086604284', 'cluster_addr': '172.21.15.153:6821/3086604284', 'heartbeat_back_addr': '172.21.15.153:6825/3086604284', 'heartbeat_front_addr': '172.21.15.153:6823/3086604284', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.805055+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.746266+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.706883+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.826277+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.402228+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.536018+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.375016+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.454835+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '1.0', 'mappings': [{'from': 5, 'to': 4}]}, {'pgid': '3.15', 'mappings': [{'from': 0, 'to': 7}]}, {'pgid': '4.d', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-25T12:35:31.546 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-04-25T12:35:31.547 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:31.547 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:31.547 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:31.547 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:31.547 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:31.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:31.587 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:31.586+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:31.658 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:31.659+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:31.760 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:31.759+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:31.852 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:31.852 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:31.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:31.852 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:31.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.852 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.852 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.822884549999998, 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53759151699999996, 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024334+0000", 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7179999999999999e-06 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024345+0000", 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0608000000000001e-05 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024373+0000", 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7966e-05 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024392+0000", 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024490+0000", 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7823000000000002e-05 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for peered", 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024520+0000", 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0056000000000002e-05 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.559770+0000", 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53525036000000004 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560001+0000", 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023139100000000001 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560330+0000", 2024-04-25T12:35:31.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032824500000000001 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561700+0000", 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013704730000000001 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561742+0000", 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1826000000000002e-05 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561855+0000", 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011287599999999999 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561924+0000", 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9484000000000001e-05 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.859 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.259961331, 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095393000000000006, 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587257+0000", 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9100000000000003e-07 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587490+0000", 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000233186 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587502+0000", 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2499e-05 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587512+0000", 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7699999999999996e-06 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.591891+0000", 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043783770000000001 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592036+0000", 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014543999999999999 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592306+0000", 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026964399999999998 2024-04-25T12:35:31.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595185+0000", 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028791730000000001 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596722+0000", 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001537131 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596762+0000", 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0491000000000002e-05 2024-04-25T12:35:31.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596795+0000", 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2798e-05 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.259688412000003, 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012300247, 2024-04-25T12:35:31.865 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 24 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587530+0000", 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3799999999999996e-07 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587688+0000", 2024-04-25T12:35:31.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015835999999999999 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587699+0000", 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0727000000000001e-05 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587710+0000", 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0987e-05 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593399+0000", 2024-04-25T12:35:31.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005688935 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593685+0000", 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028662199999999999 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593893+0000", 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020758500000000001 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595599+0000", 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017058209999999999 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599668+0000", 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040696439999999999 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599769+0000", 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100752 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599829+0000", 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0075999999999999e-05 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.259466353999997, 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012351434, 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.871 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 25 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587752+0000", 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587758+0000", 2024-04-25T12:35:31.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2199999999999997e-06 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587770+0000", 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2129e-05 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587778+0000", 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594340+0000", 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065618270000000001 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594415+0000", 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4535000000000004e-05 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594510+0000", 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5100999999999996e-05 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595607+0000", 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010968239999999999 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599926+0000", 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043194469999999997 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600027+0000", 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100961 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600102+0000", 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5339000000000005e-05 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.054517164000004, 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091591999999999993, 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792701+0000", 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.17e-06 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792911+0000", 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000210163 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792932+0000", 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0472e-05 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792952+0000", 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0553000000000001e-05 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799157+0000", 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062046030000000004 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799246+0000", 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8505999999999996e-05 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799389+0000", 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014319100000000001 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801232+0000", 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018427599999999999 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801753+0000", 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052153699999999998 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801814+0000", 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1014000000000002e-05 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801859+0000", 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5231000000000001e-05 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.882 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.054078816000001, 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092910930000000003, 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 52 2024-04-25T12:35:31.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793140+0000", 2024-04-25T12:35:31.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-06 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793151+0000", 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0966e-05 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793168+0000", 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793184+0000", 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6200000000000001e-05 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801271+0000", 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080863189999999998 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801360+0000", 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9248999999999999e-05 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801515+0000", 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000154883 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802197+0000", 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068243299999999998 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802339+0000", 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014159 2024-04-25T12:35:31.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802393+0000", 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4215000000000002e-05 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802430+0000", 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6434999999999999e-05 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.596075545999994, 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012798129, 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.889 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 70 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251143+0000", 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2729999999999999e-06 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251153+0000", 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8439999999999993e-06 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251172+0000", 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8906000000000002e-05 2024-04-25T12:35:31.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251183+0000", 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1653e-05 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257332+0000", 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061486459999999998 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257490+0000", 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000157947 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257724+0000", 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023408200000000001 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259695+0000", 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001971169 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263731+0000", 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035683 2024-04-25T12:35:31.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263842+0000", 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110985 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263940+0000", 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7941000000000003e-05 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.596018637, 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013164844, 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 71 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251200+0000", 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2999999999999999e-07 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251206+0000", 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9399999999999999e-06 2024-04-25T12:35:31.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251215+0000", 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251223+0000", 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2260000000000005e-06 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259955+0000", 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087315629999999995 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260077+0000", 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122263 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260310+0000", 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232702 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263699+0000", 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033888080000000001 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264218+0000", 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051973199999999996 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264304+0000", 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5433999999999996e-05 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264363+0000", 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9743e-05 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.899 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.595977027999993, 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017184050999999999, 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 72 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251241+0000", 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.23e-07 2024-04-25T12:35:31.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251477+0000", 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023584499999999999 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251504+0000", 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7633000000000001e-05 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251516+0000", 2024-04-25T12:35:31.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.164e-05 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260993+0000", 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094771119999999993 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261122+0000", 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012831400000000001 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261319+0000", 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019727199999999999 2024-04-25T12:35:31.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264078+0000", 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027589120000000001 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268258+0000", 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041806569999999999 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268357+0000", 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8721000000000006e-05 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268424+0000", 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7222000000000001e-05 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.595682084999993, 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017105597, 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.905 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 73 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:31.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251536+0000", 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4300000000000001e-07 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251543+0000", 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3840000000000002e-06 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251552+0000", 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6860000000000001e-06 2024-04-25T12:35:31.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251560+0000", 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0499999999999992e-06 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262615+0000", 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011054494 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262757+0000", 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142521 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262999+0000", 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024147400000000001 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264190+0000", 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011911840000000001 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268472+0000", 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042823940000000001 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268571+0000", 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8218000000000001e-05 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268641+0000", 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0248999999999998e-05 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.595645098999995, 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017309201999999999, 2024-04-25T12:35:31.911 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 74 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251573+0000", 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9999999999999997e-07 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251579+0000", 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6330000000000001e-06 2024-04-25T12:35:31.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251592+0000", 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3869999999999999e-05 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251621+0000", 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8966e-05 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264505+0000", 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012883307 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264627+0000", 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012204 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264832+0000", 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020571899999999999 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268227+0000", 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003394308 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268663+0000", 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043630699999999998 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268761+0000", 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7774000000000003e-05 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268881+0000", 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120578 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.595574990000003, 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01751983, 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 75 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251643+0000", 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5300000000000003e-07 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251849+0000", 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000206345 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251863+0000", 2024-04-25T12:35:31.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3077e-05 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251873+0000", 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.08e-05 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265525+0000", 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013651876 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265667+0000", 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014156699999999999 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265939+0000", 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027219700000000001 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268931+0000", 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029921990000000001 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269008+0000", 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6619000000000006e-05 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269082+0000", 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4401999999999997e-05 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269162+0000", 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9895000000000007e-05 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.922 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.595327322000003, 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018357248, 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 76 2024-04-25T12:35:31.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251891+0000", 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7300000000000005e-07 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251905+0000", 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4279e-05 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251915+0000", 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251935+0000", 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.961e-05 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267152+0000", 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015217341000000001 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267276+0000", 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012396 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267509+0000", 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023329500000000001 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268988+0000", 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014781709999999999 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270041+0000", 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001053806 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270161+0000", 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119494 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270247+0000", 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6382000000000001e-05 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.576102813000006, 2024-04-25T12:35:31.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099439890000000003, 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 80 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271115+0000", 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271221+0000", 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105713 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271236+0000", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5415999999999999e-05 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271240+0000", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3449999999999999e-06 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275441+0000", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042009090000000001 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275579+0000", 2024-04-25T12:35:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013740700000000001 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275795+0000", 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021634900000000001 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.277926+0000", 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021311220000000001 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.280958+0000", 2024-04-25T12:35:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030320490000000002 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281016+0000", 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8232000000000001e-05 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281058+0000", 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1993999999999998e-05 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.221086447999994, 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094693499999999996, 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 92 2024-04-25T12:35:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626132+0000", 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4399999999999999e-07 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626137+0000", 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4929999999999998e-06 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626147+0000", 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5300000000000002e-06 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626155+0000", 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4130000000000006e-06 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629532+0000", 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033774830000000001 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629626+0000", 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3394999999999996e-05 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629792+0000", 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016611100000000001 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633343+0000", 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00355055 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635469+0000", 2024-04-25T12:35:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021262009999999999 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635548+0000", 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9167999999999998e-05 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635600+0000", 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2262000000000002e-05 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.221050112, 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096152330000000008, 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 93 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626168+0000", 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9699999999999995e-07 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626290+0000", 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121941 2024-04-25T12:35:31.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626304+0000", 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3729e-05 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626321+0000", 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7062999999999999e-05 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630147+0000", 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003826 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630229+0000", 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1953000000000006e-05 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630372+0000", 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014345199999999999 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633348+0000", 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029755789999999999 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635672+0000", 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023242829999999999 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635733+0000", 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0899000000000001e-05 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635782+0000", 2024-04-25T12:35:31.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9636999999999999e-05 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.220876192000006, 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097157200000000006, 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:31.945 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 94 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:31.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626342+0000", 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4300000000000002e-07 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626348+0000", 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0059999999999998e-06 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626358+0000", 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:31.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626367+0000", 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8939999999999999e-06 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631146+0000", 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047795429999999998 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631307+0000", 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016041699999999999 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631539+0000", 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232258 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635089+0000", 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035503980000000002 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635880+0000", 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079070200000000001 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635984+0000", 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104221 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.636057+0000", 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2597999999999998e-05 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.950 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.729166640000003, 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010749583, 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118054+0000", 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118075+0000", 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118119+0000", 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4372000000000001e-05 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118163+0000", 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3766000000000001e-05 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119484+0000", 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001320494 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119803+0000", 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031975599999999998 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120121+0000", 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031758999999999997 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128491+0000", 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083702940000000003 2024-04-25T12:35:31.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128605+0000", 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113158 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128733+0000", 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012861099999999999 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128800+0000", 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7086000000000001e-05 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.728967654000002, 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011301215999999999, 2024-04-25T12:35:31.956 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118251+0000", 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6160000000000001e-06 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118263+0000", 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2243e-05 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118305+0000", 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1838999999999997e-05 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118327+0000", 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2082999999999998e-05 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120981+0000", 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002653201 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121221+0000", 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000240058 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121655+0000", 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043405400000000002 2024-04-25T12:35:31.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128501+0000", 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068465519999999997 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128984+0000", 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048240400000000003 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129487+0000", 2024-04-25T12:35:31.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050289499999999995 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129551+0000", 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4270999999999995e-05 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.728860222999998, 2024-04-25T12:35:31.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012520414000000001, 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118358+0000", 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3230000000000001e-06 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118368+0000", 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3899999999999999e-06 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2481999999999999e-05 2024-04-25T12:35:31.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118410+0000", 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9304e-05 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126108+0000", 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076979409999999998 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126379+0000", 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027136700000000001 2024-04-25T12:35:31.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126653+0000", 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274238 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129707+0000", 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030541800000000001 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130686+0000", 2024-04-25T12:35:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097908900000000004 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130791+0000", 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104824 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130877+0000", 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6275999999999999e-05 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.967 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.968 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:31.968 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:31.968 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:31.969 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:31.970 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:31.969+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:32.070 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:32.070+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:32.171 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:32.171+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:32.272 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:32.272+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:32.288 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:32.289 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:32.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:32.373 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:32.373+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:32.474 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:32.474+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:32.574 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:32.575+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:32.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:32.632 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:32.632 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:32.632 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:32.632 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:32.632 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:32.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:32.693 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:32.693+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:32.776 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:32.775+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:32.877 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:32.877+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:32.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.889+0000 7f0ebb400640 1 Processor -- start 2024-04-25T12:35:32.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.890+0000 7f0ebb400640 1 -- start start 2024-04-25T12:35:32.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0ebb400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0eb40c9f00 0x7f0eb40c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0ebb400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eb4074b20 con 0x7f0eb40730e0 2024-04-25T12:35:32.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0ebb400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eb40ca780 con 0x7f0eb4154650 2024-04-25T12:35:32.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0ebb400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb40caa60 con 0x7f0eb40c9f00 2024-04-25T12:35:32.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0eb9400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0eb40730e0 0x7f0eb4164d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48054/0 (socket says 172.21.15.123:48054) 2024-04-25T12:35:32.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0eb9400640 1 -- 172.21.15.123:0/2433195187 learned_addr learned my addr 172.21.15.123:0/2433195187 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:32.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0eb8a00640 1 --1- 172.21.15.123:0/2433195187 >> v1:172.21.15.153:6789/0 conn(0x7f0eb4154650 0x7f0eb415ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60344/0 (socket says 172.21.15.123:60344) 2024-04-25T12:35:32.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.892+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/2433195187 >> v2:172.21.15.123:3300/0 conn(0x7f0eb40c9f00 0x7f0eb40c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.893+0000 7f0eb3400640 1 -- 172.21.15.123:0/2433195187 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767732055 0 0) 0x7f0eb40ca780 con 0x7f0eb4154650 2024-04-25T12:35:32.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.893+0000 7f0eb3400640 1 -- 172.21.15.123:0/2433195187 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e9c003520 con 0x7f0eb4154650 2024-04-25T12:35:32.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.893+0000 7f0eb3400640 1 -- 172.21.15.123:0/2433195187 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427688142 0 0) 0x7f0eb4074b20 con 0x7f0eb40730e0 2024-04-25T12:35:32.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.893+0000 7f0eb3400640 1 -- 172.21.15.123:0/2433195187 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eb40ca780 con 0x7f0eb40730e0 2024-04-25T12:35:32.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.893+0000 7f0eb3e00640 1 -- 172.21.15.123:0/2433195187 >> v1:172.21.15.153:6789/0 conn(0x7f0eb4154650 legacy=0x7f0eb415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:32.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.893+0000 7f0eb3e00640 1 -- 172.21.15.123:0/2433195187 >> v1:172.21.15.123:6789/0 conn(0x7f0eb40730e0 legacy=0x7f0eb4164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:32.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.893+0000 7f0eb3e00640 1 -- 172.21.15.123:0/2433195187 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb415da50 con 0x7f0eb40c9f00 2024-04-25T12:35:32.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.894+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/2433195187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb40c9f00 0x7f0eb40c8170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00097a0 tx=0x7f0ea002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e6cdc5880755db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:32.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.894+0000 7f0eb3400640 1 -- 172.21.15.123:0/2433195187 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0037070 con 0x7f0eb40c9f00 2024-04-25T12:35:32.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.894+0000 7f0eb3400640 1 -- 172.21.15.123:0/2433195187 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea0032440 con 0x7f0eb40c9f00 2024-04-25T12:35:32.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.894+0000 7f0eb3400640 1 -- 172.21.15.123:0/2433195187 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00327f0 con 0x7f0eb40c9f00 2024-04-25T12:35:32.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.895+0000 7f0ebb400640 1 -- 172.21.15.123:0/2433195187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb40c9f00 msgr2=0x7f0eb40c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:32.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.895+0000 7f0ebb400640 1 --2- 172.21.15.123:0/2433195187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb40c9f00 0x7f0eb40c8170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00097a0 tx=0x7f0ea002a230 comp rx=0 tx=0).stop 2024-04-25T12:35:32.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.895+0000 7f0ebb400640 1 -- 172.21.15.123:0/2433195187 shutdown_connections 2024-04-25T12:35:32.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.895+0000 7f0ebb400640 1 --2- 172.21.15.123:0/2433195187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb40c9f00 0x7f0eb40c8170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:32.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.895+0000 7f0ebb400640 1 -- 172.21.15.123:0/2433195187 >> 172.21.15.123:0/2433195187 conn(0x7f0eb406f440 msgr2=0x7f0eb406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:32.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.895+0000 7f0ebb400640 1 -- 172.21.15.123:0/2433195187 shutdown_connections 2024-04-25T12:35:32.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.896+0000 7f0ebb400640 1 -- 172.21.15.123:0/2433195187 wait complete. 2024-04-25T12:35:32.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.898+0000 7f0ebb400640 1 Processor -- start 2024-04-25T12:35:32.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.898+0000 7f0ebb400640 1 -- start start 2024-04-25T12:35:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.899+0000 7f0ebb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb40730e0 0x7f0eb40afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0ebb400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb40c9f00 0x7f0eb414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0ebb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4154650 0x7f0eb41f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0ebb400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb415e2f0 con 0x7f0eb4154650 2024-04-25T12:35:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0ebb400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb415e010 con 0x7f0eb40730e0 2024-04-25T12:35:32.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0ebb400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb4154a60 con 0x7f0eb40c9f00 2024-04-25T12:35:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb40730e0 0x7f0eb40afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb40c9f00 0x7f0eb414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb40730e0 0x7f0eb40afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33336/0 (socket says 172.21.15.123:33336) 2024-04-25T12:35:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb8a00640 1 -- 172.21.15.123:0/1938016583 learned_addr learned my addr 172.21.15.123:0/1938016583 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb9400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4154650 0x7f0eb41f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb3e00640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb40730e0 msgr2=0x7f0eb40afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb40730e0 0x7f0eb40afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb3e00640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4154650 msgr2=0x7f0eb41f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4154650 0x7f0eb41f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.900+0000 7f0eb3e00640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb415b500 con 0x7f0eb40c9f00 2024-04-25T12:35:32.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.901+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0eb40730e0 0x7f0eb40afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:32.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.901+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb40c9f00 0x7f0eb414f670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0031000 tx=0x7f0ea00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:32.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.901+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0041030 con 0x7f0eb40c9f00 2024-04-25T12:35:32.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.901+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea003d4e0 con 0x7f0eb40c9f00 2024-04-25T12:35:32.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.901+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb40b0390 con 0x7f0eb40c9f00 2024-04-25T12:35:32.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.901+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0037040 con 0x7f0eb40c9f00 2024-04-25T12:35:32.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.901+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb40af370 con 0x7f0eb40c9f00 2024-04-25T12:35:32.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.902+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f0eb4076700 con 0x7f0eb40c9f00 2024-04-25T12:35:32.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.905+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ea0044070 con 0x7f0eb40c9f00 2024-04-25T12:35:32.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.906+0000 7f0eb0c00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e940744e0 0x7f0e940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.906+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 6619+0+0 (secure 0 0 0) 0x7f0ea00b53b0 con 0x7f0eb40c9f00 2024-04-25T12:35:32.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.906+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e940744e0 0x7f0e940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.906+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v35) ==== 54+0+16 (secure 0 0 0) 0x7f0ea004b020 con 0x7f0eb40c9f00 2024-04-25T12:35:32.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.906+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f0eb40af540 0x7f0eb40c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.906+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0eb40b1ca0 con 0x7f0eb40af540 2024-04-25T12:35:32.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.907+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e940744e0 0x7f0e940769f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8000f60 tx=0x7f0ea8007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:32.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.907+0000 7f0eb9400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f0eb40af540 0x7f0eb40c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.907+0000 7f0eb9400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f0eb40af540 0x7f0eb40c8c10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:32.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.908+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb40b1ca0 con 0x7f0eb40af540 2024-04-25T12:35:32.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.926+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb40744e0 con 0x7f0eb40af540 2024-04-25T12:35:32.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.929+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb40744e0 con 0x7f0eb40af540 2024-04-25T12:35:32.930 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:32.931 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:32.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.930+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0eb414ff70 0x7f0eb41f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.930+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f0eb4074100 con 0x7f0eb414ff70 2024-04-25T12:35:32.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.930+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0eb414ff70 0x7f0eb41f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.931+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0eb414ff70 0x7f0eb41f6800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:32.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.932+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb4074100 con 0x7f0eb414ff70 2024-04-25T12:35:32.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.965+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb40c8070 con 0x7f0eb414ff70 2024-04-25T12:35:32.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.968+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb40c8070 con 0x7f0eb414ff70 2024-04-25T12:35:32.968 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:32.969 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:32.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.968+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f0eb4147000 0x7f0eb4147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:32.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.968+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f0eb4147950 con 0x7f0eb4147000 2024-04-25T12:35:32.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.968+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f0eb4147000 0x7f0eb4147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:32.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.969+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f0eb4147000 0x7f0eb4147410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:32.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.971+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb4147950 con 0x7f0eb4147000 2024-04-25T12:35:32.978 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:32.977+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:32.982 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:32.982 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:32.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:32.982 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 74.029075603999999, 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:35:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:35:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 74.028963661000006, 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:35:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:35:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 74.028914861000004, 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 74.028843701, 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 74.028795289000001, 2024-04-25T12:35:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 74.020599360999995, 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:35:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:35:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.914355362999999, 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.914173362, 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.862139374999998, 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:35:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:35:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.862063419000002, 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.857634679, 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:35:33.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:35:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.857452616000003, 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.044 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.857348252000001, 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:35:33.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:35:33.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:35:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.856072695000002, 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:33.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:35:33.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.855018471000001, 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.053 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:22 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.851652125000001, 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026067460000000001, 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128290+0000", 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.474e-06 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128302+0000", 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2001000000000001e-05 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128323+0000", 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0613999999999999e-05 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128339+0000", 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5713999999999999e-05 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128423+0000", 2024-04-25T12:35:33.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4154999999999996e-05 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128451+0000", 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8473999999999999e-05 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130796+0000", 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002344245 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130896+0000", 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010006899999999999 2024-04-25T12:35:33.061 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.826152831000002, 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:33.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:35:33.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:33.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:44 2.14 2:2e2fa760:::200.00000001:head [write 3170~258 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.104868014000001, 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026976719999999999, 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875073+0000", 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875077+0000", 2024-04-25T12:35:33.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0749999999999999e-06 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875102+0000", 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5102000000000001e-05 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875144+0000", 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1647999999999998e-05 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875170+0000", 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5876000000000001e-05 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875363+0000", 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019320700000000001 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875551+0000", 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000188037 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875700+0000", 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148837 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877284+0000", 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001584483 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877626+0000", 2024-04-25T12:35:33.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034161599999999999 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877700+0000", 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4467000000000004e-05 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.877771+0000", 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0324e-05 2024-04-25T12:35:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.104641336, 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:35:33.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:35:33.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:35:33.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.062080783999999, 2024-04-25T12:35:33.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:33.083 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:33.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:32.999+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb40b1430 con 0x7f0eb4147000 2024-04-25T12:35:33.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.003+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb40b1430 con 0x7f0eb4147000 2024-04-25T12:35:33.085 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:33.085 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:33.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.003+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0eb4063660 0x7f0eb4063a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.003+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f0eb4063fb0 con 0x7f0eb4063660 2024-04-25T12:35:33.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.004+0000 7f0eb9400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0eb4063660 0x7f0eb4063a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.005+0000 7f0eb9400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0eb4063660 0x7f0eb4063a70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.006+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb4063fb0 con 0x7f0eb4063660 2024-04-25T12:35:33.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:33.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.023+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb4063fb0 con 0x7f0eb4063660 2024-04-25T12:35:33.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.026+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb4063fb0 con 0x7f0eb4063660 2024-04-25T12:35:33.088 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.026+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0eb4152430 0x7f0eb4152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.026+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f0eb42085f0 con 0x7f0eb4152430 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.027+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0eb4152430 0x7f0eb4152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.028+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0eb4152430 0x7f0eb4152840 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.029+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb42085f0 con 0x7f0eb4152430 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.044+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb4000950 con 0x7f0eb4152430 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.046+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb4000950 con 0x7f0eb4152430 2024-04-25T12:35:33.089 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:33.090 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:33.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.047+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f0eb4207850 0x7f0eb4207c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.047+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f0eb420ba10 con 0x7f0eb4207850 2024-04-25T12:35:33.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.047+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f0eb4207850 0x7f0eb4207c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.048+0000 7f0eb3e00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f0eb4207850 0x7f0eb4207c60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.049+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb420ba10 con 0x7f0eb4207850 2024-04-25T12:35:33.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.064+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb402c9d0 con 0x7f0eb4207850 2024-04-25T12:35:33.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.066+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb402c9d0 con 0x7f0eb4207850 2024-04-25T12:35:33.091 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:33.092 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:33.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.067+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0eb402cee0 0x7f0eb402d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.067+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f0eb402d900 con 0x7f0eb402cee0 2024-04-25T12:35:33.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.067+0000 7f0eb9400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0eb402cee0 0x7f0eb402d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.068+0000 7f0eb9400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0eb402cee0 0x7f0eb402d2f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.070+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb402d900 con 0x7f0eb402cee0 2024-04-25T12:35:33.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.084+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb4208570 con 0x7f0eb402cee0 2024-04-25T12:35:33.093 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:33.093 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:33.085+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:33.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.087+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb4208570 con 0x7f0eb402cee0 2024-04-25T12:35:33.094 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:33.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.087+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0eb420cdb0 0x7f0eb420f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.087+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f0eb4211fa0 con 0x7f0eb420cdb0 2024-04-25T12:35:33.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.088+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0eb420cdb0 0x7f0eb420f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.088+0000 7f0eb8a00640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0eb420cdb0 0x7f0eb420f1d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.089+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb4211fa0 con 0x7f0eb420cdb0 2024-04-25T12:35:33.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.104+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0eb420f710 con 0x7f0eb420cdb0 2024-04-25T12:35:33.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.106+0000 7f0eb0c00640 1 -- 172.21.15.123:0/1938016583 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0eb420f710 con 0x7f0eb420cdb0 2024-04-25T12:35:33.107 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:33.108 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f0eb40af540 msgr2=0x7f0eb40c8c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f0eb40af540 0x7f0eb40c8c10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0eb414ff70 msgr2=0x7f0eb41f6800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0eb414ff70 0x7f0eb41f6800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f0eb4147000 msgr2=0x7f0eb4147410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f0eb4147000 0x7f0eb4147410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0eb4063660 msgr2=0x7f0eb4063a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0eb4063660 0x7f0eb4063a70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0eb4152430 msgr2=0x7f0eb4152840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0eb4152430 0x7f0eb4152840 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f0eb4207850 msgr2=0x7f0eb4207c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f0eb4207850 0x7f0eb4207c60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0eb9400640 1 -- 172.21.15.123:0/1938016583 reap_dead start 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0eb402cee0 msgr2=0x7f0eb402d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0eb402cee0 0x7f0eb402d2f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0eb9400640 1 -- 172.21.15.123:0/1938016583 reap_dead start 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0eb420cdb0 msgr2=0x7f0eb420f1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0eb420cdb0 0x7f0eb420f1d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0eb9400640 1 -- 172.21.15.123:0/1938016583 reap_dead start 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0eb9400640 1 -- 172.21.15.123:0/1938016583 reap_dead start 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e940744e0 msgr2=0x7f0e940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.107+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e940744e0 0x7f0e940769f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8000f60 tx=0x7f0ea8007e30 comp rx=0 tx=0).stop 2024-04-25T12:35:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb40c9f00 msgr2=0x7f0eb414f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb40c9f00 0x7f0eb414f670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0031000 tx=0x7f0ea00046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 shutdown_connections 2024-04-25T12:35:33.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e940744e0 0x7f0e940769f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 --2- 172.21.15.123:0/1938016583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0eb40c9f00 0x7f0eb414f670 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 >> 172.21.15.123:0/1938016583 conn(0x7f0eb406f440 msgr2=0x7f0eb40b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:33.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 shutdown_connections 2024-04-25T12:35:33.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.108+0000 7f0ebb400640 1 -- 172.21.15.123:0/1938016583 wait complete. 2024-04-25T12:35:33.185 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:33.185+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:33.286 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:33.286+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:33.356 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:33.356 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:33.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:33.420 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:33.420+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:33.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:35:33.488 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:33.489+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:33.589 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:33.589+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:33.691 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:33.691+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:33.691 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:33.691 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:33.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:33.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.727+0000 7fae49400640 1 Processor -- start 2024-04-25T12:35:33.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.727+0000 7fae49400640 1 -- start start 2024-04-25T12:35:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae49400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fae440c4240 0x7fae440c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae49400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae49400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae440c4810 con 0x7fae440c90d0 2024-04-25T12:35:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae49400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fae4415b080 con 0x7fae440c4240 2024-04-25T12:35:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae49400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fae440ce380 con 0x7fae4415a730 2024-04-25T12:35:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae42800640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fae440c90d0 0x7fae440cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60348/0 (socket says 172.21.15.123:60348) 2024-04-25T12:35:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae41e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fae440c4240 0x7fae440c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae43200640 1 --2- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.730+0000 7fae42800640 1 -- 172.21.15.123:0/3220167631 learned_addr learned my addr 172.21.15.123:0/3220167631 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:33.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.731+0000 7fae41e00640 1 -- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 msgr2=0x7fae4415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:33.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.731+0000 7fae41e00640 1 --2- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.731+0000 7fae41e00640 1 -- 172.21.15.123:0/3220167631 >> v1:172.21.15.153:6789/0 conn(0x7fae440c90d0 legacy=0x7fae440cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.731+0000 7fae41e00640 1 -- 172.21.15.123:0/3220167631 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae440c83f0 con 0x7fae440c4240 2024-04-25T12:35:33.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.731+0000 7fae43200640 1 --2- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:33.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.732+0000 7fae41e00640 1 --2- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae440c2870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fae300097d0 tx=0x7fae3002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429c821757d0aa92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.732+0000 7fae41400640 1 -- 172.21.15.123:0/3220167631 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30037070 con 0x7fae440c4240 2024-04-25T12:35:33.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.732+0000 7fae41400640 1 -- 172.21.15.123:0/3220167631 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae30032440 con 0x7fae440c4240 2024-04-25T12:35:33.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.732+0000 7fae41400640 1 -- 172.21.15.123:0/3220167631 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30032810 con 0x7fae440c4240 2024-04-25T12:35:33.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 -- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 msgr2=0x7fae440c2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 --2- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae440c2870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fae300097d0 tx=0x7fae3002a250 comp rx=0 tx=0).stop 2024-04-25T12:35:33.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 -- 172.21.15.123:0/3220167631 shutdown_connections 2024-04-25T12:35:33.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 --2- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 --2- 172.21.15.123:0/3220167631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae440c2870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 -- 172.21.15.123:0/3220167631 >> 172.21.15.123:0/3220167631 conn(0x7fae4406c750 msgr2=0x7fae4406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:33.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 -- 172.21.15.123:0/3220167631 shutdown_connections 2024-04-25T12:35:33.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.733+0000 7fae49400640 1 -- 172.21.15.123:0/3220167631 wait complete. 2024-04-25T12:35:33.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.736+0000 7fae49400640 1 Processor -- start 2024-04-25T12:35:33.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.736+0000 7fae49400640 1 -- start start 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.737+0000 7fae49400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae44155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.737+0000 7fae49400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae440c90d0 0x7fae441f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.737+0000 7fae49400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.737+0000 7fae49400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae440c8c90 con 0x7fae440c90d0 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.737+0000 7fae49400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae440c89b0 con 0x7fae440c4240 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.737+0000 7fae49400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fae44073db0 con 0x7fae4415a730 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae440c90d0 0x7fae441f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae42800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae44155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae43200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae440c90d0 0x7fae441f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56308/0 (socket says 172.21.15.123:56308) 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 -- 172.21.15.123:0/2837578774 learned_addr learned my addr 172.21.15.123:0/2837578774 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae42800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae44155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33356/0 (socket says 172.21.15.123:33356) 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 -- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 msgr2=0x7fae4414a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4414a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 -- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 msgr2=0x7fae44155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae44155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.738+0000 7fae41e00640 1 -- 172.21.15.123:0/2837578774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441f9720 con 0x7fae440c90d0 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.739+0000 7fae41e00640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae440c90d0 0x7fae441f91e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae3002a840 tx=0x7fae300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.739+0000 7fae3aa00640 1 -- 172.21.15.123:0/2837578774 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30037040 con 0x7fae440c90d0 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.739+0000 7fae3aa00640 1 -- 172.21.15.123:0/2837578774 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae30042070 con 0x7fae440c90d0 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.739+0000 7fae3aa00640 1 -- 172.21.15.123:0/2837578774 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3003b930 con 0x7fae440c90d0 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.739+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae44155650 con 0x7fae440c90d0 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.739+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae44155910 con 0x7fae440c90d0 2024-04-25T12:35:33.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.740+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4406d500 con 0x7fae440c90d0 2024-04-25T12:35:33.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.742+0000 7fae3aa00640 1 -- 172.21.15.123:0/2837578774 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fae3003bad0 con 0x7fae440c90d0 2024-04-25T12:35:33.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.742+0000 7fae3aa00640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fae240744b0 0x7fae240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:33.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.743+0000 7fae3aa00640 1 -- 172.21.15.123:0/2837578774 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6619+0+0 (secure 0 0 0) 0x7fae300b5d50 con 0x7fae440c90d0 2024-04-25T12:35:33.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.743+0000 7fae42800640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fae240744b0 0x7fae240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:33.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.744+0000 7fae42800640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fae240744b0 0x7fae240769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae34007fd0 tx=0x7fae34007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:33.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.747+0000 7fae3aa00640 1 -- 172.21.15.123:0/2837578774 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae3007fb80 con 0x7fae440c90d0 2024-04-25T12:35:33.791 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:33.791+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:33.894 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:33.891+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:33.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:33.961+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fae44076700 con 0x7fae440c90d0 2024-04-25T12:35:33.977 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:33.977 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:33.992 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:33.993+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:34.000 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:34.094 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:34.094+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:34.123 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:34.123 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:34.132 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:34.194 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:34.195+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:34.244 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:34.244 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:34.253 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:34.296 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:34.296+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:34.406 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:34.405+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:34.407 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:34.407 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:34.419 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:35:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.421+0000 7fae3aa00640 1 -- 172.21.15.123:0/2837578774 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v36) ==== 101+0+0 (secure 0 0 0) 0x7fae30083e80 con 0x7fae440c90d0 2024-04-25T12:35:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:35:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.425+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fae240744b0 msgr2=0x7fae240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.425+0000 7fae49400640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fae240744b0 0x7fae240769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae34007fd0 tx=0x7fae34007e80 comp rx=0 tx=0).stop 2024-04-25T12:35:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.425+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae440c90d0 msgr2=0x7fae441f91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.425+0000 7fae49400640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae440c90d0 0x7fae441f91e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae3002a840 tx=0x7fae300046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:34.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 shutdown_connections 2024-04-25T12:35:34.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fae240744b0 0x7fae240769c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:34.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae4415a730 0x7fae4414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:34.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae440c90d0 0x7fae441f91e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:34.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 --2- 172.21.15.123:0/2837578774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae440c4240 0x7fae44155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:34.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 >> 172.21.15.123:0/2837578774 conn(0x7fae4406c750 msgr2=0x7fae4414cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:34.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 shutdown_connections 2024-04-25T12:35:34.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:34.426+0000 7fae49400640 1 -- 172.21.15.123:0/2837578774 wait complete. 2024-04-25T12:35:34.508 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:34.507+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:34.534 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:34.534 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:34.543 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:35:34.598 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:34.599+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:34.657 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:34.657 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:34.666 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:35:34.711 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:34.711+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:34.801 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:34.801+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:34.823 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:34.823 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:34.832 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:34.902 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:34.902+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:34.947 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:34.948 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:34.956 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:35.002 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:35.003+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:35.068 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:35.068 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:35.078 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:35.104 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:35.104+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:35.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:35.204 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:35.205+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:35.234 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:35.234 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:35.243 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:35.327 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:35.328+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:35.396 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:35.396 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:35.405 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:35.406 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:35.406+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.414+0000 7fe2f3000640 1 Processor -- start 2024-04-25T12:35:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.414+0000 7fe2f3000640 1 -- start start 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.416+0000 7fe2f3000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe2ec0c3730 0x7fe2ec0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.416+0000 7fe2f3000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2ec0ca8e0 0x7fe2ec159c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.416+0000 7fe2f3000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ec0cacf0 con 0x7fe2ec0c94a0 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.416+0000 7fe2f3000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ec15a180 con 0x7fe2ec0ca8e0 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.416+0000 7fe2f3000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ec0c8520 con 0x7fe2ec0c3730 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2ec0ca8e0 0x7fe2ec159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2ec0ca8e0 0x7fe2ec159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56312/0 (socket says 172.21.15.123:56312) 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1200640 1 -- 172.21.15.123:0/2152473920 learned_addr learned my addr 172.21.15.123:0/2152473920 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:35.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/2152473920 >> v2:172.21.15.153:3300/0 conn(0x7fe2ec0c3730 0x7fe2ec0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2ebe00640 1 -- 172.21.15.123:0/2152473920 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260109588 0 0) 0x7fe2e4001490 con 0x7fe2ec0c94a0 2024-04-25T12:35:35.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2ebe00640 1 -- 172.21.15.123:0/2152473920 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2d4003520 con 0x7fe2ec0c94a0 2024-04-25T12:35:35.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1200640 1 -- 172.21.15.123:0/2152473920 >> v2:172.21.15.153:3300/0 conn(0x7fe2ec0c3730 msgr2=0x7fe2ec0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:35.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1200640 1 --2- 172.21.15.123:0/2152473920 >> v2:172.21.15.153:3300/0 conn(0x7fe2ec0c3730 0x7fe2ec0c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1200640 1 -- 172.21.15.123:0/2152473920 >> v1:172.21.15.153:6789/0 conn(0x7fe2ec0c94a0 legacy=0x7fe2ec06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.417+0000 7fe2f1200640 1 -- 172.21.15.123:0/2152473920 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec0c7f60 con 0x7fe2ec0ca8e0 2024-04-25T12:35:35.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.419+0000 7fe2f1200640 1 --2- 172.21.15.123:0/2152473920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 0x7fe2ec159c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec074340 tx=0x7fe2d802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3da077aeedc8e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.419+0000 7fe2ebe00640 1 -- 172.21.15.123:0/2152473920 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8037070 con 0x7fe2ec0ca8e0 2024-04-25T12:35:35.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.419+0000 7fe2ebe00640 1 -- 172.21.15.123:0/2152473920 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d802ae50 con 0x7fe2ec0ca8e0 2024-04-25T12:35:35.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.419+0000 7fe2ebe00640 1 -- 172.21.15.123:0/2152473920 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8032840 con 0x7fe2ec0ca8e0 2024-04-25T12:35:35.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 -- 172.21.15.123:0/2152473920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 msgr2=0x7fe2ec159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 --2- 172.21.15.123:0/2152473920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 0x7fe2ec159c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec074340 tx=0x7fe2d802a300 comp rx=0 tx=0).stop 2024-04-25T12:35:35.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 -- 172.21.15.123:0/2152473920 shutdown_connections 2024-04-25T12:35:35.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 --2- 172.21.15.123:0/2152473920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 0x7fe2ec159c40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 --2- 172.21.15.123:0/2152473920 >> v2:172.21.15.153:3300/0 conn(0x7fe2ec0c3730 0x7fe2ec0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 -- 172.21.15.123:0/2152473920 >> 172.21.15.123:0/2152473920 conn(0x7fe2ec070b90 msgr2=0x7fe2ec070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 -- 172.21.15.123:0/2152473920 shutdown_connections 2024-04-25T12:35:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.420+0000 7fe2f3000640 1 -- 172.21.15.123:0/2152473920 wait complete. 2024-04-25T12:35:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.422+0000 7fe2f3000640 1 Processor -- start 2024-04-25T12:35:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.423+0000 7fe2f3000640 1 -- start start 2024-04-25T12:35:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f3000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ec0c3730 0x7fe2ec1f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f3000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ec0c94a0 0x7fe2ec072ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 0x7fe2ec154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec0ca0e0 con 0x7fe2ec0ca8e0 2024-04-25T12:35:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f3000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec0c9e00 con 0x7fe2ec0c94a0 2024-04-25T12:35:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f3000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec0ca3c0 con 0x7fe2ec0c3730 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f1c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ec0c3730 0x7fe2ec1f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 0x7fe2ec154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ec0c94a0 0x7fe2ec072ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f1c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ec0c3730 0x7fe2ec1f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:46968/0 (socket says 172.21.15.123:46968) 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.424+0000 7fe2f1c00640 1 -- 172.21.15.123:0/3325955935 learned_addr learned my addr 172.21.15.123:0/3325955935 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2f1c00640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ec0c94a0 msgr2=0x7fe2ec072ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ec0c94a0 0x7fe2ec072ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2f1c00640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 msgr2=0x7fe2ec154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2ec0ca8e0 0x7fe2ec154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2f1c00640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec1f6620 con 0x7fe2ec0c3730 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ec0c3730 0x7fe2ec1f60e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000e870 tx=0x7fe2e000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0017070 con 0x7fe2ec0c3730 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e000c940 con 0x7fe2ec0c3730 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.425+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000cd30 con 0x7fe2ec0c3730 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.426+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec0c6e10 con 0x7fe2ec0c3730 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.426+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec0c73d0 con 0x7fe2ec0c3730 2024-04-25T12:35:35.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.427+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe2ec076da0 con 0x7fe2ec0c3730 2024-04-25T12:35:35.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.428+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe2e001a070 con 0x7fe2ec0c3730 2024-04-25T12:35:35.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.429+0000 7fe2e9600640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2cc0744b0 0x7fe2cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.429+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 6619+0+0 (secure 0 0 0) 0x7fe2e0095740 con 0x7fe2ec0c3730 2024-04-25T12:35:35.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.429+0000 7fe2f1200640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2cc0744b0 0x7fe2cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.429+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v36) ==== 54+0+16 (secure 0 0 0) 0x7fe2e009b170 con 0x7fe2ec0c3730 2024-04-25T12:35:35.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.430+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe2ec1f67f0 0x7fe2ec154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.430+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec0c8740 con 0x7fe2ec1f67f0 2024-04-25T12:35:35.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.430+0000 7fe2f1200640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2cc0744b0 0x7fe2cc0769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8034000 tx=0x7fe2d802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.430+0000 7fe2f2600640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe2ec1f67f0 0x7fe2ec154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.430+0000 7fe2f2600640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe2ec1f67f0 0x7fe2ec154fd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.431+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec0c8740 con 0x7fe2ec1f67f0 2024-04-25T12:35:35.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.448+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec0008d0 con 0x7fe2ec1f67f0 2024-04-25T12:35:35.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.450+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec0008d0 con 0x7fe2ec1f67f0 2024-04-25T12:35:35.451 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:35.451 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:35.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.451+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe2ec152550 0x7fe2ec152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.451+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec158a40 con 0x7fe2ec152550 2024-04-25T12:35:35.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.451+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe2ec152550 0x7fe2ec152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.452+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe2ec152550 0x7fe2ec152960 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.453+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec158a40 con 0x7fe2ec152550 2024-04-25T12:35:35.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.469+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec0c7610 con 0x7fe2ec152550 2024-04-25T12:35:35.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.472+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec0c7610 con 0x7fe2ec152550 2024-04-25T12:35:35.473 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:35.473 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:35.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.473+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe2ec147fa0 0x7fe2ec1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.473+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec1488f0 con 0x7fe2ec147fa0 2024-04-25T12:35:35.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.473+0000 7fe2f1200640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe2ec147fa0 0x7fe2ec1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.474+0000 7fe2f1200640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe2ec147fa0 0x7fe2ec1483b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.475+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec1488f0 con 0x7fe2ec147fa0 2024-04-25T12:35:35.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.490+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec0c7820 con 0x7fe2ec147fa0 2024-04-25T12:35:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.492+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec0c7820 con 0x7fe2ec147fa0 2024-04-25T12:35:35.492 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:35.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.492+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe2ec0638b0 0x7fe2ec0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.492+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec063cc0 con 0x7fe2ec0638b0 2024-04-25T12:35:35.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.492+0000 7fe2f2600640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe2ec0638b0 0x7fe2ec0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.494+0000 7fe2f2600640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe2ec0638b0 0x7fe2ec0cb6c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.496+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec063cc0 con 0x7fe2ec0638b0 2024-04-25T12:35:35.507 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:35.507+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:35.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.511+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec0cbc00 con 0x7fe2ec0638b0 2024-04-25T12:35:35.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.515+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec0cbc00 con 0x7fe2ec0638b0 2024-04-25T12:35:35.516 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:35.516 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:35.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.516+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe2ec02c9d0 0x7fe2ec02cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.516+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec0cc550 con 0x7fe2ec02c9d0 2024-04-25T12:35:35.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.516+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe2ec02c9d0 0x7fe2ec02cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.517+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe2ec02c9d0 0x7fe2ec02cde0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.518+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec0cc550 con 0x7fe2ec02c9d0 2024-04-25T12:35:35.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.533+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec06d9f0 con 0x7fe2ec02c9d0 2024-04-25T12:35:35.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.535+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec06d9f0 con 0x7fe2ec02c9d0 2024-04-25T12:35:35.536 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:35.536 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:35.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.536+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fe2ec02d830 0x7fe2ec209230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.536+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec02de20 con 0x7fe2ec02d830 2024-04-25T12:35:35.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.536+0000 7fe2f1200640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fe2ec02d830 0x7fe2ec209230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.537+0000 7fe2f1200640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fe2ec02d830 0x7fe2ec209230 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.538+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec02de20 con 0x7fe2ec02d830 2024-04-25T12:35:35.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.553+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec02dda0 con 0x7fe2ec02d830 2024-04-25T12:35:35.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.555+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec02dda0 con 0x7fe2ec02d830 2024-04-25T12:35:35.556 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:35.556 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:35.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.556+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe2ec209c80 0x7fe2ec20c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.556+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec20f0c0 con 0x7fe2ec209c80 2024-04-25T12:35:35.557 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:35.557 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.557+0000 7fe2f2600640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe2ec209c80 0x7fe2ec20c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.558+0000 7fe2f2600640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe2ec209c80 0x7fe2ec20c0a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.559+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec20f0c0 con 0x7fe2ec209c80 2024-04-25T12:35:35.566 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:35.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.574+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec20f050 con 0x7fe2ec209c80 2024-04-25T12:35:35.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.577+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec20f050 con 0x7fe2ec209c80 2024-04-25T12:35:35.577 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:35.577 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:35.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.577+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe2ec20e2b0 0x7fe2ec20e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:35.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.577+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec2124c0 con 0x7fe2ec20e2b0 2024-04-25T12:35:35.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.578+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe2ec20e2b0 0x7fe2ec20e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:35.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.578+0000 7fe2f1c00640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe2ec20e2b0 0x7fe2ec20e6c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:35.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.579+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec2124c0 con 0x7fe2ec20e2b0 2024-04-25T12:35:35.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.594+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2ec20c660 con 0x7fe2ec20e2b0 2024-04-25T12:35:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2e9600640 1 -- 172.21.15.123:0/3325955935 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2ec20c660 con 0x7fe2ec20e2b0 2024-04-25T12:35:35.597 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe2ec1f67f0 msgr2=0x7fe2ec154fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe2ec1f67f0 0x7fe2ec154fd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe2ec152550 msgr2=0x7fe2ec152960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe2ec152550 0x7fe2ec152960 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe2ec147fa0 msgr2=0x7fe2ec1483b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe2ec147fa0 0x7fe2ec1483b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe2ec0638b0 msgr2=0x7fe2ec0cb6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe2ec0638b0 0x7fe2ec0cb6c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f2600640 1 -- 172.21.15.123:0/3325955935 reap_dead start 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe2ec02c9d0 msgr2=0x7fe2ec02cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe2ec02c9d0 0x7fe2ec02cde0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f2600640 1 -- 172.21.15.123:0/3325955935 reap_dead start 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fe2ec02d830 msgr2=0x7fe2ec209230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fe2ec02d830 0x7fe2ec209230 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe2ec209c80 msgr2=0x7fe2ec20c0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.597+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe2ec209c80 0x7fe2ec20c0a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe2ec20e2b0 msgr2=0x7fe2ec20e6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe2ec20e2b0 0x7fe2ec20e6c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:35.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2cc0744b0 msgr2=0x7fe2cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2cc0744b0 0x7fe2cc0769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8034000 tx=0x7fe2d802e000 comp rx=0 tx=0).stop 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ec0c3730 msgr2=0x7fe2ec1f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 --2- 172.21.15.123:0/3325955935 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ec0c3730 0x7fe2ec1f60e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000e870 tx=0x7fe2e000ed30 comp rx=0 tx=0).stop 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f2600640 1 -- 172.21.15.123:0/3325955935 reap_dead start 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f2600640 1 -- 172.21.15.123:0/3325955935 reap_dead start 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 shutdown_connections 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 >> 172.21.15.123:0/3325955935 conn(0x7fe2ec070b90 msgr2=0x7fe2ec157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.598+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 shutdown_connections 2024-04-25T12:35:35.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:35.599+0000 7fe2f3000640 1 -- 172.21.15.123:0/3325955935 wait complete. 2024-04-25T12:35:35.610 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:35.611+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 76.729835464000004, 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:35:35.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:35:35.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:35:35.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 76.729747810000006, 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:35:35.687 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 76.729403242000004, 2024-04-25T12:35:35.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:35.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:35:35.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 76.729330771999997, 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.697 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:35:35.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 76.729256793999994, 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.702 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.703 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:35:35.707 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 76.727219422000005, 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:35.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:35:35.709 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.711 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 76.726359286000005, 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.713 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.565673631000003, 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.716 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.717 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.718 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.719 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:35:35.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.565597517, 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.721 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.722 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.723 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:35:35.724 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:35:35.725 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.565551847999998, 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.726 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.727 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:35:35.729 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:35:35.729 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.729 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:35:35.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.731 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:35:35.732 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.565507072999999, 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.733 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.734 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:35:35.735 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.736 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.558890648999999, 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.737 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.738 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.739 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.558831159, 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.741 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.743 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 40.527860867000001, 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.744 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.747 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.748 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 40.527783356, 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.749 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.750 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:35:35.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:35:35.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:35:35.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 40.527735507000003, 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.754 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.755 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.758 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.759 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 40.527693616999997, 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.764 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 40.527659444999998, 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:35:35.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.769 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 40.527376703000002, 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:35:35.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 40.523236271999998, 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.773 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:35:35.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:35.776 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:35.777 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:35.778 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:35.778+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:35.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:35.879 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:35.878+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:35.979 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:35.980+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:36.058 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:36.058 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:36.058 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:36.059 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:36.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:36.081 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:36.080+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:36.181 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:36.181+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:36.282 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:36.283+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:36.384 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:36.384+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:36.401 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:36.401 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:36.401 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:36.401 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:36.402 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:36.423 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:36.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:35:36.520 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:36.520+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:36.547 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:35:36.547 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:35:36.548 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-04-25T12:35:36.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-04-25T12:35:36.595 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:36.595+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:36.686 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:36.686+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:36.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.723+0000 7f1f5f200640 1 Processor -- start 2024-04-25T12:35:36.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.723+0000 7f1f5f200640 1 -- start start 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.725+0000 7f1f5f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f580cda90 0x7f1f580cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.725+0000 7f1f5f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f580c2b80 con 0x7f1f580c8250 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.725+0000 7f1f5f200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f580c4400 con 0x7f1f58074690 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.725+0000 7f1f5f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1f5815d520 con 0x7f1f580cda90 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.725+0000 7f1f5d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f580cda90 0x7f1f580cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.725+0000 7f1f57400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1f580c8250 0x7f1f5815ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48058/0 (socket says 172.21.15.123:48058) 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.726+0000 7f1f5d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f580cda90 0x7f1f580cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56322/0 (socket says 172.21.15.123:56322) 2024-04-25T12:35:36.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.726+0000 7f1f57400640 1 -- 172.21.15.123:0/3948214480 learned_addr learned my addr 172.21.15.123:0/3948214480 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:36.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.726+0000 7f1f5d200640 1 -- 172.21.15.123:0/3948214480 >> v1:172.21.15.153:6789/0 conn(0x7f1f58074690 legacy=0x7f1f581566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.726+0000 7f1f5d200640 1 -- 172.21.15.123:0/3948214480 >> v1:172.21.15.123:6789/0 conn(0x7f1f580c8250 legacy=0x7f1f5815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.726+0000 7f1f5d200640 1 -- 172.21.15.123:0/3948214480 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f580c46e0 con 0x7f1f580cda90 2024-04-25T12:35:36.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.726+0000 7f1f56a00640 1 -- 172.21.15.123:0/3948214480 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 624368063 0 0) 0x7f1f580c2b80 con 0x7f1f580c8250 2024-04-25T12:35:36.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.727+0000 7f1f5d200640 1 --2- 172.21.15.123:0/3948214480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f580cdea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009d90 tx=0x7f1f5002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0019b3d932504c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:36.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.728+0000 7f1f56a00640 1 -- 172.21.15.123:0/3948214480 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50007740 con 0x7f1f580cda90 2024-04-25T12:35:36.728 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:36.728 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:36.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:36.728 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:36.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.775907677999996, 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012628781, 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951638+0000", 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951659+0000", 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1027999999999999e-05 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951691+0000", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2267000000000001e-05 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951710+0000", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9063999999999999e-05 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952026+0000", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000315719 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952367+0000", 2024-04-25T12:35:36.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034072600000000002 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955799+0000", 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034325969999999999 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962852+0000", 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070528889999999997 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964076+0000", 2024-04-25T12:35:36.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012234119999999999 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964178+0000", 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102166 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964264+0000", 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6175999999999996e-05 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:36.733 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.775769847000007, 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012746265999999999, 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:36.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951774+0000", 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-06 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951782+0000", 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2090000000000005e-06 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951796+0000", 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3559e-05 2024-04-25T12:35:36.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951808+0000", 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2183e-05 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956591+0000", 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047823550000000003 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956696+0000", 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105696 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956907+0000", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000211165 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962907+0000", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059995650000000001 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964283+0000", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013758189999999999 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964381+0000", 2024-04-25T12:35:36.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7977000000000001e-05 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964519+0000", 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138675 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:31 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.774601122999997, 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076541370000000001, 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952943+0000", 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952999+0000", 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5968999999999999e-05 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953032+0000", 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2598999999999997e-05 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953058+0000", 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6585999999999999e-05 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953207+0000", 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014848300000000001 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953263+0000", 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6005999999999999e-05 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960434+0000", 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071716219999999999 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960596+0000", 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016173899999999999 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.774419203999997, 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010940785, 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953125+0000", 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.22e-06 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953138+0000", 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2996000000000001e-05 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953158+0000", 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0173000000000002e-05 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953176+0000", 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7927e-05 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954316+0000", 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011402599999999999 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954375+0000", 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963968+0000", 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095929269999999994 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964065+0000", 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6628999999999999e-05 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:36.746 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.774320380000006, 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080627230000000008, 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953225+0000", 2024-04-25T12:35:36.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3640000000000002e-06 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953259+0000", 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3955999999999998e-05 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953292+0000", 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3549e-05 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953320+0000", 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7892e-05 2024-04-25T12:35:36.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953495+0000", 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017444800000000001 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953705+0000", 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021011000000000001 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953862+0000", 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015660399999999999 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959861+0000", 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059993959999999997 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961071+0000", 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012101900000000001 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961192+0000", 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121279 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961285+0000", 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2935000000000004e-05 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.774143643000002, 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010652488999999999, 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953401+0000", 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.513e-06 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953412+0000", 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209000000000001e-05 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953473+0000", 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0668999999999998e-05 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953496+0000", 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3246000000000001e-05 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956356+0000", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00286002 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956620+0000", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026411099999999998 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.957005+0000", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038473499999999998 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962757+0000", 2024-04-25T12:35:36.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057517080000000003 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963927+0000", 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00117023 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964002+0000", 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5483999999999998e-05 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964052+0000", 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9564000000000002e-05 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.774015192999997, 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011827087999999999, 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953529+0000", 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953594+0000", 2024-04-25T12:35:36.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4807999999999993e-05 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953617+0000", 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3220000000000001e-05 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953683+0000", 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5328999999999997e-05 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954191+0000", 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000508428 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040280099999999999 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954922+0000", 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032838300000000002 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960432+0000", 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055099989999999998 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965226+0000", 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047938700000000004 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965297+0000", 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0992000000000001e-05 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965355+0000", 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7695000000000002e-05 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:19 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.611111393000002, 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010949354999999999, 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116434+0000", 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116475+0000", 2024-04-25T12:35:36.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1486000000000002e-05 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116505+0000", 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9709e-05 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116529+0000", 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3839999999999999e-05 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116651+0000", 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122013 2024-04-25T12:35:36.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116685+0000", 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4513e-05 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127273+0000", 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010587653000000001 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127381+0000", 2024-04-25T12:35:36.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108204 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:20 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.610956266000002, 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014061374, 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116588+0000", 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0499999999999999e-06 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116602+0000", 2024-04-25T12:35:36.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3793e-05 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116621+0000", 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9545999999999999e-05 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116637+0000", 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5696e-05 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117243+0000", 2024-04-25T12:35:36.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060668400000000002 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117280+0000", 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6619000000000003e-05 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130559+0000", 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013278570999999999 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130648+0000", 2024-04-25T12:35:36.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9414999999999997e-05 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:27 2.9 e25/18 2:91bd965c:::501.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.610529616000001, 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013688128000000001, 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117015+0000", 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3200000000000001e-06 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117121+0000", 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106451 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117145+0000", 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.404e-05 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117210+0000", 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4314999999999997e-05 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120564+0000", 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033548050000000002 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120624+0000", 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9234999999999998e-05 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130654+0000", 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010030437999999999 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130702+0000", 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7524000000000001e-05 2024-04-25T12:35:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:18 2.13 e25/18 2:ca4dd7f5:::611.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.610176316999997, 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010076254, 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:36.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117369+0000", 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.88e-06 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117385+0000", 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6670000000000001e-05 2024-04-25T12:35:36.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117413+0000", 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7691999999999998e-05 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117435+0000", 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2416e-05 2024-04-25T12:35:36.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117549+0000", 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113436 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1509e-05 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127347+0000", 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097767759999999992 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127443+0000", 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5874999999999999e-05 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.608964643999997, 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012295627, 2024-04-25T12:35:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.728+0000 7f1f56a00640 1 -- 172.21.15.123:0/3948214480 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f500078e0 con 0x7f1f580cda90 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.728+0000 7f1f56a00640 1 -- 172.21.15.123:0/3948214480 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50007cb0 con 0x7f1f580cda90 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118582+0000", 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000001e-06 2024-04-25T12:35:36.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118599+0000", 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7176e-05 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2505999999999997e-05 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118659+0000", 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7559000000000002e-05 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120040+0000", 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001380956 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120305+0000", 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026497700000000001 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120707+0000", 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040189499999999999 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128035+0000", 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073284049999999996 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130711+0000", 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026752910000000002 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130787+0000", 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6384999999999995e-05 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130874+0000", 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7020000000000004e-05 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.608808644, 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012319366, 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.785 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118747+0000", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1286e-05 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118772+0000", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5150000000000001e-05 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118793+0000", 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0778999999999999e-05 2024-04-25T12:35:36.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121911+0000", 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031179409999999999 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122331+0000", 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041955999999999997 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124441+0000", 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021107629999999999 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130205+0000", 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057636140000000002 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130888+0000", 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068289100000000003 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130961+0000", 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3026000000000002e-05 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131054+0000", 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2855999999999993e-05 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:21 2.8 2:1470b2f8:::613.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.608719915999998, 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012409474, 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118824+0000", 2024-04-25T12:35:36.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118834+0000", 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118855+0000", 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999999e-05 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118875+0000", 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9809999999999998e-05 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119612+0000", 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073729299999999998 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119912+0000", 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029931000000000001 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120202+0000", 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000289907 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130342+0000", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010140188 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131091+0000", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074893900000000001 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131166+0000", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5272999999999996e-05 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131233+0000", 2024-04-25T12:35:36.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6625000000000007e-05 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.608611169, 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012992702, 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118933+0000", 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3069999999999999e-06 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118943+0000", 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7200000000000001e-06 2024-04-25T12:35:36.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118967+0000", 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3828999999999998e-05 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118986+0000", 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9057000000000001e-05 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124788+0000", 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00580271 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125092+0000", 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030394999999999999 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125479+0000", 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038660000000000002 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131299+0000", 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058200559999999997 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131823+0000", 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052382700000000004 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131888+0000", 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4740999999999996e-05 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131925+0000", 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6905e-05 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.799 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:5 2.7 e25/18 2:e455cf45:::60b.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.608487721000003, 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011712752, 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119057+0000", 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.296e-06 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119068+0000", 2024-04-25T12:35:36.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.189e-05 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119091+0000", 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2802999999999998e-05 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119110+0000", 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8473000000000001e-05 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121085+0000", 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019750689999999999 2024-04-25T12:35:36.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121149+0000", 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3979000000000005e-05 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130709+0000", 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009559817 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130768+0000", 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9425000000000001e-05 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.803 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:21 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.572698432999999, 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066623919999999996, 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154846+0000", 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.06e-06 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154856+0000", 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0713000000000001e-05 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154878+0000", 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1180000000000001e-05 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154896+0000", 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8343e-05 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155332+0000", 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043663000000000003 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155389+0000", 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6818999999999999e-05 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161381+0000", 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059914440000000003 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161507+0000", 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126203 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.807 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:22 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.572424662000003, 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064678330000000001, 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155120+0000", 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155131+0000", 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1793e-05 2024-04-25T12:35:36.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155155+0000", 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3306e-05 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155174+0000", 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9383000000000001e-05 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157101+0000", 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019271570000000001 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157161+0000", 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9945000000000003e-05 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161512+0000", 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043506389999999999 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161586+0000", 2024-04-25T12:35:36.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4250000000000002e-05 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:26 2.11 2:893cd530:::mds2_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.571487689000001, 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064127289999999998, 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156057+0000", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8330000000000001e-06 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156090+0000", 2024-04-25T12:35:36.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2719000000000003e-05 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156116+0000", 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6531999999999999e-05 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156140+0000", 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3453999999999999e-05 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156350+0000", 2024-04-25T12:35:36.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021060999999999999 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156467+0000", 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116823 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156594+0000", 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012698900000000001 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:36.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161255+0000", 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046609570000000003 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162360+0000", 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001105167 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162416+0000", 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6076999999999999e-05 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162468+0000", 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1567999999999998e-05 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.571337040000003, 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066007829999999998, 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156208+0000", 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5570000000000001e-06 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:36.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156216+0000", 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0900000000000005e-06 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156237+0000", 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1452999999999999e-05 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156255+0000", 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8091999999999999e-05 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156841+0000", 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00058573200000000005 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156983+0000", 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000141676 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162807+0000", 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058241830000000001 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:36.820 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.728+0000 7f1f5f200640 1 -- 172.21.15.123:0/3948214480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 msgr2=0x7f1f580cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.728+0000 7f1f5f200640 1 --2- 172.21.15.123:0/3948214480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f580cdea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009d90 tx=0x7f1f5002a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.729+0000 7f1f5f200640 1 -- 172.21.15.123:0/3948214480 shutdown_connections 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.729+0000 7f1f5f200640 1 --2- 172.21.15.123:0/3948214480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f580cdea0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.729+0000 7f1f5f200640 1 -- 172.21.15.123:0/3948214480 >> 172.21.15.123:0/3948214480 conn(0x7f1f5806c750 msgr2=0x7f1f5806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.729+0000 7f1f5f200640 1 -- 172.21.15.123:0/3948214480 shutdown_connections 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.729+0000 7f1f5f200640 1 -- 172.21.15.123:0/3948214480 wait complete. 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.731+0000 7f1f5f200640 1 Processor -- start 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.731+0000 7f1f5f200640 1 -- start start 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f58074690 0x7f1f581f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f580c8250 0x7f1f5815ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f5815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580c49c0 con 0x7f1f580cda90 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5815c8f0 con 0x7f1f58074690 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1f58074aa0 con 0x7f1f580c8250 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f5815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f57e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f58074690 0x7f1f581f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f5815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56324/0 (socket says 172.21.15.123:56324) 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f5d200640 1 -- 172.21.15.123:0/500523218 learned_addr learned my addr 172.21.15.123:0/500523218 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.733+0000 7f1f57400640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f580c8250 0x7f1f5815ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f57e00640 1 -- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f580c8250 msgr2=0x7f1f5815ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f57e00640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f580c8250 0x7f1f5815ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f57e00640 1 -- 172.21.15.123:0/500523218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 msgr2=0x7f1f5815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f57e00640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f5815d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f57e00640 1 -- 172.21.15.123:0/500523218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f58155a40 con 0x7f1f58074690 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f57400640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f580c8250 0x7f1f5815ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:36.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f5d200640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f5815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.734+0000 7f1f57e00640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f58074690 0x7f1f581f5930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800c390 tx=0x7f1f4800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.735+0000 7f1f4fe00640 1 -- 172.21.15.123:0/500523218 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48017070 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.735+0000 7f1f4fe00640 1 -- 172.21.15.123:0/500523218 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f480052d0 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.735+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f581f25d0 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.735+0000 7f1f4fe00640 1 -- 172.21.15.123:0/500523218 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48003830 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.735+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f58155ce0 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.736+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f58076700 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.737+0000 7f1f4fe00640 1 -- 172.21.15.123:0/500523218 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f4801a070 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.738+0000 7f1f4fe00640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1f380744e0 0x7f1f380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.738+0000 7f1f4fe00640 1 -- 172.21.15.123:0/500523218 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6619+0+0 (secure 0 0 0) 0x7f1f48094840 con 0x7f1f58074690 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.738+0000 7f1f57400640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1f380744e0 0x7f1f380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.739+0000 7f1f57400640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1f380744e0 0x7f1f380769f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000f60 tx=0x7f1f44007e40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.743+0000 7f1f4fe00640 1 -- 172.21.15.123:0/500523218 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f4805e550 con 0x7f1f58074690 2024-04-25T12:35:36.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:36.829 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:36.824+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:36.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.813+0000 7f78d1c00640 1 Processor -- start 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.813+0000 7f78d1c00640 1 -- start start 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.814+0000 7f78d1c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc154490 0x7f78cc0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.814+0000 7f78d1c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f78cc06c570 0x7f78cc073d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.814+0000 7f78d1c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78cc074280 0x7f78cc15de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.814+0000 7f78d1c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f78cc0c8b40 con 0x7f78cc074280 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.814+0000 7f78d1c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f78cc0c9280 con 0x7f78cc06c570 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.814+0000 7f78d1c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc0c9560 con 0x7f78cc154490 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78cc074280 0x7f78cc15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78cc074280 0x7f78cc15de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56330/0 (socket says 172.21.15.123:56330) 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 -- 172.21.15.123:0/961316632 learned_addr learned my addr 172.21.15.123:0/961316632 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78c3e00640 1 --2- 172.21.15.123:0/961316632 >> v2:172.21.15.153:3300/0 conn(0x7f78cc06c570 0x7f78cc073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78cbe00640 1 --2- 172.21.15.123:0/961316632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc154490 0x7f78cc0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 -- 172.21.15.123:0/961316632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc154490 msgr2=0x7f78cc0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 --2- 172.21.15.123:0/961316632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc154490 0x7f78cc0c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 -- 172.21.15.123:0/961316632 >> v2:172.21.15.153:3300/0 conn(0x7f78cc06c570 msgr2=0x7f78cc073d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 --2- 172.21.15.123:0/961316632 >> v2:172.21.15.153:3300/0 conn(0x7f78cc06c570 0x7f78cc073d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.815+0000 7f78d1200640 1 -- 172.21.15.123:0/961316632 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc0c9840 con 0x7f78cc074280 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.816+0000 7f78d1200640 1 --2- 172.21.15.123:0/961316632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc074280 0x7f78cc15de30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0cb6b0 tx=0x7f78c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf6510714ad64c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.816+0000 7f78cb400640 1 -- 172.21.15.123:0/961316632 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4036070 con 0x7f78cc074280 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.816+0000 7f78cb400640 1 -- 172.21.15.123:0/961316632 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c4031600 con 0x7f78cc074280 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78cb400640 1 -- 172.21.15.123:0/961316632 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c40319d0 con 0x7f78cc074280 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 -- 172.21.15.123:0/961316632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc074280 msgr2=0x7f78cc15de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 --2- 172.21.15.123:0/961316632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc074280 0x7f78cc15de30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0cb6b0 tx=0x7f78c402a6a0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 -- 172.21.15.123:0/961316632 shutdown_connections 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 --2- 172.21.15.123:0/961316632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc074280 0x7f78cc15de30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 --2- 172.21.15.123:0/961316632 >> v2:172.21.15.153:3300/0 conn(0x7f78cc06c570 0x7f78cc073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 --2- 172.21.15.123:0/961316632 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc154490 0x7f78cc0c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 -- 172.21.15.123:0/961316632 >> 172.21.15.123:0/961316632 conn(0x7f78cc0cad30 msgr2=0x7f78cc073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 -- 172.21.15.123:0/961316632 shutdown_connections 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.817+0000 7f78d1c00640 1 -- 172.21.15.123:0/961316632 wait complete. 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.820+0000 7f78d1c00640 1 Processor -- start 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.820+0000 7f78d1c00640 1 -- start start 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.821+0000 7f78d1c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc06c570 0x7f78cc1f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.821+0000 7f78d1c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78cc074280 0x7f78cc14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.821+0000 7f78d1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc154490 0x7f78cc156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.821+0000 7f78d1c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0c8f30 con 0x7f78cc154490 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.821+0000 7f78d1c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1548a0 con 0x7f78cc074280 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.821+0000 7f78d1c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc0c9b20 con 0x7f78cc06c570 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc154490 0x7f78cc156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc154490 0x7f78cc156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56340/0 (socket says 172.21.15.123:56340) 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 -- 172.21.15.123:0/1451221304 learned_addr learned my addr 172.21.15.123:0/1451221304 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78cbe00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc06c570 0x7f78cc1f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78c3e00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78cc074280 0x7f78cc14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 -- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc06c570 msgr2=0x7f78cc1f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc06c570 0x7f78cc1f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 -- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78cc074280 msgr2=0x7f78cc14fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78cc074280 0x7f78cc14fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 -- 172.21.15.123:0/1451221304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc15acf0 con 0x7f78cc154490 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78cbe00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78cc06c570 0x7f78cc1f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.822+0000 7f78d1200640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc154490 0x7f78cc156690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f78c402ac90 tx=0x7f78c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.823+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c403f020 con 0x7f78cc154490 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.823+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c4037040 con 0x7f78cc154490 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.823+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1f5f70 con 0x7f78cc154490 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.823+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4036070 con 0x7f78cc154490 2024-04-25T12:35:36.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.823+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc15af60 con 0x7f78cc154490 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.824+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78cc076da0 con 0x7f78cc154490 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.825+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f78c4042020 con 0x7f78cc154490 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.827+0000 7f78c8c00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78a80744b0 0x7f78a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.827+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6619+0+0 (secure 0 0 0) 0x7f78c40b57b0 con 0x7f78cc154490 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.827+0000 7f78cbe00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78a80744b0 0x7f78a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.827+0000 7f78c8c00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f78a807dd10 0x7f78a8080200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.828+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78c4047eb0 con 0x7f78a807dd10 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.828+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f78c40bc050 con 0x7f78cc154490 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.828+0000 7f78c3e00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f78a807dd10 0x7f78a8080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.828+0000 7f78cbe00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78a80744b0 0x7f78a80769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009e10 tx=0x7f78bc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.829+0000 7f78c3e00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f78a807dd10 0x7f78a8080200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:36.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.829+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78c4047eb0 con 0x7f78a807dd10 2024-04-25T12:35:36.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.848+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f78cc06d320 con 0x7f78a807dd10 2024-04-25T12:35:36.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.848+0000 7f78c8c00640 1 -- 172.21.15.123:0/1451221304 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f78cc06d320 con 0x7f78a807dd10 2024-04-25T12:35:36.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f78a807dd10 msgr2=0x7f78a8080200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f78a807dd10 0x7f78a8080200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78a80744b0 msgr2=0x7f78a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78a80744b0 0x7f78a80769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009e10 tx=0x7f78bc009190 comp rx=0 tx=0).stop 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc154490 msgr2=0x7f78cc156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 --2- 172.21.15.123:0/1451221304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc154490 0x7f78cc156690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f78c402ac90 tx=0x7f78c4004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1200640 1 -- 172.21.15.123:0/1451221304 reap_dead start 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 shutdown_connections 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 >> 172.21.15.123:0/1451221304 conn(0x7f78cc0cad30 msgr2=0x7f78cc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 shutdown_connections 2024-04-25T12:35:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.849+0000 7f78d1c00640 1 -- 172.21.15.123:0/1451221304 wait complete. 2024-04-25T12:35:36.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-04-25T12:35:36.924 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:36.924+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:36.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:36.956+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f1f581f23c0 con 0x7f1f58074690 2024-04-25T12:35:37.025 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:37.025+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:37.076 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:37.076 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:37.076 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:37.076 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:37.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:37.125 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:37.126+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:37.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.150+0000 7fe007200640 1 Processor -- start 2024-04-25T12:35:37.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.150+0000 7fe007200640 1 -- start start 2024-04-25T12:35:37.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.152+0000 7fe007200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0080ca3b0 0x7fe0080716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.152+0000 7fe007200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0080c3960 con 0x7fe008071c10 2024-04-25T12:35:37.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.152+0000 7fe007200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0080c51c0 con 0x7fe008160260 2024-04-25T12:35:37.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.152+0000 7fe007200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe0080c3730 con 0x7fe0080ca3b0 2024-04-25T12:35:37.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.152+0000 7fe005e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0080ca3b0 0x7fe0080716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.152+0000 7fe005e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0080ca3b0 0x7fe0080716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56356/0 (socket says 172.21.15.123:56356) 2024-04-25T12:35:37.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.153+0000 7fe005e00640 1 -- 172.21.15.123:0/3918134323 learned_addr learned my addr 172.21.15.123:0/3918134323 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:37.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.153+0000 7fe005400640 1 --1- 172.21.15.123:0/3918134323 >> v1:172.21.15.123:6789/0 conn(0x7fe008071c10 0x7fe00815fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48070/0 (socket says 172.21.15.123:48070) 2024-04-25T12:35:37.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.153+0000 7fe005e00640 1 -- 172.21.15.123:0/3918134323 >> v1:172.21.15.153:6789/0 conn(0x7fe008160260 legacy=0x7fe008166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.153+0000 7fe005e00640 1 -- 172.21.15.123:0/3918134323 >> v1:172.21.15.123:6789/0 conn(0x7fe008071c10 legacy=0x7fe00815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.153+0000 7fe005e00640 1 -- 172.21.15.123:0/3918134323 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00815d1e0 con 0x7fe0080ca3b0 2024-04-25T12:35:37.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.154+0000 7fe005e00640 1 --2- 172.21.15.123:0/3918134323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0080ca3b0 0x7fe0080716d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009870 tx=0x7fdff802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179f12debaa3eb58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.154+0000 7fe004a00640 1 -- 172.21.15.123:0/3918134323 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8037070 con 0x7fe0080ca3b0 2024-04-25T12:35:37.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.154+0000 7fe004a00640 1 -- 172.21.15.123:0/3918134323 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff802ae50 con 0x7fe0080ca3b0 2024-04-25T12:35:37.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.154+0000 7fe004a00640 1 -- 172.21.15.123:0/3918134323 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8032860 con 0x7fe0080ca3b0 2024-04-25T12:35:37.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.155+0000 7fe007200640 1 -- 172.21.15.123:0/3918134323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0080ca3b0 msgr2=0x7fe0080716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.155+0000 7fe007200640 1 --2- 172.21.15.123:0/3918134323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0080ca3b0 0x7fe0080716d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009870 tx=0x7fdff802a300 comp rx=0 tx=0).stop 2024-04-25T12:35:37.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.155+0000 7fe007200640 1 -- 172.21.15.123:0/3918134323 shutdown_connections 2024-04-25T12:35:37.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.155+0000 7fe007200640 1 --2- 172.21.15.123:0/3918134323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0080ca3b0 0x7fe0080716d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.155+0000 7fe007200640 1 -- 172.21.15.123:0/3918134323 >> 172.21.15.123:0/3918134323 conn(0x7fe008074520 msgr2=0x7fe0080ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.155+0000 7fe007200640 1 -- 172.21.15.123:0/3918134323 shutdown_connections 2024-04-25T12:35:37.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.155+0000 7fe007200640 1 -- 172.21.15.123:0/3918134323 wait complete. 2024-04-25T12:35:37.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.157+0000 7fe007200640 1 Processor -- start 2024-04-25T12:35:37.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.158+0000 7fe007200640 1 -- start start 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe007200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe008071c10 0x7fe008156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe007200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0080ca3b0 0x7fe0080c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe007200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe008160260 0x7fe0080cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe007200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0080c3c00 con 0x7fe008071c10 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe007200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0080c4510 con 0x7fe0080ca3b0 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe007200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe00815d4c0 con 0x7fe008160260 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe005e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe008071c10 0x7fe008156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe005e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe008071c10 0x7fe008156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56364/0 (socket says 172.21.15.123:56364) 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe005e00640 1 -- 172.21.15.123:0/2576451150 learned_addr learned my addr 172.21.15.123:0/2576451150 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.159+0000 7fe005400640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0080ca3b0 0x7fe0080c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe006800640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe008160260 0x7fe0080cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe005e00640 1 -- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe008160260 msgr2=0x7fe0080cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe005e00640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe008160260 0x7fe0080cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe005e00640 1 -- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0080ca3b0 msgr2=0x7fe0080c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe005e00640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0080ca3b0 0x7fe0080c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe005e00640 1 -- 172.21.15.123:0/2576451150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0080c8270 con 0x7fe008071c10 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe005400640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0080ca3b0 0x7fe0080c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe006800640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe008160260 0x7fe0080cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.160+0000 7fe005e00640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe008071c10 0x7fe008156760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdff8000c00 tx=0x7fdff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.161+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8041070 con 0x7fe008071c10 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.161+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff803c870 con 0x7fe008071c10 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.161+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8037040 con 0x7fe008071c10 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.161+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081f2ee0 con 0x7fe008071c10 2024-04-25T12:35:37.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.161+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081f34a0 con 0x7fe008071c10 2024-04-25T12:35:37.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.162+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe008151590 con 0x7fe008071c10 2024-04-25T12:35:37.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.163+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdff8044070 con 0x7fe008071c10 2024-04-25T12:35:37.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.164+0000 7fdffe000640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdfe80744b0 0x7fdfe80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.164+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6619+0+0 (secure 0 0 0) 0x7fdff80b5f60 con 0x7fe008071c10 2024-04-25T12:35:37.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.164+0000 7fdffe000640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdfe807dd10 0x7fdfe8080200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.164+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdff8041e70 con 0x7fdfe807dd10 2024-04-25T12:35:37.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.164+0000 7fe005400640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdfe80744b0 0x7fdfe80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.165+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdff80b8520 con 0x7fe008071c10 2024-04-25T12:35:37.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.165+0000 7fe006800640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdfe807dd10 0x7fdfe8080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.165+0000 7fe005400640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdfe80744b0 0x7fdfe80769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe008152f20 tx=0x7fdff4007e20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.165+0000 7fe006800640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdfe807dd10 0x7fdfe8080200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.166+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdff8041e70 con 0x7fdfe807dd10 2024-04-25T12:35:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.184+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fe0080c7970 con 0x7fdfe807dd10 2024-04-25T12:35:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.184+0000 7fdffe000640 1 -- 172.21.15.123:0/2576451150 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fe0080c7970 con 0x7fdfe807dd10 2024-04-25T12:35:37.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdfe807dd10 msgr2=0x7fdfe8080200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdfe807dd10 0x7fdfe8080200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdfe80744b0 msgr2=0x7fdfe80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdfe80744b0 0x7fdfe80769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe008152f20 tx=0x7fdff4007e20 comp rx=0 tx=0).stop 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe008071c10 msgr2=0x7fe008156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 --2- 172.21.15.123:0/2576451150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe008071c10 0x7fe008156760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdff8000c00 tx=0x7fdff80046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe006800640 1 -- 172.21.15.123:0/2576451150 reap_dead start 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 shutdown_connections 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 >> 172.21.15.123:0/2576451150 conn(0x7fe008074520 msgr2=0x7fe00815f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 shutdown_connections 2024-04-25T12:35:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.185+0000 7fe007200640 1 -- 172.21.15.123:0/2576451150 wait complete. 2024-04-25T12:35:37.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-04-25T12:35:37.226 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:37.227+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:37.327 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:37.327+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:37.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:35:37.348 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:35:37.377 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:37.377 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:37.377 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:37.377 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:37.378 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:37.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:37.439 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:37.439+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:37.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.443+0000 7f1f4fe00640 1 -- 172.21.15.123:0/500523218 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v37) ==== 87+0+0 (secure 0 0 0) 0x7f1f48062850 con 0x7f1f58074690 2024-04-25T12:35:37.448 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:35:37.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1f380744e0 msgr2=0x7f1f380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1f380744e0 0x7f1f380769f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000f60 tx=0x7f1f44007e40 comp rx=0 tx=0).stop 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f58074690 msgr2=0x7f1f581f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f58074690 0x7f1f581f5930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800c390 tx=0x7f1f4800c850 comp rx=0 tx=0).stop 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 shutdown_connections 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1f380744e0 0x7f1f380769f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f580cda90 0x7f1f5815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f580c8250 0x7f1f5815ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 --2- 172.21.15.123:0/500523218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f58074690 0x7f1f581f5930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 >> 172.21.15.123:0/500523218 conn(0x7f1f5806c750 msgr2=0x7f1f580c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 shutdown_connections 2024-04-25T12:35:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.446+0000 7f1f5f200640 1 -- 172.21.15.123:0/500523218 wait complete. 2024-04-25T12:35:37.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.513+0000 7f64b0a00640 1 Processor -- start 2024-04-25T12:35:37.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.513+0000 7f64b0a00640 1 -- start start 2024-04-25T12:35:37.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.514+0000 7f64b0a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f64ac0c9f00 0x7f64ac0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.514+0000 7f64b0a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac0730e0 0x7f64ac0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64b0a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64ac074b20 con 0x7f64ac154650 2024-04-25T12:35:37.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64b0a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0ca780 con 0x7f64ac0c9f00 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64b0a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac0caa60 con 0x7f64ac0730e0 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a9400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f64ac0c9f00 0x7f64ac0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a9e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f64ac154650 0x7f64ac15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60382/0 (socket says 172.21.15.123:60382) 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a9e00640 1 -- 172.21.15.123:0/3915031297 learned_addr learned my addr 172.21.15.123:0/3915031297 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64aa800640 1 --2- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac0730e0 0x7f64ac0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a9400640 1 -- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac0730e0 msgr2=0x7f64ac0734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a9400640 1 --2- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac0730e0 0x7f64ac0734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a9400640 1 -- 172.21.15.123:0/3915031297 >> v1:172.21.15.153:6789/0 conn(0x7f64ac154650 legacy=0x7f64ac15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a9400640 1 -- 172.21.15.123:0/3915031297 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac15da50 con 0x7f64ac0c9f00 2024-04-25T12:35:37.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.515+0000 7f64a8a00640 1 -- 172.21.15.123:0/3915031297 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218009268 0 0) 0x7f64ac074b20 con 0x7f64ac154650 2024-04-25T12:35:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.516+0000 7f64a9400640 1 --2- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0c9f00 0x7f64ac0c8170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64980097a0 tx=0x7f649802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a5b1071f7d2186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.516+0000 7f64a8a00640 1 -- 172.21.15.123:0/3915031297 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498037070 con 0x7f64ac0c9f00 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64a8a00640 1 -- 172.21.15.123:0/3915031297 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498032440 con 0x7f64ac0c9f00 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64a8a00640 1 -- 172.21.15.123:0/3915031297 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498032810 con 0x7f64ac0c9f00 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 -- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0c9f00 msgr2=0x7f64ac0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 --2- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0c9f00 0x7f64ac0c8170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64980097a0 tx=0x7f649802a210 comp rx=0 tx=0).stop 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 -- 172.21.15.123:0/3915031297 shutdown_connections 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 --2- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac0730e0 0x7f64ac0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 --2- 172.21.15.123:0/3915031297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0c9f00 0x7f64ac0c8170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 -- 172.21.15.123:0/3915031297 >> 172.21.15.123:0/3915031297 conn(0x7f64ac06f440 msgr2=0x7f64ac06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 -- 172.21.15.123:0/3915031297 shutdown_connections 2024-04-25T12:35:37.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.517+0000 7f64b0a00640 1 -- 172.21.15.123:0/3915031297 wait complete. 2024-04-25T12:35:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.520+0000 7f64b0a00640 1 Processor -- start 2024-04-25T12:35:37.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.520+0000 7f64b0a00640 1 -- start start 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.521+0000 7f64b0a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0730e0 0x7f64ac15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.521+0000 7f64b0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac0c9f00 0x7f64ac152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.521+0000 7f64b0a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac154650 0x7f64ac14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.521+0000 7f64b0a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac15e2f0 con 0x7f64ac0c9f00 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.521+0000 7f64b0a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac15e010 con 0x7f64ac0730e0 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.521+0000 7f64b0a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac154a60 con 0x7f64ac154650 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0730e0 0x7f64ac15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0730e0 0x7f64ac15b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33410/0 (socket says 172.21.15.123:33410) 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 -- 172.21.15.123:0/2661256218 learned_addr learned my addr 172.21.15.123:0/2661256218 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64aa800640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac154650 0x7f64ac14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9400640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac0c9f00 0x7f64ac152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 -- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac154650 msgr2=0x7f64ac14fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64ac154650 0x7f64ac14fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 -- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac0c9f00 msgr2=0x7f64ac152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac0c9f00 0x7f64ac152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 -- 172.21.15.123:0/2661256218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac1525c0 con 0x7f64ac0730e0 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.522+0000 7f64a9e00640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0730e0 0x7f64ac15b8a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c3f0 tx=0x7f649c00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.523+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c0074e0 con 0x7f64ac0730e0 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.523+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649c0047e0 con 0x7f64ac0730e0 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.523+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c004c10 con 0x7f64ac0730e0 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.523+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac14ece0 con 0x7f64ac0730e0 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.523+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac14f080 con 0x7f64ac0730e0 2024-04-25T12:35:37.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.524+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64ac1f7570 con 0x7f64ac0730e0 2024-04-25T12:35:37.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.525+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f649c011070 con 0x7f64ac0730e0 2024-04-25T12:35:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.526+0000 7f64a2000640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f648c0744e0 0x7f648c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.526+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7f649c0952f0 con 0x7f64ac0730e0 2024-04-25T12:35:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.526+0000 7f64a9400640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f648c0744e0 0x7f648c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.526+0000 7f64a2000640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f648c07dd40 0x7f648c080230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.526+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f648c080900 con 0x7f648c07dd40 2024-04-25T12:35:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.526+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f649c0956a0 con 0x7f64ac0730e0 2024-04-25T12:35:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.527+0000 7f64aa800640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f648c07dd40 0x7f648c080230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.527+0000 7f64a9400640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f648c0744e0 0x7f648c0769f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6498034000 tx=0x7f649802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.527+0000 7f64aa800640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f648c07dd40 0x7f648c080230 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.528 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:37.528+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:37.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.529+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f648c080900 con 0x7f648c07dd40 2024-04-25T12:35:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.547+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f64ac1f7010 con 0x7f648c07dd40 2024-04-25T12:35:37.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.547+0000 7f64a2000640 1 -- 172.21.15.123:0/2661256218 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f64a40153f0 con 0x7f648c07dd40 2024-04-25T12:35:37.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f648c07dd40 msgr2=0x7f648c080230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f648c07dd40 0x7f648c080230 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f648c0744e0 msgr2=0x7f648c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f648c0744e0 0x7f648c0769f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6498034000 tx=0x7f649802e000 comp rx=0 tx=0).stop 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0730e0 msgr2=0x7f64ac15b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 --2- 172.21.15.123:0/2661256218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac0730e0 0x7f64ac15b8a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c3f0 tx=0x7f649c00c8b0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64aa800640 1 -- 172.21.15.123:0/2661256218 reap_dead start 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 shutdown_connections 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 >> 172.21.15.123:0/2661256218 conn(0x7f64ac06f440 msgr2=0x7f64ac0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 shutdown_connections 2024-04-25T12:35:37.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.548+0000 7f64b0a00640 1 -- 172.21.15.123:0/2661256218 wait complete. 2024-04-25T12:35:37.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-04-25T12:35:37.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:37.659 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:37.659+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 89.666784335000003, 2024-04-25T12:35:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53759151699999996, 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024334+0000", 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7179999999999999e-06 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024345+0000", 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0608000000000001e-05 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024373+0000", 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7966e-05 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024392+0000", 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024490+0000", 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7823000000000002e-05 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for peered", 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024520+0000", 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0056000000000002e-05 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.559770+0000", 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53525036000000004 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560001+0000", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023139100000000001 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560330+0000", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032824500000000001 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561700+0000", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013704730000000001 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561742+0000", 2024-04-25T12:35:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1826000000000002e-05 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561855+0000", 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011287599999999999 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561924+0000", 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9484000000000001e-05 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 89.103861116000004, 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095393000000000006, 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587257+0000", 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9100000000000003e-07 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587490+0000", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000233186 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587502+0000", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2499e-05 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587512+0000", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7699999999999996e-06 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.591891+0000", 2024-04-25T12:35:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043783770000000001 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592036+0000", 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014543999999999999 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592306+0000", 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026964399999999998 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595185+0000", 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028791730000000001 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596722+0000", 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001537131 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596762+0000", 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0491000000000002e-05 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596795+0000", 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2798e-05 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 89.103588196999993, 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012300247, 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 24 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587530+0000", 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3799999999999996e-07 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587688+0000", 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015835999999999999 2024-04-25T12:35:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587699+0000", 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0727000000000001e-05 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587710+0000", 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0987e-05 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593399+0000", 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005688935 2024-04-25T12:35:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593685+0000", 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028662199999999999 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593893+0000", 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020758500000000001 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595599+0000", 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017058209999999999 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599668+0000", 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040696439999999999 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599769+0000", 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100752 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599829+0000", 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0075999999999999e-05 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 89.103366139000002, 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012351434, 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 25 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587752+0000", 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587758+0000", 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2199999999999997e-06 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587770+0000", 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2129e-05 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587778+0000", 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594340+0000", 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065618270000000001 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594415+0000", 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4535000000000004e-05 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594510+0000", 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5100999999999996e-05 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595607+0000", 2024-04-25T12:35:37.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010968239999999999 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599926+0000", 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043194469999999997 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600027+0000", 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100961 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600102+0000", 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5339000000000005e-05 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.713 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.898416948999994, 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091591999999999993, 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-04-25T12:35:37.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792701+0000", 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.17e-06 2024-04-25T12:35:37.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792911+0000", 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000210163 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792932+0000", 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0472e-05 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792952+0000", 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0553000000000001e-05 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799157+0000", 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062046030000000004 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799246+0000", 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8505999999999996e-05 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799389+0000", 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014319100000000001 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801232+0000", 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018427599999999999 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801753+0000", 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052153699999999998 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801814+0000", 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1014000000000002e-05 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801859+0000", 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5231000000000001e-05 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.897978601000005, 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092910930000000003, 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.719 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 52 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793140+0000", 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-06 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793151+0000", 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0966e-05 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793168+0000", 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793184+0000", 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6200000000000001e-05 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801271+0000", 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080863189999999998 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801360+0000", 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9248999999999999e-05 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801515+0000", 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000154883 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802197+0000", 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068243299999999998 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802339+0000", 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014159 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802393+0000", 2024-04-25T12:35:37.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4215000000000002e-05 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802430+0000", 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6434999999999999e-05 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.439975330999999, 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012798129, 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.724 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 70 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:35:37.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251143+0000", 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2729999999999999e-06 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251153+0000", 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8439999999999993e-06 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251172+0000", 2024-04-25T12:35:37.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8906000000000002e-05 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251183+0000", 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1653e-05 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257332+0000", 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061486459999999998 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257490+0000", 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000157947 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257724+0000", 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023408200000000001 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259695+0000", 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001971169 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263731+0000", 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035683 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263842+0000", 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110985 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263940+0000", 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7941000000000003e-05 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:37.729 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.439918422000005, 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013164844, 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 71 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251200+0000", 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2999999999999999e-07 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251206+0000", 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9399999999999999e-06 2024-04-25T12:35:37.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251215+0000", 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251223+0000", 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2260000000000005e-06 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259955+0000", 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087315629999999995 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260077+0000", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122263 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260310+0000", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232702 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263699+0000", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033888080000000001 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264218+0000", 2024-04-25T12:35:37.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051973199999999996 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264304+0000", 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5433999999999996e-05 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264363+0000", 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9743e-05 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.734 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.439876812999998, 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017184050999999999, 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 72 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251241+0000", 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.23e-07 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251477+0000", 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023584499999999999 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251504+0000", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7633000000000001e-05 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251516+0000", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.164e-05 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260993+0000", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094771119999999993 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261122+0000", 2024-04-25T12:35:37.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012831400000000001 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261319+0000", 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019727199999999999 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264078+0000", 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027589120000000001 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268258+0000", 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041806569999999999 2024-04-25T12:35:37.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268357+0000", 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8721000000000006e-05 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268424+0000", 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7222000000000001e-05 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:37.739 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.439581869999998, 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017105597, 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 73 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251536+0000", 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4300000000000001e-07 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251543+0000", 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3840000000000002e-06 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251552+0000", 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6860000000000001e-06 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251560+0000", 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0499999999999992e-06 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262615+0000", 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011054494 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262757+0000", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142521 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262999+0000", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024147400000000001 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264190+0000", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011911840000000001 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268472+0000", 2024-04-25T12:35:37.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042823940000000001 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268571+0000", 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8218000000000001e-05 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268641+0000", 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0248999999999998e-05 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.744 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.439544884, 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017309201999999999, 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 74 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251573+0000", 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9999999999999997e-07 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251579+0000", 2024-04-25T12:35:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6330000000000001e-06 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251592+0000", 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3869999999999999e-05 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251621+0000", 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8966e-05 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264505+0000", 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012883307 2024-04-25T12:35:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264627+0000", 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012204 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264832+0000", 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020571899999999999 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268227+0000", 2024-04-25T12:35:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003394308 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268663+0000", 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043630699999999998 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268761+0000", 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7774000000000003e-05 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268881+0000", 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120578 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.439474774999994, 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01751983, 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 75 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251643+0000", 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5300000000000003e-07 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251849+0000", 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000206345 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251863+0000", 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3077e-05 2024-04-25T12:35:37.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251873+0000", 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.08e-05 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265525+0000", 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013651876 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265667+0000", 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014156699999999999 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265939+0000", 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027219700000000001 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268931+0000", 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029921990000000001 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269008+0000", 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6619000000000006e-05 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269082+0000", 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4401999999999997e-05 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269162+0000", 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9895000000000007e-05 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.439227106999994, 2024-04-25T12:35:37.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018357248, 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 76 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251891+0000", 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7300000000000005e-07 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251905+0000", 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4279e-05 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251915+0000", 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251935+0000", 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.961e-05 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267152+0000", 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015217341000000001 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267276+0000", 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012396 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267509+0000", 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023329500000000001 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268988+0000", 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014781709999999999 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270041+0000", 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001053806 2024-04-25T12:35:37.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270161+0000", 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119494 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270247+0000", 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6382000000000001e-05 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.760 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.420002597999996, 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099439890000000003, 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 80 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271115+0000", 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271221+0000", 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105713 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271236+0000", 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5415999999999999e-05 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271240+0000", 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3449999999999999e-06 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275441+0000", 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042009090000000001 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275579+0000", 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013740700000000001 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275795+0000", 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021634900000000001 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.277926+0000", 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021311220000000001 2024-04-25T12:35:37.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.280958+0000", 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030320490000000002 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281016+0000", 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8232000000000001e-05 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281058+0000", 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1993999999999998e-05 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.064986232999999, 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094693499999999996, 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 92 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626132+0000", 2024-04-25T12:35:37.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4399999999999999e-07 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626137+0000", 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4929999999999998e-06 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626147+0000", 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5300000000000002e-06 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626155+0000", 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4130000000000006e-06 2024-04-25T12:35:37.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629532+0000", 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033774830000000001 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629626+0000", 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3394999999999996e-05 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629792+0000", 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016611100000000001 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633343+0000", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00355055 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635469+0000", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021262009999999999 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635548+0000", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9167999999999998e-05 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635600+0000", 2024-04-25T12:35:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2262000000000002e-05 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.064949897000005, 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096152330000000008, 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.771 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 93 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626168+0000", 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9699999999999995e-07 2024-04-25T12:35:37.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626290+0000", 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121941 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626304+0000", 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3729e-05 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626321+0000", 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7062999999999999e-05 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630147+0000", 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003826 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630229+0000", 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1953000000000006e-05 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630372+0000", 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014345199999999999 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633348+0000", 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029755789999999999 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635672+0000", 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023242829999999999 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635733+0000", 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0899000000000001e-05 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635782+0000", 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9636999999999999e-05 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 88.064775976999996, 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097157200000000006, 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:35:37.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 94 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:35:37.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626342+0000", 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4300000000000002e-07 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626348+0000", 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0059999999999998e-06 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626358+0000", 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626367+0000", 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8939999999999999e-06 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631146+0000", 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047795429999999998 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631307+0000", 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016041699999999999 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631539+0000", 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232258 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635089+0000", 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035503980000000002 2024-04-25T12:35:37.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635880+0000", 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079070200000000001 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635984+0000", 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104221 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.636057+0000", 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2597999999999998e-05 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.781 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.573066425, 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010749583, 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118054+0000", 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118075+0000", 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118119+0000", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4372000000000001e-05 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118163+0000", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3766000000000001e-05 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119484+0000", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001320494 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119803+0000", 2024-04-25T12:35:37.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031975599999999998 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120121+0000", 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031758999999999997 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128491+0000", 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083702940000000003 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128605+0000", 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113158 2024-04-25T12:35:37.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128733+0000", 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012861099999999999 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128800+0000", 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7086000000000001e-05 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:37.786 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.572867438999999, 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011301215999999999, 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118251+0000", 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6160000000000001e-06 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118263+0000", 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2243e-05 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118305+0000", 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1838999999999997e-05 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118327+0000", 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2082999999999998e-05 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120981+0000", 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002653201 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121221+0000", 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000240058 2024-04-25T12:35:37.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121655+0000", 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043405400000000002 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128501+0000", 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068465519999999997 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128984+0000", 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048240400000000003 2024-04-25T12:35:37.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129487+0000", 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050289499999999995 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129551+0000", 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4270999999999995e-05 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.572760008000003, 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012520414000000001, 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:37.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118358+0000", 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3230000000000001e-06 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118368+0000", 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3899999999999999e-06 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2481999999999999e-05 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118410+0000", 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9304e-05 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126108+0000", 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076979409999999998 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126379+0000", 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027136700000000001 2024-04-25T12:35:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126653+0000", 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274238 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129707+0000", 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030541800000000001 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130686+0000", 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097908900000000004 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130791+0000", 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104824 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130877+0000", 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6275999999999999e-05 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.796 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:37.797 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:37.797 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:37.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:37.798 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:37.798+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:37.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.872+0000 7f6129000640 1 Processor -- start 2024-04-25T12:35:37.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.873+0000 7f6129000640 1 -- start start 2024-04-25T12:35:37.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.874+0000 7f6129000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c4e70 0x7f61240c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.874+0000 7f6129000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f612415c740 con 0x7f61240c5280 2024-04-25T12:35:37.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6129000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6124073a60 con 0x7f61240c9320 2024-04-25T12:35:37.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6129000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6124073d40 con 0x7f61240c4e70 2024-04-25T12:35:37.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6122800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f61240c5280 0x7f612415cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48072/0 (socket says 172.21.15.123:48072) 2024-04-25T12:35:37.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6122800640 1 -- 172.21.15.123:0/3146546600 learned_addr learned my addr 172.21.15.123:0/3146546600 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:37.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6121400640 1 --2- 172.21.15.123:0/3146546600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c4e70 0x7f61240c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6120a00640 1 -- 172.21.15.123:0/3146546600 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441290554 0 0) 0x7f6124073a60 con 0x7f61240c9320 2024-04-25T12:35:37.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6120a00640 1 -- 172.21.15.123:0/3146546600 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6108003520 con 0x7f61240c9320 2024-04-25T12:35:37.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6120a00640 1 -- 172.21.15.123:0/3146546600 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083430414 0 0) 0x7f612415c740 con 0x7f61240c5280 2024-04-25T12:35:37.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.875+0000 7f6120a00640 1 -- 172.21.15.123:0/3146546600 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6124073a60 con 0x7f61240c5280 2024-04-25T12:35:37.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.876+0000 7f6121400640 1 -- 172.21.15.123:0/3146546600 >> v1:172.21.15.153:6789/0 conn(0x7f61240c9320 legacy=0x7f6124156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.876+0000 7f6121400640 1 -- 172.21.15.123:0/3146546600 >> v1:172.21.15.123:6789/0 conn(0x7f61240c5280 legacy=0x7f612415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.876+0000 7f6121400640 1 -- 172.21.15.123:0/3146546600 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6124074020 con 0x7f61240c4e70 2024-04-25T12:35:37.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.877+0000 7f6121400640 1 --2- 172.21.15.123:0/3146546600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c4e70 0x7f61240c9a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61100097d0 tx=0x7f611002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2eaaaadad6ad4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.877+0000 7f6120a00640 1 -- 172.21.15.123:0/3146546600 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110037070 con 0x7f61240c4e70 2024-04-25T12:35:37.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.877+0000 7f6120a00640 1 -- 172.21.15.123:0/3146546600 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6110032440 con 0x7f61240c4e70 2024-04-25T12:35:37.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.877+0000 7f6120a00640 1 -- 172.21.15.123:0/3146546600 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100327f0 con 0x7f61240c4e70 2024-04-25T12:35:37.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.878+0000 7f6129000640 1 -- 172.21.15.123:0/3146546600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c4e70 msgr2=0x7f61240c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.878+0000 7f6129000640 1 --2- 172.21.15.123:0/3146546600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c4e70 0x7f61240c9a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61100097d0 tx=0x7f611002a210 comp rx=0 tx=0).stop 2024-04-25T12:35:37.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.878+0000 7f6129000640 1 -- 172.21.15.123:0/3146546600 shutdown_connections 2024-04-25T12:35:37.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.878+0000 7f6129000640 1 --2- 172.21.15.123:0/3146546600 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c4e70 0x7f61240c9a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.878+0000 7f6129000640 1 -- 172.21.15.123:0/3146546600 >> 172.21.15.123:0/3146546600 conn(0x7f612406c990 msgr2=0x7f612406cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.878+0000 7f6129000640 1 -- 172.21.15.123:0/3146546600 shutdown_connections 2024-04-25T12:35:37.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.878+0000 7f6129000640 1 -- 172.21.15.123:0/3146546600 wait complete. 2024-04-25T12:35:37.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.880+0000 7f6129000640 1 Processor -- start 2024-04-25T12:35:37.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.880+0000 7f6129000640 1 -- start start 2024-04-25T12:35:37.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6129000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61240c4e70 0x7f61241e3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6129000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c5280 0x7f61240cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6129000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61240c9320 0x7f6124150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6129000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61240c85c0 con 0x7f61240c9320 2024-04-25T12:35:37.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6129000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61240745e0 con 0x7f61240c4e70 2024-04-25T12:35:37.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6129000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f612415d350 con 0x7f61240c5280 2024-04-25T12:35:37.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6121400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c5280 0x7f61240cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6122800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61240c9320 0x7f6124150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6121e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61240c4e70 0x7f61241e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6121400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c5280 0x7f61240cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:47044/0 (socket says 172.21.15.123:47044) 2024-04-25T12:35:37.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.882+0000 7f6121400640 1 -- 172.21.15.123:0/3161665133 learned_addr learned my addr 172.21.15.123:0/3161665133 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.883+0000 7f6121400640 1 -- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61240c4e70 msgr2=0x7f61241e3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.883+0000 7f6121400640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61240c4e70 0x7f61241e3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.883+0000 7f6121400640 1 -- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61240c9320 msgr2=0x7f6124150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.883+0000 7f6121400640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61240c9320 0x7f6124150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.883+0000 7f6121400640 1 -- 172.21.15.123:0/3161665133 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61240cb8a0 con 0x7f61240c5280 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.883+0000 7f6122800640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61240c9320 0x7f6124150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.883+0000 7f6121400640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c5280 0x7f61240cc730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6110035040 tx=0x7f61100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.884+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110037040 con 0x7f61240c5280 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.884+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6110042070 con 0x7f61240c5280 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.884+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110032830 con 0x7f61240c5280 2024-04-25T12:35:37.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.884+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61240cbb40 con 0x7f61240c5280 2024-04-25T12:35:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.884+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6124153b10 con 0x7f61240c5280 2024-04-25T12:35:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.885+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6124076700 con 0x7f61240c5280 2024-04-25T12:35:37.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.886+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61100329d0 con 0x7f61240c5280 2024-04-25T12:35:37.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.888+0000 7f611a000640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61040744b0 0x7f61040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.888+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7f61100b6890 con 0x7f61240c5280 2024-04-25T12:35:37.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.888+0000 7f6121e00640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61040744b0 0x7f61040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.888+0000 7f611a000640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f610407dd10 0x7f6104080200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.888+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6110050e50 con 0x7f610407dd10 2024-04-25T12:35:37.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.888+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f61100b6c40 con 0x7f61240c5280 2024-04-25T12:35:37.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.889+0000 7f6122800640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f610407dd10 0x7f6104080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.889+0000 7f6121e00640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61040744b0 0x7f61040769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61140011e0 tx=0x7f6114009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.889+0000 7f6122800640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f610407dd10 0x7f6104080200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.891+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6110050e50 con 0x7f610407dd10 2024-04-25T12:35:37.899 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:37.899+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:37.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.899+0000 7fb41cc00640 1 Processor -- start 2024-04-25T12:35:37.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.899+0000 7fb41cc00640 1 -- start start 2024-04-25T12:35:37.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb41cc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb4180c8170 0x7fb4180ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb41cc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb418074400 con 0x7fb4181426a0 2024-04-25T12:35:37.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb41cc00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4181595e0 con 0x7fb4180c8fd0 2024-04-25T12:35:37.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb41cc00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb418074670 con 0x7fb4180c8170 2024-04-25T12:35:37.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb415400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb4181426a0 0x7fb418159dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48086/0 (socket says 172.21.15.123:48086) 2024-04-25T12:35:37.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb415400640 1 -- 172.21.15.123:0/3144736128 learned_addr learned my addr 172.21.15.123:0/3144736128 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:37.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb416800640 1 --2- 172.21.15.123:0/3144736128 >> v2:172.21.15.153:3300/0 conn(0x7fb4180c8170 0x7fb4180ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb414a00640 1 -- 172.21.15.123:0/3144736128 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1678343357 0 0) 0x7fb4181595e0 con 0x7fb4180c8fd0 2024-04-25T12:35:37.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb414a00640 1 -- 172.21.15.123:0/3144736128 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3fc003520 con 0x7fb4180c8fd0 2024-04-25T12:35:37.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb414a00640 1 -- 172.21.15.123:0/3144736128 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039846983 0 0) 0x7fb418074400 con 0x7fb4181426a0 2024-04-25T12:35:37.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb414a00640 1 -- 172.21.15.123:0/3144736128 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4181595e0 con 0x7fb4181426a0 2024-04-25T12:35:37.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.902+0000 7fb416800640 1 -- 172.21.15.123:0/3144736128 >> v1:172.21.15.153:6789/0 conn(0x7fb4180c8fd0 legacy=0x7fb418153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.903+0000 7fb416800640 1 -- 172.21.15.123:0/3144736128 >> v1:172.21.15.123:6789/0 conn(0x7fb4181426a0 legacy=0x7fb418159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.903+0000 7fb416800640 1 -- 172.21.15.123:0/3144736128 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb418074950 con 0x7fb4180c8170 2024-04-25T12:35:37.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.904+0000 7fb416800640 1 --2- 172.21.15.123:0/3144736128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 0x7fb4180ca200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb410009d90 tx=0x7fb41002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787709cb3c6dbb23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.904+0000 7fb414a00640 1 -- 172.21.15.123:0/3144736128 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410007740 con 0x7fb4180c8170 2024-04-25T12:35:37.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.904+0000 7fb414a00640 1 -- 172.21.15.123:0/3144736128 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4100078e0 con 0x7fb4180c8170 2024-04-25T12:35:37.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.904+0000 7fb414a00640 1 -- 172.21.15.123:0/3144736128 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410007cb0 con 0x7fb4180c8170 2024-04-25T12:35:37.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.905+0000 7fb41cc00640 1 -- 172.21.15.123:0/3144736128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 msgr2=0x7fb4180ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.905+0000 7fb41cc00640 1 --2- 172.21.15.123:0/3144736128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 0x7fb4180ca200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb410009d90 tx=0x7fb41002a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.905+0000 7fb41cc00640 1 -- 172.21.15.123:0/3144736128 shutdown_connections 2024-04-25T12:35:37.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.905+0000 7fb41cc00640 1 --2- 172.21.15.123:0/3144736128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 0x7fb4180ca200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.905+0000 7fb41cc00640 1 -- 172.21.15.123:0/3144736128 >> 172.21.15.123:0/3144736128 conn(0x7fb41806f440 msgr2=0x7fb41806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.905+0000 7fb41cc00640 1 -- 172.21.15.123:0/3144736128 shutdown_connections 2024-04-25T12:35:37.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.905+0000 7fb41cc00640 1 -- 172.21.15.123:0/3144736128 wait complete. 2024-04-25T12:35:37.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.907+0000 7fb41cc00640 1 Processor -- start 2024-04-25T12:35:37.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.908+0000 7fb41cc00640 1 -- start start 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb41cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 0x7fb4181f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb41cc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180c8fd0 0x7fb4180c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb41cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4181426a0 0x7fb418141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb41cc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4180c8580 con 0x7fb4181426a0 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb41cc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb4180c4880 con 0x7fb4180c8170 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb41cc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb4180c8700 con 0x7fb4180c8fd0 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb416800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4181426a0 0x7fb418141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb415e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 0x7fb4181f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.909+0000 7fb416800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4181426a0 0x7fb418141110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56388/0 (socket says 172.21.15.123:56388) 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb415e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 0x7fb4181f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33424/0 (socket says 172.21.15.123:33424) 2024-04-25T12:35:37.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 learned_addr learned my addr 172.21.15.123:0/318981607 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:37.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb415400640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180c8fd0 0x7fb4180c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180c8fd0 msgr2=0x7fb4180c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f6124073510 con 0x7f610407dd10 2024-04-25T12:35:37.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb416800640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180c8fd0 0x7fb4180c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 msgr2=0x7fb4181f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb416800640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180c8170 0x7fb4181f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb418157ad0 con 0x7fb4181426a0 2024-04-25T12:35:37.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.910+0000 7fb416800640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4181426a0 0x7fb418141110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb410009eb0 tx=0x7fb410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7f611a000640 1 -- 172.21.15.123:0/3161665133 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f6124073510 con 0x7f610407dd10 2024-04-25T12:35:37.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41003f070 con 0x7fb4181426a0 2024-04-25T12:35:37.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb410031bc0 con 0x7fb4181426a0 2024-04-25T12:35:37.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4180c6ea0 con 0x7fb4181426a0 2024-04-25T12:35:37.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410009450 con 0x7fb4181426a0 2024-04-25T12:35:37.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f610407dd10 msgr2=0x7f6104080200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7f6129000640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f610407dd10 0x7f6104080200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.911+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4180c5600 con 0x7fb4181426a0 2024-04-25T12:35:37.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61040744b0 msgr2=0x7f61040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f61040744b0 0x7f61040769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61140011e0 tx=0x7f6114009040 comp rx=0 tx=0).stop 2024-04-25T12:35:37.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c5280 msgr2=0x7f61240cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:37.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 --2- 172.21.15.123:0/3161665133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61240c5280 0x7f61240cc730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6110035040 tx=0x7f61100046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:37.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6122800640 1 -- 172.21.15.123:0/3161665133 reap_dead start 2024-04-25T12:35:37.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 shutdown_connections 2024-04-25T12:35:37.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 >> 172.21.15.123:0/3161665133 conn(0x7f612406c990 msgr2=0x7f6124154680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:37.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 shutdown_connections 2024-04-25T12:35:37.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.912+0000 7f6129000640 1 -- 172.21.15.123:0/3161665133 wait complete. 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.913+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb418157ca0 con 0x7fb4181426a0 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.914+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb410044460 con 0x7fb4181426a0 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.915+0000 7fb40e000640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3f80744e0 0x7fb3f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.915+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7fb4100b6c80 con 0x7fb4181426a0 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.915+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7fb410049060 con 0x7fb4181426a0 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.915+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3f80744e0 0x7fb3f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.916+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb4180c5a70 0x7fb418157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.916+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4180c5840 con 0x7fb4180c5a70 2024-04-25T12:35:37.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.916+0000 7fb415400640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb4180c5a70 0x7fb418157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.916+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3f80744e0 0x7fb3f80769f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4080011e0 tx=0x7fb408009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.917+0000 7fb415400640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb4180c5a70 0x7fb418157260 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.917+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4180c5840 con 0x7fb4180c5a70 2024-04-25T12:35:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.934+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb4180c9980 con 0x7fb4180c5a70 2024-04-25T12:35:37.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.936+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb4180c9980 con 0x7fb4180c5a70 2024-04-25T12:35:37.936 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:37.936 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:37.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.936+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb4180c5fc0 0x7fb4181e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.936+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb4180cce90 con 0x7fb4180c5fc0 2024-04-25T12:35:37.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.936+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb4180c5fc0 0x7fb4181e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.937+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb4180c5fc0 0x7fb4181e50f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.938+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4180cce90 con 0x7fb4180c5fc0 2024-04-25T12:35:37.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-04-25T12:35:37.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.954+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb41815a140 con 0x7fb4180c5fc0 2024-04-25T12:35:37.959 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:37.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.956+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb41815a140 con 0x7fb4180c5fc0 2024-04-25T12:35:37.959 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:37.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.956+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb4181f6270 0x7fb4181f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.956+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb4180cb570 con 0x7fb4181f6270 2024-04-25T12:35:37.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.957+0000 7fb415400640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb4181f6270 0x7fb4181f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.958+0000 7fb415400640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb4181f6270 0x7fb4181f6680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.959+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4180cb570 con 0x7fb4181f6270 2024-04-25T12:35:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.974+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb4180cba80 con 0x7fb4181f6270 2024-04-25T12:35:37.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.976+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb4180cba80 con 0x7fb4181f6270 2024-04-25T12:35:37.976 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:37.977 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:37.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.976+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb418062fe0 0x7fb4180633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.976+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb4180cba00 con 0x7fb418062fe0 2024-04-25T12:35:37.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.977+0000 7fb416800640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb418062fe0 0x7fb4180633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.977+0000 7fb416800640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb418062fe0 0x7fb4180633f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.978+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4180cba00 con 0x7fb418062fe0 2024-04-25T12:35:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.994+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb418063c00 con 0x7fb418062fe0 2024-04-25T12:35:37.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.996+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb418063c00 con 0x7fb418062fe0 2024-04-25T12:35:37.996 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:37.996 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:37.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.996+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb41802cee0 0x7fb41802d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:37.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.996+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb4180cba00 con 0x7fb41802cee0 2024-04-25T12:35:37.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.996+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb41802cee0 0x7fb41802d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:37.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.997+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb41802cee0 0x7fb41802d2f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:37.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:37.998+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4180cba00 con 0x7fb41802cee0 2024-04-25T12:35:37.999 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:38.000+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:38.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.027+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb4180cba00 con 0x7fb41802cee0 2024-04-25T12:35:38.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.030+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb4180cba00 con 0x7fb41802cee0 2024-04-25T12:35:38.030 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:38.031 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:38.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.030+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb418206f70 0x7fb418209390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.030+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb4180008d0 con 0x7fb418206f70 2024-04-25T12:35:38.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.031+0000 7fb415400640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb418206f70 0x7fb418209390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.031+0000 7fb415400640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb418206f70 0x7fb418209390 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.032+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4180008d0 con 0x7fb418206f70 2024-04-25T12:35:38.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.047+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb418063c50 con 0x7fb418206f70 2024-04-25T12:35:38.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.050+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb418063c50 con 0x7fb418206f70 2024-04-25T12:35:38.050 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:38.051 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:38.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.050+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb418209de0 0x7fb41820c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.050+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb41820efd0 con 0x7fb418209de0 2024-04-25T12:35:38.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.051+0000 7fb416800640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb418209de0 0x7fb41820c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.051+0000 7fb416800640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb418209de0 0x7fb41820c200 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.052+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb41820efd0 con 0x7fb418209de0 2024-04-25T12:35:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.067+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb41820c740 con 0x7fb418209de0 2024-04-25T12:35:38.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.070+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb41820c740 con 0x7fb418209de0 2024-04-25T12:35:38.070 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:38.070 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:38.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.070+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb41820cc50 0x7fb41820f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.070+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb418212170 con 0x7fb41820cc50 2024-04-25T12:35:38.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.071+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb41820cc50 0x7fb41820f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.074+0000 7fb415e00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb41820cc50 0x7fb41820f0d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.076+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb418212170 con 0x7fb41820cc50 2024-04-25T12:35:38.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.091+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb418210e90 con 0x7fb41820cc50 2024-04-25T12:35:38.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb40e000640 1 -- 172.21.15.123:0/318981607 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb418210e90 con 0x7fb41820cc50 2024-04-25T12:35:38.093 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:38.094 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb4180c5a70 msgr2=0x7fb418157260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb4180c5a70 0x7fb418157260 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb4180c5fc0 msgr2=0x7fb4181e50f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb4180c5fc0 0x7fb4181e50f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb4181f6270 msgr2=0x7fb4181f6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb4181f6270 0x7fb4181f6680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb418062fe0 msgr2=0x7fb4180633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.093+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb418062fe0 0x7fb4180633f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb41802cee0 msgr2=0x7fb41802d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb41802cee0 0x7fb41802d2f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb418206f70 msgr2=0x7fb418209390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fb418206f70 0x7fb418209390 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb418209de0 msgr2=0x7fb41820c200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb418209de0 0x7fb41820c200 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb41820cc50 msgr2=0x7fb41820f0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb41820cc50 0x7fb41820f0d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 reap_dead start 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 reap_dead start 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 reap_dead start 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 reap_dead start 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb416800640 1 -- 172.21.15.123:0/318981607 reap_dead start 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3f80744e0 msgr2=0x7fb3f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3f80744e0 0x7fb3f80769f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4080011e0 tx=0x7fb408009040 comp rx=0 tx=0).stop 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4181426a0 msgr2=0x7fb418141110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4181426a0 0x7fb418141110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb410009eb0 tx=0x7fb410004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:38.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 shutdown_connections 2024-04-25T12:35:38.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3f80744e0 0x7fb3f80769f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 --2- 172.21.15.123:0/318981607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4181426a0 0x7fb418141110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 >> 172.21.15.123:0/318981607 conn(0x7fb41806f440 msgr2=0x7fb418070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:38.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.094+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 shutdown_connections 2024-04-25T12:35:38.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.095+0000 7fb41cc00640 1 -- 172.21.15.123:0/318981607 wait complete. 2024-04-25T12:35:38.100 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:38.100+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:38.109 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:38.109 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:38.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:38.201 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:38.202+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:38.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.204+0000 7f3ce7200640 1 Processor -- start 2024-04-25T12:35:38.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.204+0000 7f3ce7200640 1 -- start start 2024-04-25T12:35:38.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce0073460 0x7f3ce015d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3ce7200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ce00b2a10 0x7f3ce00c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3ce7200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce0073c10 con 0x7f3ce00c3c80 2024-04-25T12:35:38.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3ce7200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3ce015d7b0 con 0x7f3ce00b2a10 2024-04-25T12:35:38.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3ce7200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce0156fb0 con 0x7f3ce0073460 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3cdf400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ce00b2a10 0x7f3ce00c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3cdf400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ce00b2a10 0x7f3ce00c3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56400/0 (socket says 172.21.15.123:56400) 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3cdf400640 1 -- 172.21.15.123:0/931867531 learned_addr learned my addr 172.21.15.123:0/931867531 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3cdfe00640 1 --2- 172.21.15.123:0/931867531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce0073460 0x7f3ce015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.206+0000 7f3cdf400640 1 -- 172.21.15.123:0/931867531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce0073460 msgr2=0x7f3ce015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.207+0000 7f3cdf400640 1 --2- 172.21.15.123:0/931867531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce0073460 0x7f3ce015d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.207+0000 7f3cdf400640 1 -- 172.21.15.123:0/931867531 >> v1:172.21.15.153:6789/0 conn(0x7f3ce00c3c80 legacy=0x7f3ce00ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.207+0000 7f3cdf400640 1 -- 172.21.15.123:0/931867531 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce0074100 con 0x7f3ce00b2a10 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.207+0000 7f3cdea00640 1 -- 172.21.15.123:0/931867531 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49490012 0 0) 0x7f3ce0073c10 con 0x7f3ce00c3c80 2024-04-25T12:35:38.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.207+0000 7f3cdfe00640 1 --2- 172.21.15.123:0/931867531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce0073460 0x7f3ce015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:35:38.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.208+0000 7f3cdf400640 1 --2- 172.21.15.123:0/931867531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce00b2a10 0x7f3ce00c3740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009870 tx=0x7f3ccc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a64927f981477b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.208+0000 7f3cdea00640 1 -- 172.21.15.123:0/931867531 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc037070 con 0x7f3ce00b2a10 2024-04-25T12:35:38.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.208+0000 7f3cdea00640 1 -- 172.21.15.123:0/931867531 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ccc032440 con 0x7f3ce00b2a10 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.208+0000 7f3cdea00640 1 -- 172.21.15.123:0/931867531 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc032810 con 0x7f3ce00b2a10 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 -- 172.21.15.123:0/931867531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce00b2a10 msgr2=0x7f3ce00c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 --2- 172.21.15.123:0/931867531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce00b2a10 0x7f3ce00c3740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009870 tx=0x7f3ccc02a250 comp rx=0 tx=0).stop 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 -- 172.21.15.123:0/931867531 shutdown_connections 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 --2- 172.21.15.123:0/931867531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce00b2a10 0x7f3ce00c3740 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 --2- 172.21.15.123:0/931867531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce0073460 0x7f3ce015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 -- 172.21.15.123:0/931867531 >> 172.21.15.123:0/931867531 conn(0x7f3ce006f440 msgr2=0x7f3ce006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 -- 172.21.15.123:0/931867531 shutdown_connections 2024-04-25T12:35:38.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.209+0000 7f3ce7200640 1 -- 172.21.15.123:0/931867531 wait complete. 2024-04-25T12:35:38.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.211+0000 7f3ce7200640 1 Processor -- start 2024-04-25T12:35:38.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.211+0000 7f3ce7200640 1 -- start start 2024-04-25T12:35:38.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce0073460 0x7f3ce0159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce00b2a10 0x7f3ce0152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ce00c3c80 0x7f3ce01e9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3ce7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0070a20 con 0x7f3ce0073460 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3ce7200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0071310 con 0x7f3ce00c3c80 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3ce7200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce0073930 con 0x7f3ce00b2a10 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3cdfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce0073460 0x7f3ce0159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3cdfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce0073460 0x7f3ce0159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56410/0 (socket says 172.21.15.123:56410) 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3cdfe00640 1 -- 172.21.15.123:0/2537200231 learned_addr learned my addr 172.21.15.123:0/2537200231 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3ce5200640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ce00c3c80 0x7f3ce01e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.213+0000 7f3cdf400640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce00b2a10 0x7f3ce0152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cdfe00640 1 -- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce00b2a10 msgr2=0x7f3ce0152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cdfe00640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ce00b2a10 0x7f3ce0152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cdfe00640 1 -- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ce00c3c80 msgr2=0x7f3ce01e9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cdfe00640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ce00c3c80 0x7f3ce01e9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cdfe00640 1 -- 172.21.15.123:0/2537200231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce015eb30 con 0x7f3ce0073460 2024-04-25T12:35:38.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cdfe00640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce0073460 0x7f3ce0159d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c820 tx=0x7f3cd000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0015070 con 0x7f3ce0073460 2024-04-25T12:35:38.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd00079d0 con 0x7f3ce0073460 2024-04-25T12:35:38.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce015e0e0 con 0x7f3ce0073460 2024-04-25T12:35:38.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0007d80 con 0x7f3ce0073460 2024-04-25T12:35:38.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.214+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce01599f0 con 0x7f3ce0073460 2024-04-25T12:35:38.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.215+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ce0076700 con 0x7f3ce0073460 2024-04-25T12:35:38.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.217+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3cd0018070 con 0x7f3ce0073460 2024-04-25T12:35:38.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.218+0000 7f3cd7e00640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3cc00744e0 0x7f3cc00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.218+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7f3cd0095260 con 0x7f3ce0073460 2024-04-25T12:35:38.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.218+0000 7f3cd7e00640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f3cc007dd40 0x7f3cc0080230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.218+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cc0080900 con 0x7f3cc007dd40 2024-04-25T12:35:38.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.218+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3cd009c050 con 0x7f3ce0073460 2024-04-25T12:35:38.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.218+0000 7f3cdf400640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3cc00744e0 0x7f3cc00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.219+0000 7f3cdf400640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3cc00744e0 0x7f3cc00769f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc034000 tx=0x7f3ccc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.220+0000 7f3ce5200640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f3cc007dd40 0x7f3cc0080230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.221+0000 7f3ce5200640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f3cc007dd40 0x7f3cc0080230 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.223+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cc0080900 con 0x7f3cc007dd40 2024-04-25T12:35:38.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.241+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f3ce00748c0 con 0x7f3cc007dd40 2024-04-25T12:35:38.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.242+0000 7f3cd7e00640 1 -- 172.21.15.123:0/2537200231 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f3ce00748c0 con 0x7f3cc007dd40 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.242+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f3cc007dd40 msgr2=0x7f3cc0080230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.242+0000 7f3ce7200640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f3cc007dd40 0x7f3cc0080230 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.242+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3cc00744e0 msgr2=0x7f3cc00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.242+0000 7f3ce7200640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3cc00744e0 0x7f3cc00769f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc034000 tx=0x7f3ccc02e000 comp rx=0 tx=0).stop 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.243+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce0073460 msgr2=0x7f3ce0159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.243+0000 7f3ce7200640 1 --2- 172.21.15.123:0/2537200231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce0073460 0x7f3ce0159d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c820 tx=0x7f3cd000cce0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.243+0000 7f3ce5200640 1 -- 172.21.15.123:0/2537200231 reap_dead start 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.243+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 shutdown_connections 2024-04-25T12:35:38.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.243+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 >> 172.21.15.123:0/2537200231 conn(0x7f3ce006f440 msgr2=0x7f3ce0155070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:38.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.243+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 shutdown_connections 2024-04-25T12:35:38.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.243+0000 7f3ce7200640 1 -- 172.21.15.123:0/2537200231 wait complete. 2024-04-25T12:35:38.269 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-04-25T12:35:38.302 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:38.302+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:38.403 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:38.403+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:38.423 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:38.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:38.504 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:38.504+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:38.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.576+0000 7f49e6000640 1 Processor -- start 2024-04-25T12:35:38.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.577+0000 7f49e6000640 1 -- start start 2024-04-25T12:35:38.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.579+0000 7f49e6000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f49e006d440 0x7f49e0156670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.579+0000 7f49e6000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f49e00c72c0 0x7f49e00c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.580+0000 7f49e6000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e006d850 con 0x7f49e01606e0 2024-04-25T12:35:38.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.580+0000 7f49e6000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f49e0160af0 con 0x7f49e00c72c0 2024-04-25T12:35:38.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.580+0000 7f49e6000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f49e015fc40 con 0x7f49e006d440 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.580+0000 7f49df400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f49e01606e0 0x7f49e015d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60426/0 (socket says 172.21.15.123:60426) 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.580+0000 7f49df400640 1 -- 172.21.15.123:0/601380041 learned_addr learned my addr 172.21.15.123:0/601380041 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.580+0000 7f49dea00640 1 --2- 172.21.15.123:0/601380041 >> v2:172.21.15.153:3300/0 conn(0x7f49e006d440 0x7f49e0156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.580+0000 7f49dfe00640 1 --2- 172.21.15.123:0/601380041 >> v2:172.21.15.123:3300/0 conn(0x7f49e00c72c0 0x7f49e00c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.581+0000 7f49dea00640 1 -- 172.21.15.123:0/601380041 >> v2:172.21.15.123:3300/0 conn(0x7f49e00c72c0 msgr2=0x7f49e00c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.581+0000 7f49dea00640 1 --2- 172.21.15.123:0/601380041 >> v2:172.21.15.123:3300/0 conn(0x7f49e00c72c0 0x7f49e00c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.581+0000 7f49dea00640 1 -- 172.21.15.123:0/601380041 >> v1:172.21.15.153:6789/0 conn(0x7f49e01606e0 legacy=0x7f49e015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.581+0000 7f49de000640 1 -- 172.21.15.123:0/601380041 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425857311 0 0) 0x7f49d0001110 con 0x7f49e01606e0 2024-04-25T12:35:38.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.581+0000 7f49dea00640 1 -- 172.21.15.123:0/601380041 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e00cc920 con 0x7f49e006d440 2024-04-25T12:35:38.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.582+0000 7f49dea00640 1 --2- 172.21.15.123:0/601380041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e0156670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f49e008ed70 tx=0x7f49c802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e7c2ddabb80480 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.582+0000 7f49de000640 1 -- 172.21.15.123:0/601380041 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8007740 con 0x7f49e006d440 2024-04-25T12:35:38.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.582+0000 7f49de000640 1 -- 172.21.15.123:0/601380041 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c80078e0 con 0x7f49e006d440 2024-04-25T12:35:38.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.582+0000 7f49de000640 1 -- 172.21.15.123:0/601380041 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8007cb0 con 0x7f49e006d440 2024-04-25T12:35:38.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 -- 172.21.15.123:0/601380041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 msgr2=0x7f49e0156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 --2- 172.21.15.123:0/601380041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e0156670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f49e008ed70 tx=0x7f49c802a710 comp rx=0 tx=0).stop 2024-04-25T12:35:38.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 -- 172.21.15.123:0/601380041 shutdown_connections 2024-04-25T12:35:38.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 --2- 172.21.15.123:0/601380041 >> v2:172.21.15.123:3300/0 conn(0x7f49e00c72c0 0x7f49e00c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 --2- 172.21.15.123:0/601380041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e0156670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 -- 172.21.15.123:0/601380041 >> 172.21.15.123:0/601380041 conn(0x7f49e0068a50 msgr2=0x7f49e00d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:38.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 -- 172.21.15.123:0/601380041 shutdown_connections 2024-04-25T12:35:38.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.583+0000 7f49e6000640 1 -- 172.21.15.123:0/601380041 wait complete. 2024-04-25T12:35:38.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.586+0000 7f49e6000640 1 Processor -- start 2024-04-25T12:35:38.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.586+0000 7f49e6000640 1 -- start start 2024-04-25T12:35:38.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.587+0000 7f49e6000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.587+0000 7f49e6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49e00c72c0 0x7f49e01e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.587+0000 7f49e6000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49e01606e0 0x7f49e00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.587+0000 7f49e6000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01601c0 con 0x7f49e00c72c0 2024-04-25T12:35:38.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.587+0000 7f49e6000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49e015fee0 con 0x7f49e006d440 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.587+0000 7f49e6000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f49e00cc620 con 0x7f49e01606e0 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.587+0000 7f49df400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33446/0 (socket says 172.21.15.123:33446) 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49e01606e0 0x7f49e00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 -- 172.21.15.123:0/3905409699 learned_addr learned my addr 172.21.15.123:0/3905409699 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49dea00640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49e00c72c0 0x7f49e01e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 -- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49e01606e0 msgr2=0x7f49e00cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49e01606e0 0x7f49e00cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 -- 172.21.15.123:0/3905409699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49e00c72c0 msgr2=0x7f49e01e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49e00c72c0 0x7f49e01e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 -- 172.21.15.123:0/3905409699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e015c8f0 con 0x7f49e006d440 2024-04-25T12:35:38.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.588+0000 7f49df400640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e015a600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c490 tx=0x7f49d000c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.589+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0018070 con 0x7f49e006d440 2024-04-25T12:35:38.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.589+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d0007500 con 0x7f49e006d440 2024-04-25T12:35:38.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.589+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e015ab40 con 0x7f49e006d440 2024-04-25T12:35:38.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.589+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0004a20 con 0x7f49e006d440 2024-04-25T12:35:38.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.589+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e01f2a20 con 0x7f49e006d440 2024-04-25T12:35:38.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.590+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49e01562b0 con 0x7f49e006d440 2024-04-25T12:35:38.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.592+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49d001b070 con 0x7f49e006d440 2024-04-25T12:35:38.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.593+0000 7f49d7400640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f49bc0744b0 0x7f49bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.593+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7f49d0094750 con 0x7f49e006d440 2024-04-25T12:35:38.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.594+0000 7f49dea00640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f49bc0744b0 0x7f49bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.594+0000 7f49d7400640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f49bc07dd10 0x7f49bc080200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.594+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49bc0808d0 con 0x7f49bc07dd10 2024-04-25T12:35:38.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.594+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f49d009b050 con 0x7f49e006d440 2024-04-25T12:35:38.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.594+0000 7f49dfe00640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f49bc07dd10 0x7f49bc080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.594+0000 7f49dea00640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f49bc0744b0 0x7f49bc0769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034000 tx=0x7f49c802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.594+0000 7f49dfe00640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f49bc07dd10 0x7f49bc080200 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.595+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49bc0808d0 con 0x7f49bc07dd10 2024-04-25T12:35:38.604 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:38.605+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:38.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.614+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f49e0076700 con 0x7f49bc07dd10 2024-04-25T12:35:38.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.614+0000 7f49d7400640 1 -- 172.21.15.123:0/3905409699 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f49e0076700 con 0x7f49bc07dd10 2024-04-25T12:35:38.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f49bc07dd10 msgr2=0x7f49bc080200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7f49bc07dd10 0x7f49bc080200 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f49bc0744b0 msgr2=0x7f49bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f49bc0744b0 0x7f49bc0769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034000 tx=0x7f49c802f000 comp rx=0 tx=0).stop 2024-04-25T12:35:38.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 msgr2=0x7f49e015a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 --2- 172.21.15.123:0/3905409699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e006d440 0x7f49e015a600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c490 tx=0x7f49d000c950 comp rx=0 tx=0).stop 2024-04-25T12:35:38.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49dfe00640 1 -- 172.21.15.123:0/3905409699 reap_dead start 2024-04-25T12:35:38.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 shutdown_connections 2024-04-25T12:35:38.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 >> 172.21.15.123:0/3905409699 conn(0x7f49e0068a50 msgr2=0x7f49e00b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:38.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 shutdown_connections 2024-04-25T12:35:38.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.615+0000 7f49e6000640 1 -- 172.21.15.123:0/3905409699 wait complete. 2024-04-25T12:35:38.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-04-25T12:35:38.707 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:38.707 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:38.708 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:38.708+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:38.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:38.818 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:38.818+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:38.908 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:38.908+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:38.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.943+0000 7fa179000640 1 Processor -- start 2024-04-25T12:35:38.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.943+0000 7fa179000640 1 -- start start 2024-04-25T12:35:38.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.946+0000 7fa179000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa1740c8170 0x7fa1740ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.946+0000 7fa179000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa174074400 con 0x7fa1740c8fd0 2024-04-25T12:35:38.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.946+0000 7fa179000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1741595e0 con 0x7fa1741426a0 2024-04-25T12:35:38.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.946+0000 7fa179000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa174074670 con 0x7fa1740c8170 2024-04-25T12:35:38.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.946+0000 7fa171e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa1740c8fd0 0x7fa174153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48096/0 (socket says 172.21.15.123:48096) 2024-04-25T12:35:38.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.946+0000 7fa171e00640 1 -- 172.21.15.123:0/3115469540 learned_addr learned my addr 172.21.15.123:0/3115469540 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:38.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.946+0000 7fa172800640 1 --2- 172.21.15.123:0/3115469540 >> v2:172.21.15.153:3300/0 conn(0x7fa1740c8170 0x7fa1740ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.947+0000 7fa171400640 1 -- 172.21.15.123:0/3115469540 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142203681 0 0) 0x7fa174074400 con 0x7fa1740c8fd0 2024-04-25T12:35:38.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.947+0000 7fa171400640 1 -- 172.21.15.123:0/3115469540 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa158005730 con 0x7fa1740c8fd0 2024-04-25T12:35:38.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.947+0000 7fa172800640 1 -- 172.21.15.123:0/3115469540 >> v1:172.21.15.153:6789/0 conn(0x7fa1741426a0 legacy=0x7fa174159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.947+0000 7fa172800640 1 -- 172.21.15.123:0/3115469540 >> v1:172.21.15.123:6789/0 conn(0x7fa1740c8fd0 legacy=0x7fa174153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.947+0000 7fa172800640 1 -- 172.21.15.123:0/3115469540 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa174074950 con 0x7fa1740c8170 2024-04-25T12:35:38.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.948+0000 7fa172800640 1 --2- 172.21.15.123:0/3115469540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1740c8170 0x7fa1740ca200 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009d90 tx=0x7fa16c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1fad8facfb56b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.948+0000 7fa171400640 1 -- 172.21.15.123:0/3115469540 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c007740 con 0x7fa1740c8170 2024-04-25T12:35:38.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.948+0000 7fa171400640 1 -- 172.21.15.123:0/3115469540 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c0078e0 con 0x7fa1740c8170 2024-04-25T12:35:38.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.948+0000 7fa171400640 1 -- 172.21.15.123:0/3115469540 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c007cb0 con 0x7fa1740c8170 2024-04-25T12:35:38.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.949+0000 7fa179000640 1 -- 172.21.15.123:0/3115469540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1740c8170 msgr2=0x7fa1740ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.949+0000 7fa179000640 1 --2- 172.21.15.123:0/3115469540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1740c8170 0x7fa1740ca200 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009d90 tx=0x7fa16c02a6f0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.949+0000 7fa179000640 1 -- 172.21.15.123:0/3115469540 shutdown_connections 2024-04-25T12:35:38.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.949+0000 7fa179000640 1 --2- 172.21.15.123:0/3115469540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1740c8170 0x7fa1740ca200 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.949+0000 7fa179000640 1 -- 172.21.15.123:0/3115469540 >> 172.21.15.123:0/3115469540 conn(0x7fa17406f440 msgr2=0x7fa17406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:38.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.949+0000 7fa179000640 1 -- 172.21.15.123:0/3115469540 shutdown_connections 2024-04-25T12:35:38.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.949+0000 7fa179000640 1 -- 172.21.15.123:0/3115469540 wait complete. 2024-04-25T12:35:38.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.951+0000 7fa179000640 1 Processor -- start 2024-04-25T12:35:38.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.951+0000 7fa179000640 1 -- start start 2024-04-25T12:35:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa179000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1740c8170 0x7fa1741e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa179000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1740c8fd0 0x7fa1740c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa179000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1741426a0 0x7fa1740c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa179000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740c8580 con 0x7fa1740c8170 2024-04-25T12:35:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa179000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740c4880 con 0x7fa1741426a0 2024-04-25T12:35:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa179000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa1740c8700 con 0x7fa1740c8fd0 2024-04-25T12:35:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa171e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1740c8170 0x7fa1741e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa171e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1740c8170 0x7fa1741e59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56438/0 (socket says 172.21.15.123:56438) 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa171e00640 1 -- 172.21.15.123:0/105873775 learned_addr learned my addr 172.21.15.123:0/105873775 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa172800640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1741426a0 0x7fa1740c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.953+0000 7fa16be00640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1740c8fd0 0x7fa1740c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa171e00640 1 -- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1740c8fd0 msgr2=0x7fa1740c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa171e00640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1740c8fd0 0x7fa1740c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa171e00640 1 -- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1741426a0 msgr2=0x7fa1740c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa171e00640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1741426a0 0x7fa1740c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa171e00640 1 -- 172.21.15.123:0/105873775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa174140d50 con 0x7fa1740c8170 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa172800640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1741426a0 0x7fa1740c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa171e00640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1740c8170 0x7fa1741e59f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c440 tx=0x7fa16400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164017070 con 0x7fa1740c8170 2024-04-25T12:35:38.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1741fa3d0 con 0x7fa1740c8170 2024-04-25T12:35:38.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.954+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa164005ce0 con 0x7fa1740c8170 2024-04-25T12:35:38.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.955+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1640072b0 con 0x7fa1740c8170 2024-04-25T12:35:38.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.955+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1740c73d0 con 0x7fa1740c8170 2024-04-25T12:35:38.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.956+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1741faa90 con 0x7fa1740c8170 2024-04-25T12:35:38.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.957+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa16401a070 con 0x7fa1740c8170 2024-04-25T12:35:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.958+0000 7fa16a000640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1500744e0 0x7fa1500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.958+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7fa164094d50 con 0x7fa1740c8170 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.958+0000 7fa16be00640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1500744e0 0x7fa1500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.958+0000 7fa16a000640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa15007dcc0 0x7fa1500801b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.958+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa150080880 con 0x7fa15007dcc0 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.958+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa164095100 con 0x7fa1740c8170 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.959+0000 7fa172800640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa15007dcc0 0x7fa1500801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.959+0000 7fa16be00640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1500744e0 0x7fa1500769f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1740c7d10 tx=0x7fa15c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.959+0000 7fa172800640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa15007dcc0 0x7fa1500801b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:38.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.960+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa150080880 con 0x7fa15007dcc0 2024-04-25T12:35:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.979+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fa17406d1e0 con 0x7fa15007dcc0 2024-04-25T12:35:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa16a000640 1 -- 172.21.15.123:0/105873775 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fa17406d1e0 con 0x7fa15007dcc0 2024-04-25T12:35:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa15007dcc0 msgr2=0x7fa1500801b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa15007dcc0 0x7fa1500801b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1500744e0 msgr2=0x7fa1500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1500744e0 0x7fa1500769f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1740c7d10 tx=0x7fa15c009040 comp rx=0 tx=0).stop 2024-04-25T12:35:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1740c8170 msgr2=0x7fa1741e59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 --2- 172.21.15.123:0/105873775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1740c8170 0x7fa1741e59f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c440 tx=0x7fa16400c900 comp rx=0 tx=0).stop 2024-04-25T12:35:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa172800640 1 -- 172.21.15.123:0/105873775 reap_dead start 2024-04-25T12:35:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 shutdown_connections 2024-04-25T12:35:38.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.980+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 >> 172.21.15.123:0/105873775 conn(0x7fa17406f440 msgr2=0x7fa174075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:38.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.981+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 shutdown_connections 2024-04-25T12:35:38.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:38.981+0000 7fa179000640 1 -- 172.21.15.123:0/105873775 wait complete. 2024-04-25T12:35:39.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-04-25T12:35:39.008 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:39.009+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:39.021 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:39.021 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:39.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:39.109 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:39.110+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:39.210 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:39.210+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:39.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.275+0000 7f68cda00640 1 Processor -- start 2024-04-25T12:35:39.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.276+0000 7f68cda00640 1 -- start start 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68cda00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68cda00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f68c80c8170 0x7f68c80ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68cda00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c8074400 con 0x7f68c80c8fd0 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68cda00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f68c81595e0 con 0x7f68c80c8170 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68cda00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f68c8074670 con 0x7f68c81426a0 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68c7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f68c80c8170 0x7f68c80ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68c6a00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f68c80c8fd0 0x7f68c8153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60448/0 (socket says 172.21.15.123:60448) 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68c6a00640 1 -- 172.21.15.123:0/3966359963 learned_addr learned my addr 172.21.15.123:0/3966359963 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.278+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.279+0000 7f68c6000640 1 -- 172.21.15.123:0/3966359963 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974658172 0 0) 0x7f68c8074400 con 0x7f68c80c8fd0 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.279+0000 7f68c7400640 1 -- 172.21.15.123:0/3966359963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 msgr2=0x7f68c8159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.279+0000 7f68c7400640 1 --2- 172.21.15.123:0/3966359963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.279+0000 7f68c7400640 1 -- 172.21.15.123:0/3966359963 >> v1:172.21.15.153:6789/0 conn(0x7f68c80c8fd0 legacy=0x7f68c8153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.279+0000 7f68c7400640 1 -- 172.21.15.123:0/3966359963 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c8074950 con 0x7f68c80c8170 2024-04-25T12:35:39.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.279+0000 7f68bfe00640 1 --2- 172.21.15.123:0/3966359963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:39.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.280+0000 7f68c7400640 1 --2- 172.21.15.123:0/3966359963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8170 0x7f68c80ca200 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009d90 tx=0x7f68c002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55266609437c093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:39.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.280+0000 7f68c6000640 1 -- 172.21.15.123:0/3966359963 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0007740 con 0x7f68c80c8170 2024-04-25T12:35:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.280+0000 7f68c6000640 1 -- 172.21.15.123:0/3966359963 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c00078e0 con 0x7f68c80c8170 2024-04-25T12:35:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.280+0000 7f68c6000640 1 -- 172.21.15.123:0/3966359963 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0007cb0 con 0x7f68c80c8170 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 -- 172.21.15.123:0/3966359963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8170 msgr2=0x7f68c80ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 --2- 172.21.15.123:0/3966359963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8170 0x7f68c80ca200 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009d90 tx=0x7f68c002a660 comp rx=0 tx=0).stop 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 -- 172.21.15.123:0/3966359963 shutdown_connections 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 --2- 172.21.15.123:0/3966359963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8170 0x7f68c80ca200 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 --2- 172.21.15.123:0/3966359963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 -- 172.21.15.123:0/3966359963 >> 172.21.15.123:0/3966359963 conn(0x7f68c806f440 msgr2=0x7f68c806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 -- 172.21.15.123:0/3966359963 shutdown_connections 2024-04-25T12:35:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.281+0000 7f68cda00640 1 -- 172.21.15.123:0/3966359963 wait complete. 2024-04-25T12:35:39.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.284+0000 7f68cda00640 1 Processor -- start 2024-04-25T12:35:39.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.284+0000 7f68cda00640 1 -- start start 2024-04-25T12:35:39.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.285+0000 7f68cda00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68c80c8170 0x7f68c81f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.285+0000 7f68cda00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8fd0 0x7f68c81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.285+0000 7f68cda00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.285+0000 7f68cda00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80c4880 con 0x7f68c80c8fd0 2024-04-25T12:35:39.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.285+0000 7f68cda00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68c815a300 con 0x7f68c80c8170 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.285+0000 7f68cda00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f68c80c8580 con 0x7f68c81426a0 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.285+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8fd0 0x7f68c81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8fd0 0x7f68c81527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56464/0 (socket says 172.21.15.123:56464) 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 -- 172.21.15.123:0/1014549724 learned_addr learned my addr 172.21.15.123:0/1014549724 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68c6a00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68c80c8170 0x7f68c81f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68c7400640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 -- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 msgr2=0x7f68c8157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68c81426a0 0x7f68c8157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 -- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68c80c8170 msgr2=0x7f68c81f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68c80c8170 0x7f68c81f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 -- 172.21.15.123:0/1014549724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c81f3110 con 0x7f68c80c8fd0 2024-04-25T12:35:39.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bfe00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8fd0 0x7f68c81527e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f68b000e990 tx=0x7f68b000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:39.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.286+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0017070 con 0x7f68c80c8fd0 2024-04-25T12:35:39.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.287+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b000c9e0 con 0x7f68c80c8fd0 2024-04-25T12:35:39.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.287+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c81e5230 con 0x7f68c80c8fd0 2024-04-25T12:35:39.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.287+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b000cdb0 con 0x7f68c80c8fd0 2024-04-25T12:35:39.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.287+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81e3a20 con 0x7f68c80c8fd0 2024-04-25T12:35:39.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.288+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68c8076700 con 0x7f68c80c8fd0 2024-04-25T12:35:39.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.289+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f68b001a070 con 0x7f68c80c8fd0 2024-04-25T12:35:39.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.290+0000 7f68bea00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68a40744b0 0x7f68a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.290+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7f68b0095490 con 0x7f68c80c8fd0 2024-04-25T12:35:39.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.290+0000 7f68bea00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f68a407dd10 0x7f68a4080200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.290+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68a40808d0 con 0x7f68a407dd10 2024-04-25T12:35:39.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.290+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f68b009c050 con 0x7f68c80c8fd0 2024-04-25T12:35:39.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.290+0000 7f68c6a00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68a40744b0 0x7f68a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.291+0000 7f68c7400640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f68a407dd10 0x7f68a4080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.291+0000 7f68c6a00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68a40744b0 0x7f68a40769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68b8007fd0 tx=0x7f68b8007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:39.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.291+0000 7f68c7400640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f68a407dd10 0x7f68a4080200 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:39.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.292+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68a40808d0 con 0x7f68a407dd10 2024-04-25T12:35:39.311 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:39.311+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:39.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.329+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f68c813c1c0 con 0x7f68a407dd10 2024-04-25T12:35:39.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.330+0000 7f68bea00640 1 -- 172.21.15.123:0/1014549724 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f68c813c1c0 con 0x7f68a407dd10 2024-04-25T12:35:39.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.331+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f68a407dd10 msgr2=0x7f68a4080200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.331+0000 7f68cda00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f68a407dd10 0x7f68a4080200 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68a40744b0 msgr2=0x7f68a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68a40744b0 0x7f68a40769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68b8007fd0 tx=0x7f68b8007de0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8fd0 msgr2=0x7f68c81527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 --2- 172.21.15.123:0/1014549724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c80c8fd0 0x7f68c81527e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f68b000e990 tx=0x7f68b000ee50 comp rx=0 tx=0).stop 2024-04-25T12:35:39.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68c7400640 1 -- 172.21.15.123:0/1014549724 reap_dead start 2024-04-25T12:35:39.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 shutdown_connections 2024-04-25T12:35:39.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 >> 172.21.15.123:0/1014549724 conn(0x7f68c806f440 msgr2=0x7f68c80711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:39.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 shutdown_connections 2024-04-25T12:35:39.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.332+0000 7f68cda00640 1 -- 172.21.15.123:0/1014549724 wait complete. 2024-04-25T12:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:39.391 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:39.412 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:39.413+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:39.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:35:39.513 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:39.514+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:39.615 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:39.614+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:39.715 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:39.715+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:39.719 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:39.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.735+0000 7f3619e00640 1 Processor -- start 2024-04-25T12:35:39.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.735+0000 7f3619e00640 1 -- start start 2024-04-25T12:35:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.737+0000 7f3619e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3614074870 0x7f3614154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.737+0000 7f3619e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f361415afa0 0x7f36140cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.737+0000 7f3619e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361415b3b0 con 0x7f36140c4d50 2024-04-25T12:35:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.737+0000 7f3619e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f36140c61f0 con 0x7f361415afa0 2024-04-25T12:35:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.737+0000 7f3619e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f36140c87d0 con 0x7f3614074870 2024-04-25T12:35:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.737+0000 7f3613400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f361415afa0 0x7f36140cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3612000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3614074870 0x7f3614154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3613400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f361415afa0 0x7f36140cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56478/0 (socket says 172.21.15.123:56478) 2024-04-25T12:35:39.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3613400640 1 -- 172.21.15.123:0/3572765710 learned_addr learned my addr 172.21.15.123:0/3572765710 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:39.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3612a00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f36140c4d50 0x7f36140c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60458/0 (socket says 172.21.15.123:60458) 2024-04-25T12:35:39.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3613400640 1 -- 172.21.15.123:0/3572765710 >> v2:172.21.15.153:3300/0 conn(0x7f3614074870 msgr2=0x7f3614154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:39.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3613400640 1 --2- 172.21.15.123:0/3572765710 >> v2:172.21.15.153:3300/0 conn(0x7f3614074870 0x7f3614154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3613400640 1 -- 172.21.15.123:0/3572765710 >> v1:172.21.15.153:6789/0 conn(0x7f36140c4d50 legacy=0x7f36140c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.738+0000 7f3613400640 1 -- 172.21.15.123:0/3572765710 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3614073ce0 con 0x7f361415afa0 2024-04-25T12:35:39.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.739+0000 7f3613400640 1 --2- 172.21.15.123:0/3572765710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361415afa0 0x7f36140cda90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3614072a70 tx=0x7f360c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3de318791955cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:39.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.739+0000 7f3611600640 1 -- 172.21.15.123:0/3572765710 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c036070 con 0x7f361415afa0 2024-04-25T12:35:39.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.739+0000 7f3611600640 1 -- 172.21.15.123:0/3572765710 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c031600 con 0x7f361415afa0 2024-04-25T12:35:39.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.739+0000 7f3611600640 1 -- 172.21.15.123:0/3572765710 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c0319f0 con 0x7f361415afa0 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 -- 172.21.15.123:0/3572765710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361415afa0 msgr2=0x7f36140cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 --2- 172.21.15.123:0/3572765710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361415afa0 0x7f36140cda90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3614072a70 tx=0x7f360c02a7c0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 -- 172.21.15.123:0/3572765710 shutdown_connections 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 --2- 172.21.15.123:0/3572765710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361415afa0 0x7f36140cda90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 --2- 172.21.15.123:0/3572765710 >> v2:172.21.15.153:3300/0 conn(0x7f3614074870 0x7f3614154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 -- 172.21.15.123:0/3572765710 >> 172.21.15.123:0/3572765710 conn(0x7f361406c750 msgr2=0x7f361406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 -- 172.21.15.123:0/3572765710 shutdown_connections 2024-04-25T12:35:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.740+0000 7f3619e00640 1 -- 172.21.15.123:0/3572765710 wait complete. 2024-04-25T12:35:39.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.742+0000 7f3619e00640 1 Processor -- start 2024-04-25T12:35:39.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.743+0000 7f3619e00640 1 -- start start 2024-04-25T12:35:39.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3619e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3614074870 0x7f36140c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3619e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36140c4d50 0x7f361414c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3619e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f361415afa0 0x7f36140ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3619e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36140c8460 con 0x7f36140c4d50 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3619e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36140742a0 con 0x7f3614074870 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3619e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f36140c9370 con 0x7f361415afa0 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3612000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36140c4d50 0x7f361414c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3612a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3614074870 0x7f36140c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3612a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3614074870 0x7f36140c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:33500/0 (socket says 172.21.15.123:33500) 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3613400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f361415afa0 0x7f36140ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3612a00640 1 -- 172.21.15.123:0/4076535394 learned_addr learned my addr 172.21.15.123:0/4076535394 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:39.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.744+0000 7f3612000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36140c4d50 0x7f361414c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56484/0 (socket says 172.21.15.123:56484) 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3612a00640 1 -- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f361415afa0 msgr2=0x7f36140ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3612a00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f361415afa0 0x7f36140ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3612a00640 1 -- 172.21.15.123:0/4076535394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36140c4d50 msgr2=0x7f361414c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3612a00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36140c4d50 0x7f361414c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3612a00640 1 -- 172.21.15.123:0/4076535394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36140c4410 con 0x7f3614074870 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3612000640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36140c4d50 0x7f361414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3613400640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f361415afa0 0x7f36140ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f3612a00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3614074870 0x7f36140c3ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f360400c3c0 tx=0x7f360400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.745+0000 7f360b400640 1 -- 172.21.15.123:0/4076535394 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040074e0 con 0x7f3614074870 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.746+0000 7f360b400640 1 -- 172.21.15.123:0/4076535394 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604007680 con 0x7f3614074870 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.746+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36141594f0 con 0x7f3614074870 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.746+0000 7f360b400640 1 -- 172.21.15.123:0/4076535394 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604004a00 con 0x7f3614074870 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.746+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36140c33f0 con 0x7f3614074870 2024-04-25T12:35:39.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.747+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3614076700 con 0x7f3614074870 2024-04-25T12:35:39.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.748+0000 7f360b400640 1 -- 172.21.15.123:0/4076535394 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f360401a020 con 0x7f3614074870 2024-04-25T12:35:39.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.749+0000 7f360b400640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f35f40744e0 0x7f35f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:39.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.749+0000 7f360b400640 1 -- 172.21.15.123:0/4076535394 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7f3604094e90 con 0x7f3614074870 2024-04-25T12:35:39.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.749+0000 7f3612000640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f35f40744e0 0x7f35f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:39.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.750+0000 7f3612000640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f35f40744e0 0x7f35f40769f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f361415a3c0 tx=0x7f3600009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:39.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.753+0000 7f360b400640 1 -- 172.21.15.123:0/4076535394 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f360405ec20 con 0x7f3614074870 2024-04-25T12:35:39.818 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:39.819+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:39.844 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:39.844 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:39.853 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:39.919 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:39.919+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:39.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:39.961+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f36140b3610 con 0x7f3614074870 2024-04-25T12:35:40.005 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:40.005 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:40.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:40.020 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:40.020+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:40.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:40.121 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:40.122+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:40.125 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:40.125 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:40.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:35:40.222 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:40.222+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:40.287 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:40.287 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:40.296 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:35:40.337 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:40.338+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:40.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.392+0000 7ff78e000640 1 Processor -- start 2024-04-25T12:35:40.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.392+0000 7ff78e000640 1 -- start start 2024-04-25T12:35:40.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.394+0000 7ff78e000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff788154350 0x7ff7880c6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.394+0000 7ff78e000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff7880c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.394+0000 7ff78e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff788073270 0x7ff78815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.394+0000 7ff78e000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff78815d780 con 0x7ff788073270 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.394+0000 7ff78e000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff7880b9d90 con 0x7ff788154350 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.394+0000 7ff78e000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff78815a410 con 0x7ff788072d60 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff7880c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff78cc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff788154350 0x7ff7880c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff7880c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:47138/0 (socket says 172.21.15.123:47138) 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 -- 172.21.15.123:0/4224723962 learned_addr learned my addr 172.21.15.123:0/4224723962 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff78d600640 1 --2- 172.21.15.123:0/4224723962 >> v2:172.21.15.123:3300/0 conn(0x7ff788073270 0x7ff78815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 -- 172.21.15.123:0/4224723962 >> v2:172.21.15.153:3300/0 conn(0x7ff788154350 msgr2=0x7ff7880c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 --2- 172.21.15.123:0/4224723962 >> v2:172.21.15.153:3300/0 conn(0x7ff788154350 0x7ff7880c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 -- 172.21.15.123:0/4224723962 >> v2:172.21.15.123:3300/0 conn(0x7ff788073270 msgr2=0x7ff78815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 --2- 172.21.15.123:0/4224723962 >> v2:172.21.15.123:3300/0 conn(0x7ff788073270 0x7ff78815dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff77fe00640 1 -- 172.21.15.123:0/4224723962 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78815a6d0 con 0x7ff788072d60 2024-04-25T12:35:40.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff78d600640 1 --2- 172.21.15.123:0/4224723962 >> v2:172.21.15.123:3300/0 conn(0x7ff788073270 0x7ff78815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:35:40.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.395+0000 7ff78cc00640 1 --2- 172.21.15.123:0/4224723962 >> v2:172.21.15.153:3300/0 conn(0x7ff788154350 0x7ff7880c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:40.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.396+0000 7ff77fe00640 1 --2- 172.21.15.123:0/4224723962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff7880c6ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff788073240 tx=0x7ff77002a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5d1001961798980 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.397+0000 7ff787e00640 1 -- 172.21.15.123:0/4224723962 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770037070 con 0x7ff788072d60 2024-04-25T12:35:40.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.397+0000 7ff787e00640 1 -- 172.21.15.123:0/4224723962 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff770032440 con 0x7ff788072d60 2024-04-25T12:35:40.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.397+0000 7ff787e00640 1 -- 172.21.15.123:0/4224723962 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770032810 con 0x7ff788072d60 2024-04-25T12:35:40.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.397+0000 7ff78e000640 1 -- 172.21.15.123:0/4224723962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 msgr2=0x7ff7880c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.397+0000 7ff78e000640 1 --2- 172.21.15.123:0/4224723962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff7880c6ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff788073240 tx=0x7ff77002a1d0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.398+0000 7ff78e000640 1 -- 172.21.15.123:0/4224723962 shutdown_connections 2024-04-25T12:35:40.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.398+0000 7ff78e000640 1 --2- 172.21.15.123:0/4224723962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff7880c6ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.398+0000 7ff78e000640 1 --2- 172.21.15.123:0/4224723962 >> v2:172.21.15.123:3300/0 conn(0x7ff788073270 0x7ff78815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.398+0000 7ff78e000640 1 --2- 172.21.15.123:0/4224723962 >> v2:172.21.15.153:3300/0 conn(0x7ff788154350 0x7ff7880c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.398+0000 7ff78e000640 1 -- 172.21.15.123:0/4224723962 >> 172.21.15.123:0/4224723962 conn(0x7ff78806cba0 msgr2=0x7ff7880cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:40.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.398+0000 7ff78e000640 1 -- 172.21.15.123:0/4224723962 shutdown_connections 2024-04-25T12:35:40.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.398+0000 7ff78e000640 1 -- 172.21.15.123:0/4224723962 wait complete. 2024-04-25T12:35:40.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.401+0000 7ff78e000640 1 Processor -- start 2024-04-25T12:35:40.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.401+0000 7ff78e000640 1 -- start start 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78e000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff78814b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff788073270 0x7ff788157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff788154350 0x7ff788152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff78815af70 con 0x7ff788073270 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78e000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff78815ac90 con 0x7ff788154350 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78e000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff78815a100 con 0x7ff788072d60 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff77fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff788073270 0x7ff788157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78cc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff78814b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78cc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff78814b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:47142/0 (socket says 172.21.15.123:47142) 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78cc00640 1 -- 172.21.15.123:0/864152066 learned_addr learned my addr 172.21.15.123:0/864152066 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.403+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff788154350 0x7ff788152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78d600640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 msgr2=0x7ff78814b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff78814b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78d600640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff788073270 msgr2=0x7ff788157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff788073270 0x7ff788157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78d600640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff788158490 con 0x7ff788154350 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff788072d60 0x7ff78814b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff788154350 0x7ff788152520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff780007d40 tx=0x7ff78000d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78001a070 con 0x7ff788154350 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff780015400 con 0x7ff788154350 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7800157f0 con 0x7ff788154350 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78814fac0 con 0x7ff788154350 2024-04-25T12:35:40.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.404+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7881f6ea0 con 0x7ff788154350 2024-04-25T12:35:40.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.405+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff788076da0 con 0x7ff788154350 2024-04-25T12:35:40.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.407+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff78001d070 con 0x7ff788154350 2024-04-25T12:35:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.407+0000 7ff785600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7640744b0 0x7ff7640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.408+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6619+0+0 (secure 0 0 0) 0x7ff7800956c0 con 0x7ff788154350 2024-04-25T12:35:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.408+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7640744b0 0x7ff7640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.408+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7ff780010070 con 0x7ff788154350 2024-04-25T12:35:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.408+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff7881586c0 0x7ff7881500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.408+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff788076860 con 0x7ff7881586c0 2024-04-25T12:35:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.408+0000 7ff77fe00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff7881586c0 0x7ff7881500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.408+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7640744b0 0x7ff7640769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7780095e0 tx=0x7ff778009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.409+0000 7ff77fe00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff7881586c0 0x7ff7881500e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.410+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff788076860 con 0x7ff7881586c0 2024-04-25T12:35:40.424 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:40.425+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:40.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.426+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff788070b20 con 0x7ff7881586c0 2024-04-25T12:35:40.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.428+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff788070b20 con 0x7ff7881586c0 2024-04-25T12:35:40.430 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.428+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff788150620 0x7ff788146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.428+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff788070840 con 0x7ff788150620 2024-04-25T12:35:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.429+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff788150620 0x7ff788146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.430+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff788150620 0x7ff788146140 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.431+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff788070840 con 0x7ff788150620 2024-04-25T12:35:40.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.447+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff788071590 con 0x7ff788150620 2024-04-25T12:35:40.449 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:40.449 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.450+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff788071590 con 0x7ff788150620 2024-04-25T12:35:40.450 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:40.450 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.450+0000 7f360b400640 1 -- 172.21.15.123:0/4076535394 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v38) ==== 97+0+0 (secure 0 0 0) 0x7f36040672f0 con 0x7f3614074870 2024-04-25T12:35:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.450+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff788148a70 0x7ff788148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.450+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff788158c40 con 0x7ff788148a70 2024-04-25T12:35:40.451 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:35:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.451+0000 7ff77fe00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff788148a70 0x7ff788148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.452+0000 7ff77fe00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff788148a70 0x7ff788148e80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.453+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff788158c40 con 0x7ff788148a70 2024-04-25T12:35:40.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.453+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f35f40744e0 msgr2=0x7f35f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.453+0000 7f3619e00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f35f40744e0 0x7f35f40769f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f361415a3c0 tx=0x7f3600009190 comp rx=0 tx=0).stop 2024-04-25T12:35:40.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3614074870 msgr2=0x7f36140c3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3614074870 0x7f36140c3ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f360400c3c0 tx=0x7f360400c880 comp rx=0 tx=0).stop 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 shutdown_connections 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f35f40744e0 0x7f35f40769f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f361415afa0 0x7f36140ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36140c4d50 0x7f361414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 --2- 172.21.15.123:0/4076535394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3614074870 0x7f36140c3ed0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 >> 172.21.15.123:0/4076535394 conn(0x7f361406c750 msgr2=0x7f361406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 shutdown_connections 2024-04-25T12:35:40.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.454+0000 7f3619e00640 1 -- 172.21.15.123:0/4076535394 wait complete. 2024-04-25T12:35:40.460 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:35:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.469+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff788152a60 con 0x7ff788148a70 2024-04-25T12:35:40.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.472+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff788152a60 con 0x7ff788148a70 2024-04-25T12:35:40.472 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.472+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff7880638b0 0x7ff7880cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.472+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff788152c70 con 0x7ff7880638b0 2024-04-25T12:35:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.473+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff7880638b0 0x7ff7880cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.473+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff7880638b0 0x7ff7880cbbb0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.474+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff788152c70 con 0x7ff7880638b0 2024-04-25T12:35:40.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.489+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff788152c70 con 0x7ff7880638b0 2024-04-25T12:35:40.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.491+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff788152c70 con 0x7ff7880638b0 2024-04-25T12:35:40.492 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.492+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff7880cc600 0x7ff78802c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.492+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff788152c70 con 0x7ff7880cc600 2024-04-25T12:35:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.492+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff7880cc600 0x7ff78802c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.493+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff7880cc600 0x7ff78802c9d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.494+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff788152c70 con 0x7ff7880cc600 2024-04-25T12:35:40.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.508+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff7880cca40 con 0x7ff7880cc600 2024-04-25T12:35:40.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.510+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff7880cca40 con 0x7ff7880cc600 2024-04-25T12:35:40.510 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:40.511 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:40.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.510+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff78802d420 0x7ff78802d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.510+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff78802ddf0 con 0x7ff78802d420 2024-04-25T12:35:40.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.511+0000 7ff77fe00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff78802d420 0x7ff78802d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.512+0000 7ff77fe00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff78802d420 0x7ff78802d830 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.513+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff78802ddf0 con 0x7ff78802d420 2024-04-25T12:35:40.526 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:40.526+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:40.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.527+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff7880008d0 con 0x7ff78802d420 2024-04-25T12:35:40.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.530+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff7880008d0 con 0x7ff78802d420 2024-04-25T12:35:40.530 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:40.530 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:40.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.530+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff788209940 0x7ff78820bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.530+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff78820ecc0 con 0x7ff788209940 2024-04-25T12:35:40.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.530+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff788209940 0x7ff78820bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.531+0000 7ff78d600640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff788209940 0x7ff78820bd60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.532+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff78820ecc0 con 0x7ff788209940 2024-04-25T12:35:40.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.546+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff78820d9e0 con 0x7ff788209940 2024-04-25T12:35:40.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.548+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff78820d9e0 con 0x7ff788209940 2024-04-25T12:35:40.548 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.548+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff78820e100 0x7ff78820e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.549+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff788212090 con 0x7ff78820e100 2024-04-25T12:35:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.549+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff78820e100 0x7ff78820e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.549+0000 7ff78cc00640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff78820e100 0x7ff78820e510 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:40.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.550+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff788212090 con 0x7ff78820e100 2024-04-25T12:35:40.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.565+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff788210f10 con 0x7ff78820e100 2024-04-25T12:35:40.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.567+0000 7ff785600640 1 -- 172.21.15.123:0/864152066 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff788210f10 con 0x7ff78820e100 2024-04-25T12:35:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:40.569 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:40.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff7881586c0 msgr2=0x7ff7881500e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff7881586c0 0x7ff7881500e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff788150620 msgr2=0x7ff788146140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7ff788150620 0x7ff788146140 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff788148a70 msgr2=0x7ff788148e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7ff788148a70 0x7ff788148e80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff7880638b0 msgr2=0x7ff7880cbbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7ff7880638b0 0x7ff7880cbbb0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff7880cc600 msgr2=0x7ff78802c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7ff7880cc600 0x7ff78802c9d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78d600640 1 -- 172.21.15.123:0/864152066 reap_dead start 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff78802d420 msgr2=0x7ff78802d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7ff78802d420 0x7ff78802d830 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78d600640 1 -- 172.21.15.123:0/864152066 reap_dead start 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff788209940 msgr2=0x7ff78820bd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7ff788209940 0x7ff78820bd60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff78820e100 msgr2=0x7ff78820e510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7ff78820e100 0x7ff78820e510 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.568+0000 7ff78d600640 1 -- 172.21.15.123:0/864152066 reap_dead start 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7640744b0 msgr2=0x7ff7640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7640744b0 0x7ff7640769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7780095e0 tx=0x7ff778009190 comp rx=0 tx=0).stop 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff788154350 msgr2=0x7ff788152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff788154350 0x7ff788152520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff780007d40 tx=0x7ff78000d4a0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 shutdown_connections 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7640744b0 0x7ff7640769c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 --2- 172.21.15.123:0/864152066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff788154350 0x7ff788152520 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 >> 172.21.15.123:0/864152066 conn(0x7ff78806cba0 msgr2=0x7ff7881512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 shutdown_connections 2024-04-25T12:35:40.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:40.569+0000 7ff78e000640 1 -- 172.21.15.123:0/864152066 wait complete. 2024-04-25T12:35:40.615 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:23 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'13, mlcod=16'8)", 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.023037054, 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039927010000000004, 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.616 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592465+0000", 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7399999999999998e-07 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592515+0000", 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9832e-05 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592524+0000", 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2429999999999994e-06 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592532+0000", 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.576e-06 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592580+0000", 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8093000000000002e-05 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.592597+0000", 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6688999999999999e-05 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.596424+0000", 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0038271770000000002 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.596457+0000", 2024-04-25T12:35:40.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.3617e-05 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:24 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'14, mlcod=16'9)", 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.021457686000005, 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052768190000000003, 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.620 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594044+0000", 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7599999999999997e-07 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594111+0000", 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6662999999999995e-05 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594121+0000", 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8949999999999993e-06 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594129+0000", 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2679999999999994e-06 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594173+0000", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3340999999999997e-05 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594188+0000", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5838999999999999e-05 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599274+0000", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0050853210000000003 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599321+0000", 2024-04-25T12:35:40.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7015999999999998e-05 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:25 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'15, mlcod=16'9)", 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:08.594612+0000", 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.020889217000004, 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047303079999999999, 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594612+0000", 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594612+0000", 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594613+0000", 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500000000000001e-07 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.625 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594618+0000", 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1089999999999997e-06 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594626+0000", 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8390000000000007e-06 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594634+0000", 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7510000000000008e-06 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.626 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594667+0000", 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.3776000000000003e-05 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.594681+0000", 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3949e-05 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599324+0000", 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0046431099999999998 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:08.599343+0000", 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8399000000000001e-05 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:65 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'40)", 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.251375+0000", 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 91.364126205999995, 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036189350000000002, 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.628 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251375+0000", 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251375+0000", 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251376+0000", 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7599999999999997e-07 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.629 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251383+0000", 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5719999999999999e-06 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251389+0000", 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0970000000000002e-06 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251394+0000", 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6760000000000004e-06 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251439+0000", 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4836999999999998e-05 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.251461+0000", 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1756e-05 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.254963+0000", 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003501905 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.631 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.254994+0000", 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1615999999999997e-05 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:79 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'57, mlcod=16'53)", 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.274232+0000", 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 91.341269632999996, 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003717354, 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274232+0000", 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274232+0000", 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274233+0000", 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.58e-07 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.633 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274322+0000", 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9189000000000003e-05 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274330+0000", 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1089999999999998e-06 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274337+0000", 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.255e-06 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274384+0000", 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6873999999999999e-05 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.274399+0000", 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4987999999999999e-05 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.277912+0000", 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035133930000000001 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.277949+0000", 2024-04-25T12:35:40.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7088000000000001e-05 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:80 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'58, mlcod=16'53)", 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 91.339539884000004, 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047846920000000001, 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.636 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275962+0000", 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4200000000000001e-07 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.275995+0000", 2024-04-25T12:35:40.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2805999999999997e-05 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276004+0000", 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.5960000000000004e-06 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276011+0000", 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3540000000000002e-06 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276056+0000", 2024-04-25T12:35:40.638 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4888000000000001e-05 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.276071+0000", 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5560999999999999e-05 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.280709+0000", 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0046372460000000002 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.280746+0000", 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7799000000000002e-05 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.639 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:90 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'64, mlcod=16'60)", 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 90.987772899000007, 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047246010000000001, 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:40.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627729+0000", 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3799999999999997e-07 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627853+0000", 2024-04-25T12:35:40.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012308499999999999 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627863+0000", 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0135999999999999e-05 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627869+0000", 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7190000000000004e-06 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627906+0000", 2024-04-25T12:35:40.642 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.65e-05 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.627919+0000", 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2955e-05 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632420+0000", 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004501672 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632453+0000", 2024-04-25T12:35:40.643 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2895999999999998e-05 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:91 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'65, mlcod=17'61)", 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 90.986572152999997, 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035550040000000001, 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.644 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628930+0000", 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2099999999999996e-07 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628938+0000", 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5490000000000001e-06 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628948+0000", 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7929999999999992e-06 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628955+0000", 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4760000000000001e-06 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.646 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.628997+0000", 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.193e-05 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.629011+0000", 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632455+0000", 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003444253 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.647 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.632485+0000", 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9059000000000001e-05 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:92 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'66, mlcod=17'61)", 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 90.985466885999998, 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0050398830000000002, 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630035+0000", 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2799999999999996e-07 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630042+0000", 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.861e-06 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630052+0000", 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630059+0000", 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.63e-06 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630093+0000", 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4567000000000003e-05 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630107+0000", 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.322e-05 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635043+0000", 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049359740000000001 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635075+0000", 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2076000000000003e-05 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:93 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'61)", 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 90.984880911999994, 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044888319999999999, 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.652 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630621+0000", 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2600000000000002e-07 2024-04-25T12:35:40.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630657+0000", 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5537000000000002e-05 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630667+0000", 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0107e-05 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.654 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630675+0000", 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8699999999999992e-06 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630718+0000", 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3050999999999998e-05 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.630732+0000", 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4154e-05 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.655 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635078+0000", 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043455569999999999 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635110+0000", 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1930000000000001e-05 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(client.4175.0:94 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'62)", 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:09.631743+0000", 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 90.983758209000001, 2024-04-25T12:35:40.656 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034012959999999998, 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631743+0000", 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631743+0000", 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.657 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631744+0000", 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631751+0000", 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7499999999999997e-06 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631761+0000", 2024-04-25T12:35:40.658 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8430000000000005e-06 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631767+0000", 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1820000000000003e-06 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631811+0000", 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3634999999999998e-05 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.631824+0000", 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3584000000000001e-05 2024-04-25T12:35:40.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635111+0000", 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0032869750000000001 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:09.635145+0000", 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.3707000000000001e-05 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.950975+0000", 2024-04-25T12:35:40.660 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.664526296000005, 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033063559999999999, 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950975+0000", 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950975+0000", 2024-04-25T12:35:40.661 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950976+0000", 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.850000000000001e-07 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4749999999999999e-06 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951002+0000", 2024-04-25T12:35:40.662 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6501e-05 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951014+0000", 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1234e-05 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951081+0000", 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6964000000000005e-05 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7339999999999998e-05 2024-04-25T12:35:40.663 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954232+0000", 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031145080000000002 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954282+0000", 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9348999999999998e-05 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:30 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:40.664 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953314+0000", 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.662188125, 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010402056, 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953314+0000", 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953314+0000", 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.665 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953328+0000", 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4114000000000001e-05 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953341+0000", 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3162e-05 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953356+0000", 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5218000000000001e-05 2024-04-25T12:35:40.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953368+0000", 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2150999999999999e-05 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953488+0000", 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000119907 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953638+0000", 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015016300000000001 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.667 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953791+0000", 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015301900000000001 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955457+0000", 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001665435 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.963628+0000", 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081709910000000007 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.668 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.963667+0000", 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9196999999999998e-05 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.963716+0000", 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954270+0000", 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.661231736000005, 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0045854889999999999, 2024-04-25T12:35:40.669 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954270+0000", 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954270+0000", 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.670 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954271+0000", 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.085e-06 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954287+0000", 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6257999999999999e-05 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954298+0000", 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0616e-05 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.671 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954308+0000", 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0139999999999999e-05 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954381+0000", 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2992999999999997e-05 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954501+0000", 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000120127 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.672 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958855+0000", 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043542700000000004 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.660907390000006, 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043590520000000004, 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.673 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954595+0000", 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0440000000000001e-06 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954601+0000", 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.818e-06 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954654+0000", 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2880000000000002e-05 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954668+0000", 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4136e-05 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954732+0000", 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3540999999999994e-05 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954883+0000", 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015099199999999999 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958953+0000", 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040706409999999998 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.676 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115422+0000", 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 50.500079943000003, 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004970716, 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115422+0000", 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115422+0000", 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115424+0000", 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2400000000000002e-06 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115434+0000", 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0338e-05 2024-04-25T12:35:40.678 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115453+0000", 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9040999999999999e-05 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115468+0000", 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4187e-05 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115555+0000", 2024-04-25T12:35:40.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7397000000000004e-05 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115703+0000", 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001484 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 6", 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115862+0000", 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015840099999999999 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119676+0000", 2024-04-25T12:35:40.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0038141529999999998 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120295+0000", 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00061928999999999999 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120336+0000", 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1304000000000003e-05 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120392+0000", 2024-04-25T12:35:40.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5964999999999997e-05 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1 epoch 25/18 rep_tid 7 entries 25'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116283+0000", 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 50.499218472000003, 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003323505, 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116283+0000", 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116283+0000", 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116285+0000", 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.77e-06 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116300+0000", 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5498999999999998e-05 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116328+0000", 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7024000000000001e-05 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116367+0000", 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9910000000000002e-05 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116599+0000", 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000231757 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119607+0000", 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030075449999999999 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117004+0000", 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 50.498498054000002, 2024-04-25T12:35:40.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039571210000000001, 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117004+0000", 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117004+0000", 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117005+0000", 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.372e-06 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117011+0000", 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9689999999999999e-06 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117022+0000", 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1521e-05 2024-04-25T12:35:40.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117035+0000", 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.216e-05 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117101+0000", 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6558999999999998e-05 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117253+0000", 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000151736 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120961+0000", 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037078039999999999 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117569+0000", 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 50.497933179999997, 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035307899999999998, 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.689 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117569+0000", 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117569+0000", 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.9800000000000002e-07 2024-04-25T12:35:40.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117577+0000", 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0330000000000003e-06 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117587+0000", 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0767e-05 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117598+0000", 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1174e-05 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117677+0000", 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8745000000000006e-05 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117852+0000", 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017459600000000001 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121099+0000", 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0032474769999999999 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.692 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152648+0000", 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.462853795000001, 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047121940000000003, 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152648+0000", 2024-04-25T12:35:40.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152648+0000", 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152650+0000", 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.075e-06 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152658+0000", 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5850000000000002e-06 2024-04-25T12:35:40.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152674+0000", 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6121999999999999e-05 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152686+0000", 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2446000000000001e-05 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152822+0000", 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135771 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152972+0000", 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014999499999999999 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157360+0000", 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043882000000000001 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:40.696 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:40.697 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:40.698 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:40.698+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:40.798 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:40.799+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:40.851 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:40.851 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:40.851 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:40.851 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:40.860 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:40.901 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:40.902+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:41.000 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:41.000+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:41.013 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:41.013 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:41.013 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:41.023 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:41.100 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:41.101+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 24/18 rep_tid 1 entries 24'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.181169280999995, 2024-04-25T12:35:41.136 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075768199999999997, 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.137 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954840+0000", 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6869999999999999e-06 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954864+0000", 2024-04-25T12:35:41.138 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3824000000000001e-05 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954902+0000", 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500999999999999e-05 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954919+0000", 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7618e-05 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.139 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955011+0000", 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2250000000000006e-05 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955596+0000", 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058480200000000004 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.140 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:25 2.1b e25/18 2:dbae76b5:::mds1_inotable:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.019484198999997, 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036130300000000001, 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116525+0000", 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.455e-06 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.142 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116539+0000", 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4e-05 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116563+0000", 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4111999999999999e-05 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116577+0000", 2024-04-25T12:35:41.143 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4059999999999999e-05 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116689+0000", 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011209000000000001 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116719+0000", 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.989e-05 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.144 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120082+0000", 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033628130000000001 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120137+0000", 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4610000000000001e-05 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.145 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 4 entries 25'3 (0'0) error 2:db457a8a:::201.00000001:head by mds.1.9:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.018427658, 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030622480000000001, 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.146 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117581+0000", 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0279999999999999e-06 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.147 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117596+0000", 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5109999999999999e-05 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117631+0000", 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4230000000000003e-05 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117655+0000", 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4987e-05 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.148 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117751+0000", 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5311999999999998e-05 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120642+0000", 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028915809999999998 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.149 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 5 entries 25'4 (0'0) error 2:d80f8093:::201.00000002:head by mds.1.9:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.017926426999999, 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0025905469999999999, 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.150 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118082+0000", 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4000000000000001e-07 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.151 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118091+0000", 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9919999999999996e-06 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118108+0000", 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7207999999999999e-05 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.152 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118124+0000", 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5378000000000001e-05 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118203+0000", 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8929000000000003e-05 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.153 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120672+0000", 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024692999999999998 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:34 2.1b e25/18 2:db457a8a:::201.00000001:head v 25'5, mlcod=25'3)", 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:41.154 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.014452093000003, 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016516759999999999, 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.155 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121557+0000", 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.423e-06 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121568+0000", 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0499000000000001e-05 2024-04-25T12:35:41.156 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121585+0000", 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7283e-05 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121597+0000", 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1821000000000001e-05 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.157 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121653+0000", 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6533000000000002e-05 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121671+0000", 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7285000000000001e-05 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.158 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123158+0000", 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0014869519999999999 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123207+0000", 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9879999999999997e-05 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.159 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:35 2.a 2:54ba6923:::mds1_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.992339776000001, 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016436459999999999, 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.160 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.161 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143670+0000", 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4040000000000002e-06 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.162 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143685+0000", 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4768e-05 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143703+0000", 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7424000000000001e-05 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.163 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143721+0000", 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8685000000000002e-05 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143824+0000", 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010286800000000001 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.164 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144032+0000", 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020812700000000001 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144205+0000", 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017276999999999999 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144982+0000", 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077719300000000002 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.165 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145196+0000", 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000213308 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145254+0000", 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8619000000000002e-05 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145312+0000", 2024-04-25T12:35:41.166 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7479999999999999e-05 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.1.9:35 2.a e25/18 ondisk, result = 0)", 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.990913208999999, 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00022686700000000001, 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145096+0000", 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2309999999999999e-06 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145102+0000", 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1639999999999998e-06 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145115+0000", 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2809000000000001e-05 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145124+0000", 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9989999999999997e-06 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145178+0000", 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4287999999999999e-05 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145189+0000", 2024-04-25T12:35:41.170 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.084e-05 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145321+0000", 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013253599999999999 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:11 2.c 2:36924187:::618.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.171 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.983663943000003, 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070050249999999998, 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.172 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152345+0000", 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6270000000000001e-06 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152355+0000", 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4069999999999999e-06 2024-04-25T12:35:41.173 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152384+0000", 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9479e-05 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152399+0000", 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.174 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152478+0000", 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9702000000000004e-05 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152615+0000", 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013665500000000001 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152730+0000", 2024-04-25T12:35:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011504499999999999 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156394+0000", 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036643259999999999 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028370299999999999 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.176 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159289+0000", 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7268999999999997e-05 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159349+0000", 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0161999999999998e-05 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.177 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.983577369000002, 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070895239999999998, 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.178 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152431+0000", 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6899999999999996e-07 2024-04-25T12:35:41.179 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152440+0000", 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3259999999999996e-06 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152450+0000", 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0874e-05 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.180 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152470+0000", 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9658999999999999e-05 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153265+0000", 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079454099999999995 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153349+0000", 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4381000000000004e-05 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.181 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153485+0000", 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136365 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156400+0000", 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029144990000000001 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159418+0000", 2024-04-25T12:35:41.182 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030182299999999998 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159477+0000", 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159520+0000", 2024-04-25T12:35:41.183 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3127000000000002e-05 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.983513754999997, 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0068584939999999997, 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.184 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:41.185 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152495+0000", 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.16e-07 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152501+0000", 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7819999999999999e-06 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.186 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152510+0000", 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9199999999999993e-06 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152518+0000", 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9090000000000006e-06 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.187 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152651+0000", 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000132349 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152795+0000", 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014392800000000001 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.188 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152930+0000", 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013561399999999999 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156365+0000", 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034348830000000001 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159266+0000", 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029006959999999999 2024-04-25T12:35:41.189 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159296+0000", 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0026e-05 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159353+0000", 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6570999999999998e-05 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.190 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.983427792000001, 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069239999999999996, 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.191 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152581+0000", 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3399999999999998e-07 2024-04-25T12:35:41.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152587+0000", 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4049999999999997e-06 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152598+0000", 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1043000000000001e-05 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152607+0000", 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9409999999999996e-06 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153448+0000", 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00084054699999999997 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153563+0000", 2024-04-25T12:35:41.194 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000115584 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153719+0000", 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015566199999999999 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156373+0000", 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002653617 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159423+0000", 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030508060000000001 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.196 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159472+0000", 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9042000000000002e-05 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159504+0000", 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1519000000000001e-05 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.197 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.983344909000003, 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0086667419999999998, 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.198 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152664+0000", 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0100000000000003e-07 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152671+0000", 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8870000000000003e-06 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.200 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152679+0000", 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2730000000000002e-06 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152696+0000", 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7373000000000001e-05 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153747+0000", 2024-04-25T12:35:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001050647 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153967+0000", 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00021982699999999999 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154122+0000", 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015472399999999999 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.202 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156405+0000", 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022838110000000002 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161248+0000", 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048425669999999999 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161283+0000", 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5166000000000002e-05 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161330+0000", 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6566000000000001e-05 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.204 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.2 epoch 25/18 rep_tid 20 entries 25'1 (0'0) error 2:40dbbc66:::202.00000003:head by mds.2.13:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.981739028, 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0021562719999999999, 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.205 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154270+0000", 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.638e-06 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154290+0000", 2024-04-25T12:35:41.206 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392e-05 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154315+0000", 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4858000000000002e-05 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154335+0000", 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0044e-05 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154711+0000", 2024-04-25T12:35:41.207 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037622099999999999 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156425+0000", 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017141190000000001 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:11 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:41.208 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.976883133999998, 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00023418899999999999, 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:41.209 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159126+0000", 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7699999999999992e-07 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159131+0000", 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8109999999999999e-06 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.210 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159143+0000", 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1745e-05 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159153+0000", 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6520000000000001e-06 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159211+0000", 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8582999999999998e-05 2024-04-25T12:35:41.211 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159225+0000", 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3088000000000001e-05 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159359+0000", 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013433299999999999 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.212 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:15 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.976832737999999, 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000189507, 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.213 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159176+0000", 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6899999999999998e-07 2024-04-25T12:35:41.214 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159180+0000", 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1320000000000004e-06 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159188+0000", 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9319999999999996e-06 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159199+0000", 2024-04-25T12:35:41.215 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1165999999999999e-05 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5071000000000002e-05 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159262+0000", 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0160000000000008e-06 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.216 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159365+0000", 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010272099999999999 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:12 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.976793807999996, 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000315354, 2024-04-25T12:35:41.217 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2099999999999998e-07 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159217+0000", 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.875e-06 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.219 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159223+0000", 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2999999999999998e-06 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159229+0000", 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6740000000000002e-06 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159401+0000", 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017201599999999999 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.220 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159412+0000", 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0349e-05 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159529+0000", 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000117819 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:16 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.976765538999999, 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00026996799999999999, 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.222 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159243+0000", 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0100000000000001e-07 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.223 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159246+0000", 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.687e-06 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.072e-06 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159261+0000", 2024-04-25T12:35:41.224 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2169999999999999e-06 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159408+0000", 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000147637 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159417+0000", 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999996e-06 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.225 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159512+0000", 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5163999999999999e-05 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.226 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:23 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 45.974851383999997, 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000182072, 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.227 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.228 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161157+0000", 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7400000000000002e-07 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161164+0000", 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:41.229 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161180+0000", 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6189999999999999e-05 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161191+0000", 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0981e-05 2024-04-25T12:35:41.230 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161238+0000", 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7509999999999997e-05 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161244+0000", 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.694e-06 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.231 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161338+0000", 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4432999999999995e-05 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.232 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:41.234 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:41.234 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:41.234+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:41.334 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:41.334+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:41.386 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:41.386 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:41.386 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:41.386 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:41.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:41.435 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:41.435+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:41.537 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:41.536+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:41.548 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:41.548 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:41.548 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:41.548 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:41.549 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:41.557 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:41.637 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:41.637+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:41.672 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:41.672 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.720848981000003, 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.673 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:35:41.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.677 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.720761327000005, 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.678 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:35:41.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:35:41.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:35:41.682 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.720416759000003, 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.720344288999996, 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:35:41.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:35:41.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:35:41.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.720270310999993, 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:35:41.693 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.697 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:35:41.698 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.718232939000004, 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:41.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:35:41.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:35:41.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.702 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 82.717372803000004, 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.703 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:35:41.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.556687148000002, 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.707 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:35:41.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:35:41.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:35:41.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.712 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.556611033999999, 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.713 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.716 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.717 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:41.718 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.556565364999997, 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.719 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.720 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.721 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:35:41.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.723 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.556520589999998, 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.724 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.725 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.726 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.727 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.728 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.549904165999997, 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.729 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.730 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.731 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.732 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.549844675999999, 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.733 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.734 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.735 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:35:41.736 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.737 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 46.518874384, 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.738 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:35:41.739 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:35:41.740 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:35:41.741 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.742 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 46.518796872999999, 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.743 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:41.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:41.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:35:41.748 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 46.518749024000002, 2024-04-25T12:35:41.749 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:35:41.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:35:41.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:35:41.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.754 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 46.518707134000003, 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.759 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 46.518672961999997, 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:41.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:41.764 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 46.518390220000001, 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:41.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:35:41.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 46.514249788999997, 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:41.771 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:41.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:41.773 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:41.773+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:41.874 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:41.874+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:41.975 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:41.975+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:42.057 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:42.057 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:42.057 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:42.076 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:42.076+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:42.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:42.176 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:42.177+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:42.277 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:42.278+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:42.378 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:42.378+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:42.398 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:42.398 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:42.398 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:42.398 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:42.398 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:42.421 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:42.479 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:42.479+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:42.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:35:42.580 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:42.580+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:42.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:42.683 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:42.683+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:42.735 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.783118039000001, 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012628781, 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951638+0000", 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951659+0000", 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1027999999999999e-05 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951691+0000", 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2267000000000001e-05 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951710+0000", 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9063999999999999e-05 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952026+0000", 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000315719 2024-04-25T12:35:42.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952367+0000", 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034072600000000002 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955799+0000", 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034325969999999999 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962852+0000", 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070528889999999997 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964076+0000", 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012234119999999999 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964178+0000", 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102166 2024-04-25T12:35:42.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964264+0000", 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6175999999999996e-05 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:42.742 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.782980207999998, 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012746265999999999, 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951774+0000", 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-06 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951782+0000", 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2090000000000005e-06 2024-04-25T12:35:42.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951796+0000", 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3559e-05 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951808+0000", 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2183e-05 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956591+0000", 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047823550000000003 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956696+0000", 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105696 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956907+0000", 2024-04-25T12:35:42.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000211165 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962907+0000", 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059995650000000001 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964283+0000", 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013758189999999999 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964381+0000", 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7977000000000001e-05 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964519+0000", 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138675 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:31 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.781811484000002, 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076541370000000001, 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952943+0000", 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952999+0000", 2024-04-25T12:35:42.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5968999999999999e-05 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953032+0000", 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2598999999999997e-05 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953058+0000", 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6585999999999999e-05 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953207+0000", 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014848300000000001 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953263+0000", 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6005999999999999e-05 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960434+0000", 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071716219999999999 2024-04-25T12:35:42.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960596+0000", 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016173899999999999 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:42.753 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.781629565000003, 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010940785, 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:35:42.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953125+0000", 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.22e-06 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953138+0000", 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2996000000000001e-05 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953158+0000", 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0173000000000002e-05 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953176+0000", 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7927e-05 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954316+0000", 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011402599999999999 2024-04-25T12:35:42.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954375+0000", 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963968+0000", 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095929269999999994 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964065+0000", 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6628999999999999e-05 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.781530740999997, 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080627230000000008, 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.758 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953225+0000", 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3640000000000002e-06 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953259+0000", 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3955999999999998e-05 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953292+0000", 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3549e-05 2024-04-25T12:35:42.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953320+0000", 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7892e-05 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953495+0000", 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017444800000000001 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953705+0000", 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021011000000000001 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953862+0000", 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015660399999999999 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959861+0000", 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059993959999999997 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961071+0000", 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012101900000000001 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961192+0000", 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121279 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961285+0000", 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2935000000000004e-05 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.781354003999994, 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010652488999999999, 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.764 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953401+0000", 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.513e-06 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953412+0000", 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209000000000001e-05 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953473+0000", 2024-04-25T12:35:42.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0668999999999998e-05 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953496+0000", 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3246000000000001e-05 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956356+0000", 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00286002 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956620+0000", 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026411099999999998 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.957005+0000", 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038473499999999998 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962757+0000", 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057517080000000003 2024-04-25T12:35:42.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963927+0000", 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00117023 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964002+0000", 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5483999999999998e-05 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964052+0000", 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9564000000000002e-05 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.781225554000002, 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011827087999999999, 2024-04-25T12:35:42.770 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953529+0000", 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953594+0000", 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4807999999999993e-05 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953617+0000", 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3220000000000001e-05 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953683+0000", 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5328999999999997e-05 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954191+0000", 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000508428 2024-04-25T12:35:42.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040280099999999999 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954922+0000", 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032838300000000002 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960432+0000", 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055099989999999998 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965226+0000", 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047938700000000004 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965297+0000", 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0992000000000001e-05 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965355+0000", 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7695000000000002e-05 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:19 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:42.776 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.618321754, 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010949354999999999, 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116434+0000", 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116475+0000", 2024-04-25T12:35:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1486000000000002e-05 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116505+0000", 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9709e-05 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116529+0000", 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3839999999999999e-05 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116651+0000", 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122013 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116685+0000", 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4513e-05 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127273+0000", 2024-04-25T12:35:42.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010587653000000001 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127381+0000", 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108204 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:20 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.618166627000001, 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014061374, 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116588+0000", 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0499999999999999e-06 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116602+0000", 2024-04-25T12:35:42.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3793e-05 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116621+0000", 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9545999999999999e-05 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116637+0000", 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5696e-05 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117243+0000", 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060668400000000002 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117280+0000", 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6619000000000003e-05 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130559+0000", 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013278570999999999 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130648+0000", 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9414999999999997e-05 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:27 2.9 e25/18 2:91bd965c:::501.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:42.786 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.617739976999999, 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013688128000000001, 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117015+0000", 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3200000000000001e-06 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117121+0000", 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106451 2024-04-25T12:35:42.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117145+0000", 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.404e-05 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117210+0000", 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4314999999999997e-05 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120564+0000", 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033548050000000002 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120624+0000", 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9234999999999998e-05 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130654+0000", 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010030437999999999 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130702+0000", 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7524000000000001e-05 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:18 2.13 e25/18 2:ca4dd7f5:::611.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.617386678000003, 2024-04-25T12:35:42.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010076254, 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117369+0000", 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.88e-06 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117385+0000", 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6670000000000001e-05 2024-04-25T12:35:42.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117413+0000", 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7691999999999998e-05 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117435+0000", 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2416e-05 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117549+0000", 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113436 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1509e-05 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127347+0000", 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097767759999999992 2024-04-25T12:35:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127443+0000", 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5874999999999999e-05 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:42.796 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.616175005000002, 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012295627, 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118582+0000", 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000001e-06 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118599+0000", 2024-04-25T12:35:42.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7176e-05 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2505999999999997e-05 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118659+0000", 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7559000000000002e-05 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120040+0000", 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001380956 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120305+0000", 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026497700000000001 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120707+0000", 2024-04-25T12:35:42.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040189499999999999 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128035+0000", 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073284049999999996 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130711+0000", 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026752910000000002 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130787+0000", 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6384999999999995e-05 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130874+0000", 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7020000000000004e-05 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.616019004999998, 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012319366, 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:42.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118747+0000", 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1286e-05 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118772+0000", 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5150000000000001e-05 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118793+0000", 2024-04-25T12:35:42.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0778999999999999e-05 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121911+0000", 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031179409999999999 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122331+0000", 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041955999999999997 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124441+0000", 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021107629999999999 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130205+0000", 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057636140000000002 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130888+0000", 2024-04-25T12:35:42.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068289100000000003 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130961+0000", 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3026000000000002e-05 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131054+0000", 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2855999999999993e-05 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.808 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:21 2.8 2:1470b2f8:::613.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.615930276999997, 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012409474, 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118824+0000", 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:35:42.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118834+0000", 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118855+0000", 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999999e-05 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118875+0000", 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9809999999999998e-05 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119612+0000", 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073729299999999998 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119912+0000", 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029931000000000001 2024-04-25T12:35:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120202+0000", 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000289907 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130342+0000", 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010140188 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131091+0000", 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074893900000000001 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131166+0000", 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5272999999999996e-05 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131233+0000", 2024-04-25T12:35:42.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6625000000000007e-05 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.615821529999998, 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012992702, 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.815 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118933+0000", 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3069999999999999e-06 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118943+0000", 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7200000000000001e-06 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118967+0000", 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3828999999999998e-05 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118986+0000", 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9057000000000001e-05 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124788+0000", 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00580271 2024-04-25T12:35:42.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125092+0000", 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030394999999999999 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125479+0000", 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038660000000000002 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131299+0000", 2024-04-25T12:35:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058200559999999997 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131823+0000", 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052382700000000004 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131888+0000", 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4740999999999996e-05 2024-04-25T12:35:42.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131925+0000", 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6905e-05 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:5 2.7 e25/18 2:e455cf45:::60b.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.615698082000002, 2024-04-25T12:35:42.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011712752, 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119057+0000", 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.296e-06 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119068+0000", 2024-04-25T12:35:42.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.189e-05 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119091+0000", 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2802999999999998e-05 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119110+0000", 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8473000000000001e-05 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121085+0000", 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019750689999999999 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121149+0000", 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3979000000000005e-05 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130709+0000", 2024-04-25T12:35:42.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009559817 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130768+0000", 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9425000000000001e-05 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.826 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:21 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.579908793999998, 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066623919999999996, 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154846+0000", 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.06e-06 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154856+0000", 2024-04-25T12:35:42.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0713000000000001e-05 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154878+0000", 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1180000000000001e-05 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154896+0000", 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8343e-05 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155332+0000", 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043663000000000003 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155389+0000", 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6818999999999999e-05 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161381+0000", 2024-04-25T12:35:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059914440000000003 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161507+0000", 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126203 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:22 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:42.831 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.579635023000002, 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064678330000000001, 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155120+0000", 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155131+0000", 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1793e-05 2024-04-25T12:35:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155155+0000", 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3306e-05 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155174+0000", 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9383000000000001e-05 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157101+0000", 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019271570000000001 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157161+0000", 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9945000000000003e-05 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161512+0000", 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043506389999999999 2024-04-25T12:35:42.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161586+0000", 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4250000000000002e-05 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:26 2.11 2:893cd530:::mds2_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.57869805, 2024-04-25T12:35:42.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064127289999999998, 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:35:42.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156057+0000", 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8330000000000001e-06 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156090+0000", 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2719000000000003e-05 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156116+0000", 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6531999999999999e-05 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156140+0000", 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3453999999999999e-05 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156350+0000", 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021060999999999999 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156467+0000", 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116823 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156594+0000", 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012698900000000001 2024-04-25T12:35:42.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161255+0000", 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046609570000000003 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162360+0000", 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001105167 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162416+0000", 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6076999999999999e-05 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162468+0000", 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1567999999999998e-05 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.578547401000002, 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066007829999999998, 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:42.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156208+0000", 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5570000000000001e-06 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156216+0000", 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0900000000000005e-06 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156237+0000", 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1452999999999999e-05 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:42.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156255+0000", 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8091999999999999e-05 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156841+0000", 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00058573200000000005 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156983+0000", 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000141676 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:42.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162807+0000", 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058241830000000001 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:42.847 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:42.849 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.767+0000 7f8456a00640 1 Processor -- start 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.767+0000 7f8456a00640 1 -- start start 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8456a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8450156890 0x7f845015fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8456a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84501602c0 0x7f84500cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8456a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845015c790 con 0x7f845006d680 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8456a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f84500c6410 con 0x7f84501602c0 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8456a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f845015d380 con 0x7f8450156890 2024-04-25T12:35:42.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8454a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84501602c0 0x7f84500cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8454a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84501602c0 0x7f84500cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56508/0 (socket says 172.21.15.123:56508) 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8454a00640 1 -- 172.21.15.123:0/3393867542 learned_addr learned my addr 172.21.15.123:0/3393867542 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f844fe00640 1 --1- 172.21.15.123:0/3393867542 >> v1:172.21.15.123:6789/0 conn(0x7f845006d680 0x7f84500c5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48102/0 (socket says 172.21.15.123:48102) 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.769+0000 7f8447e00640 1 --2- 172.21.15.123:0/3393867542 >> v2:172.21.15.153:3300/0 conn(0x7f8450156890 0x7f845015fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.770+0000 7f8454a00640 1 -- 172.21.15.123:0/3393867542 >> v2:172.21.15.153:3300/0 conn(0x7f8450156890 msgr2=0x7f845015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.770+0000 7f8454a00640 1 --2- 172.21.15.123:0/3393867542 >> v2:172.21.15.153:3300/0 conn(0x7f8450156890 0x7f845015fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.770+0000 7f8454a00640 1 -- 172.21.15.123:0/3393867542 >> v1:172.21.15.123:6789/0 conn(0x7f845006d680 legacy=0x7f84500c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.770+0000 7f8454a00640 1 -- 172.21.15.123:0/3393867542 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845008d840 con 0x7f84501602c0 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.770+0000 7f844f400640 1 -- 172.21.15.123:0/3393867542 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2945693451 0 0) 0x7f845015c790 con 0x7f845006d680 2024-04-25T12:35:42.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.771+0000 7f8454a00640 1 --2- 172.21.15.123:0/3393867542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500cfc80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f845015d500 tx=0x7f844802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4175ee1446991c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.771+0000 7f844f400640 1 -- 172.21.15.123:0/3393867542 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448036070 con 0x7f84501602c0 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.771+0000 7f844f400640 1 -- 172.21.15.123:0/3393867542 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8448031600 con 0x7f84501602c0 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.771+0000 7f844f400640 1 -- 172.21.15.123:0/3393867542 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480319f0 con 0x7f84501602c0 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.772+0000 7f8456a00640 1 -- 172.21.15.123:0/3393867542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 msgr2=0x7f84500cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.772+0000 7f8456a00640 1 --2- 172.21.15.123:0/3393867542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500cfc80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f845015d500 tx=0x7f844802a7c0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.772+0000 7f8456a00640 1 -- 172.21.15.123:0/3393867542 shutdown_connections 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.772+0000 7f8456a00640 1 --2- 172.21.15.123:0/3393867542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500cfc80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.772+0000 7f8456a00640 1 --2- 172.21.15.123:0/3393867542 >> v2:172.21.15.153:3300/0 conn(0x7f8450156890 0x7f845015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.772+0000 7f8456a00640 1 -- 172.21.15.123:0/3393867542 >> 172.21.15.123:0/3393867542 conn(0x7f8450072ec0 msgr2=0x7f84500732d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.772+0000 7f8456a00640 1 -- 172.21.15.123:0/3393867542 shutdown_connections 2024-04-25T12:35:42.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.773+0000 7f8456a00640 1 -- 172.21.15.123:0/3393867542 wait complete. 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.775+0000 7f8456a00640 1 Processor -- start 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.775+0000 7f8456a00640 1 -- start start 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.776+0000 7f8456a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f845006d680 0x7f84500b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.776+0000 7f8456a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8450156890 0x7f84501e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.776+0000 7f8456a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.776+0000 7f8456a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f845015ca50 con 0x7f84501602c0 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.776+0000 7f8456a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8450156ca0 con 0x7f8450156890 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.776+0000 7f8456a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f845015cd30 con 0x7f845006d680 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56514/0 (socket says 172.21.15.123:56514) 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f844fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f845006d680 0x7f84500b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 -- 172.21.15.123:0/1010370246 learned_addr learned my addr 172.21.15.123:0/1010370246 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8447e00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8450156890 0x7f84501e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 -- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f845006d680 msgr2=0x7f84500b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f845006d680 0x7f84500b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 -- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8450156890 msgr2=0x7f84501e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8450156890 0x7f84501e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8454a00640 1 -- 172.21.15.123:0/1010370246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84501e74c0 con 0x7f84501602c0 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f844fe00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f845006d680 0x7f84500b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.777+0000 7f8447e00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8450156890 0x7f84501e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.778+0000 7f8454a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500b2960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f844802adb0 tx=0x7f8448004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.778+0000 7f844cc00640 1 -- 172.21.15.123:0/1010370246 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448036050 con 0x7f84501602c0 2024-04-25T12:35:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.778+0000 7f844cc00640 1 -- 172.21.15.123:0/1010370246 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8448042070 con 0x7f84501602c0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.778+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84501f2990 con 0x7f84501602c0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.778+0000 7f844cc00640 1 -- 172.21.15.123:0/1010370246 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448009510 con 0x7f84501602c0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.778+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84501f2f50 con 0x7f84501602c0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.779+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84501e7780 con 0x7f84501602c0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.781+0000 7f844cc00640 1 -- 172.21.15.123:0/1010370246 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8448004470 con 0x7f84501602c0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.782+0000 7f844cc00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f842c0744b0 0x7f842c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.782+0000 7f844cc00640 1 -- 172.21.15.123:0/1010370246 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6619+0+0 (secure 0 0 0) 0x7f84480b6210 con 0x7f84501602c0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.782+0000 7f844fe00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f842c0744b0 0x7f842c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.783+0000 7f844fe00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f842c0744b0 0x7f842c0769c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8440007fd0 tx=0x7f8440007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.787+0000 7f844cc00640 1 -- 172.21.15.123:0/1010370246 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8448080040 con 0x7f84501602c0 2024-04-25T12:35:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.841+0000 7fdf00c00640 1 Processor -- start 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.841+0000 7fdf00c00640 1 -- start start 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdf00c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc0c9f00 0x7fdefc0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdf00c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdefc0730e0 0x7fdefc0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdf00c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdefc074b20 con 0x7fdefc154650 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdf00c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc0ca780 con 0x7fdefc0730e0 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdf00c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc0caa60 con 0x7fdefc0c9f00 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdef9e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fdefc154650 0x7fdefc15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:33944/0 (socket says 172.21.15.123:33944) 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdef9e00640 1 -- 172.21.15.123:0/2981062187 learned_addr learned my addr 172.21.15.123:0/2981062187 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.843+0000 7fdef9400640 1 --2- 172.21.15.123:0/2981062187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc0c9f00 0x7fdefc0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.844+0000 7fdefa800640 1 --2- 172.21.15.123:0/2981062187 >> v2:172.21.15.153:3300/0 conn(0x7fdefc0730e0 0x7fdefc0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.844+0000 7fdef9400640 1 -- 172.21.15.123:0/2981062187 >> v2:172.21.15.153:3300/0 conn(0x7fdefc0730e0 msgr2=0x7fdefc0734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.844+0000 7fdef9400640 1 --2- 172.21.15.123:0/2981062187 >> v2:172.21.15.153:3300/0 conn(0x7fdefc0730e0 0x7fdefc0734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.844+0000 7fdef9400640 1 -- 172.21.15.123:0/2981062187 >> v1:172.21.15.153:6789/0 conn(0x7fdefc154650 legacy=0x7fdefc15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.844+0000 7fdef9400640 1 -- 172.21.15.123:0/2981062187 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc15da50 con 0x7fdefc0c9f00 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.844+0000 7fdef8a00640 1 -- 172.21.15.123:0/2981062187 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3202773812 0 0) 0x7fdefc074b20 con 0x7fdefc154650 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.845+0000 7fdef9400640 1 --2- 172.21.15.123:0/2981062187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc0c9f00 0x7fdefc0c8170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdee80097a0 tx=0x7fdee802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa9d624fdaf2f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.845+0000 7fdef8a00640 1 -- 172.21.15.123:0/2981062187 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8037070 con 0x7fdefc0c9f00 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.845+0000 7fdef8a00640 1 -- 172.21.15.123:0/2981062187 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee8032440 con 0x7fdefc0c9f00 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.845+0000 7fdef8a00640 1 -- 172.21.15.123:0/2981062187 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8032810 con 0x7fdefc0c9f00 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 -- 172.21.15.123:0/2981062187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc0c9f00 msgr2=0x7fdefc0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 --2- 172.21.15.123:0/2981062187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc0c9f00 0x7fdefc0c8170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdee80097a0 tx=0x7fdee802a210 comp rx=0 tx=0).stop 2024-04-25T12:35:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 -- 172.21.15.123:0/2981062187 shutdown_connections 2024-04-25T12:35:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 --2- 172.21.15.123:0/2981062187 >> v2:172.21.15.153:3300/0 conn(0x7fdefc0730e0 0x7fdefc0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 --2- 172.21.15.123:0/2981062187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc0c9f00 0x7fdefc0c8170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 -- 172.21.15.123:0/2981062187 >> 172.21.15.123:0/2981062187 conn(0x7fdefc06f440 msgr2=0x7fdefc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 -- 172.21.15.123:0/2981062187 shutdown_connections 2024-04-25T12:35:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.846+0000 7fdf00c00640 1 -- 172.21.15.123:0/2981062187 wait complete. 2024-04-25T12:35:42.864 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:42.848+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:42.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.849+0000 7fdf00c00640 1 Processor -- start 2024-04-25T12:35:42.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.849+0000 7fdf00c00640 1 -- start start 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdf00c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdefc0730e0 0x7fdefc1f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdf00c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdefc0c9f00 0x7fdefc15a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdf00c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc154650 0x7fdefc0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdf00c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc15e2f0 con 0x7fdefc0c9f00 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdf00c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc15e010 con 0x7fdefc0730e0 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdf00c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc154a60 con 0x7fdefc154650 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdefa800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc154650 0x7fdefc0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.850+0000 7fdef9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdefc0730e0 0x7fdefc1f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc154650 0x7fdefc0708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:41264/0 (socket says 172.21.15.123:41264) 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 -- 172.21.15.123:0/53148998 learned_addr learned my addr 172.21.15.123:0/53148998 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdef9400640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdefc0c9f00 0x7fdefc15a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdefc0730e0 msgr2=0x7fdefc1f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdefc0730e0 0x7fdefc1f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdefc0c9f00 msgr2=0x7fdefc15a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdefc0c9f00 0x7fdefc15a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc15a660 con 0x7fdefc154650 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdefa800640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc154650 0x7fdefc0708a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdef400efc0 tx=0x7fdef400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.851+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4018020 con 0x7fdefc154650 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.852+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef4014480 con 0x7fdefc154650 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.852+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc071200 con 0x7fdefc154650 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.852+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4014870 con 0x7fdefc154650 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.852+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1f02b0 con 0x7fdefc154650 2024-04-25T12:35:42.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.853+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fdefc076700 con 0x7fdefc154650 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.854+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdef4005d10 con 0x7fdefc154650 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.855+0000 7fdef2000640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdedc0744e0 0x7fdedc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.855+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 6619+0+0 (secure 0 0 0) 0x7fdef4096470 con 0x7fdefc154650 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.855+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdedc0744e0 0x7fdedc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.855+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v38) ==== 54+0+16 (secure 0 0 0) 0x7fdef4022070 con 0x7fdefc154650 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.856+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fdefc15a830 0x7fdefc151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.856+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdefc073c20 con 0x7fdefc15a830 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.856+0000 7fdef9400640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fdefc15a830 0x7fdefc151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.856+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdedc0744e0 0x7fdedc0769f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007fd0 tx=0x7fdeec007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.857+0000 7fdef9400640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fdefc15a830 0x7fdefc151c10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.857+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc073c20 con 0x7fdefc15a830 2024-04-25T12:35:42.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.875+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc15b480 con 0x7fdefc15a830 2024-04-25T12:35:42.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.877+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc15b480 con 0x7fdefc15a830 2024-04-25T12:35:42.877 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:42.878 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:42.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.877+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdefc146f90 0x7fdefc1473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.877+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fdefc1478e0 con 0x7fdefc146f90 2024-04-25T12:35:42.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.877+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdefc146f90 0x7fdefc1473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.878+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdefc146f90 0x7fdefc1473a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.879+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc1478e0 con 0x7fdefc146f90 2024-04-25T12:35:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.895+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc14f910 con 0x7fdefc146f90 2024-04-25T12:35:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.897+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc14f910 con 0x7fdefc146f90 2024-04-25T12:35:42.897 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.897+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fdefc1503f0 0x7fdefc062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.897+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fdefc063520 con 0x7fdefc1503f0 2024-04-25T12:35:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.898+0000 7fdef9400640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fdefc1503f0 0x7fdefc062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.898+0000 7fdef9400640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fdefc1503f0 0x7fdefc062fe0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.899+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc063520 con 0x7fdefc1503f0 2024-04-25T12:35:42.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.914+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc0639b0 con 0x7fdefc1503f0 2024-04-25T12:35:42.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.916+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc0639b0 con 0x7fdefc1503f0 2024-04-25T12:35:42.916 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:42.916 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:42.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.916+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fdefc15bca0 0x7fdefc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.916+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fdefc063f80 con 0x7fdefc15bca0 2024-04-25T12:35:42.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.916+0000 7fdefa800640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fdefc15bca0 0x7fdefc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.917+0000 7fdefa800640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fdefc15bca0 0x7fdefc15c0b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.918+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc063f80 con 0x7fdefc15bca0 2024-04-25T12:35:42.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.933+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc02c9d0 con 0x7fdefc15bca0 2024-04-25T12:35:42.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.935+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc02c9d0 con 0x7fdefc15bca0 2024-04-25T12:35:42.935 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:42.935 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:42.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.935+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdefc02d0f0 0x7fdefc02d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.935+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fdefc02da40 con 0x7fdefc02d0f0 2024-04-25T12:35:42.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.935+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdefc02d0f0 0x7fdefc02d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.936+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdefc02d0f0 0x7fdefc02d500 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.937+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc02da40 con 0x7fdefc02d0f0 2024-04-25T12:35:42.949 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:42.949+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:42.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.968+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc02da40 con 0x7fdefc02d0f0 2024-04-25T12:35:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.971+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc02da40 con 0x7fdefc02d0f0 2024-04-25T12:35:42.971 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.971+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fdefc02dc80 0x7fdefc206bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.971+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fdefc2099a0 con 0x7fdefc02dc80 2024-04-25T12:35:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.971+0000 7fdef9400640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fdefc02dc80 0x7fdefc206bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.972+0000 7fdef9400640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fdefc02dc80 0x7fdefc206bd0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.973+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.5 v2:172.21.15.153:6826/2327013656 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc2099a0 con 0x7fdefc02dc80 2024-04-25T12:35:42.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.988+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc207110 con 0x7fdefc02dc80 2024-04-25T12:35:42.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.990+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.5 v2:172.21.15.153:6826/2327013656 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc207110 con 0x7fdefc02dc80 2024-04-25T12:35:42.991 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:35:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:35:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.991+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fdefc207830 0x7fdefc209c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.991+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fdefc20cba0 con 0x7fdefc207830 2024-04-25T12:35:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.991+0000 7fdefa800640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fdefc207830 0x7fdefc209c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:42.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.992+0000 7fdefa800640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fdefc207830 0x7fdefc209c50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:42.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.993+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc20cba0 con 0x7fdefc207830 2024-04-25T12:35:42.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:42.997+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f84500b5b80 con 0x7f84501602c0 2024-04-25T12:35:43.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.022+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc000950 con 0x7fdefc207830 2024-04-25T12:35:43.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.025+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc000950 con 0x7fdefc207830 2024-04-25T12:35:43.025 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:43.026 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:43.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.025+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fdefc20a6f0 0x7fdefc20ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:43.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.025+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fdefc20ffa0 con 0x7fdefc20a6f0 2024-04-25T12:35:43.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.026+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fdefc20a6f0 0x7fdefc20ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:43.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.027+0000 7fdef9e00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fdefc20a6f0 0x7fdefc20ab60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:43.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.028+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdefc20ffa0 con 0x7fdefc20a6f0 2024-04-25T12:35:43.050 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:43.051+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:43.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.055+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fdefc20ee20 con 0x7fdefc20a6f0 2024-04-25T12:35:43.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdef2000640 1 -- 172.21.15.123:0/53148998 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdefc20ee20 con 0x7fdefc20a6f0 2024-04-25T12:35:43.058 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fdefc15a830 msgr2=0x7fdefc151c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fdefc15a830 0x7fdefc151c10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdefc146f90 msgr2=0x7fdefc1473a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fdefc146f90 0x7fdefc1473a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fdefc1503f0 msgr2=0x7fdefc062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fdefc1503f0 0x7fdefc062fe0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fdefc15bca0 msgr2=0x7fdefc15c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fdefc15bca0 0x7fdefc15c0b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdefc02d0f0 msgr2=0x7fdefc02d500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdefc02d0f0 0x7fdefc02d500 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdefa800640 1 -- 172.21.15.123:0/53148998 reap_dead start 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.058+0000 7fdefa800640 1 -- 172.21.15.123:0/53148998 reap_dead start 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fdefc02dc80 msgr2=0x7fdefc206bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6826/2327013656,v1:172.21.15.153:6827/2327013656] conn(0x7fdefc02dc80 0x7fdefc206bd0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fdefc207830 msgr2=0x7fdefc209c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fdefc207830 0x7fdefc209c50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fdefc20a6f0 msgr2=0x7fdefc20ab60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fdefc20a6f0 0x7fdefc20ab60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdedc0744e0 msgr2=0x7fdedc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdedc0744e0 0x7fdedc0769f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007fd0 tx=0x7fdeec007de0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc154650 msgr2=0x7fdefc0708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 --2- 172.21.15.123:0/53148998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdefc154650 0x7fdefc0708a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdef400efc0 tx=0x7fdef400c490 comp rx=0 tx=0).stop 2024-04-25T12:35:43.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdefa800640 1 -- 172.21.15.123:0/53148998 reap_dead start 2024-04-25T12:35:43.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 shutdown_connections 2024-04-25T12:35:43.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 >> 172.21.15.123:0/53148998 conn(0x7fdefc06f440 msgr2=0x7fdefc153130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:43.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 shutdown_connections 2024-04-25T12:35:43.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.059+0000 7fdf00c00640 1 -- 172.21.15.123:0/53148998 wait complete. 2024-04-25T12:35:43.144 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:43.144 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:43.151 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:43.151+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:43.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:43.252 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:43.252+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:43.353 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:43.353+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:43.454 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:43.454+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:43.473 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:43.474 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.479+0000 7f844cc00640 1 -- 172.21.15.123:0/1010370246 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v39) ==== 91+0+0 (secure 0 0 0) 0x7f84500b5b80 con 0x7f84501602c0 2024-04-25T12:35:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:35:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.481+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f842c0744b0 msgr2=0x7f842c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.481+0000 7f8456a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f842c0744b0 0x7f842c0769c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8440007fd0 tx=0x7f8440007de0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 msgr2=0x7f84500b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500b2960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f844802adb0 tx=0x7f8448004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 shutdown_connections 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f842c0744b0 0x7f842c0769c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84501602c0 0x7f84500b2960 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8450156890 0x7f84501e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 --2- 172.21.15.123:0/1010370246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f845006d680 0x7f84500b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 >> 172.21.15.123:0/1010370246 conn(0x7f8450072ec0 msgr2=0x7f845006db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 shutdown_connections 2024-04-25T12:35:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:43.482+0000 7f8456a00640 1 -- 172.21.15.123:0/1010370246 wait complete. 2024-04-25T12:35:43.495 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:43.555 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:43.555+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:43.656 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:43.656+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:43.756 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:43.756+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:43.809 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:43.810 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:43.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:43.870 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:43.870+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:43.957 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:43.958+0000 7fed07c66640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34707) UID: 0 2024-04-25T12:35:44.059 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:44.058+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:44.142 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:44.142 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:44.160 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:44.159+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:44.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:44.260 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:44.261+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:44.361 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:44.362+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:44.372 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:35:44.372 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:35:44.373 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:35:44.373 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T12:35:44.373 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:35:44.407 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T12:35:44.407 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-04-25T12:35:44.408 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:35:44.408 DEBUG:teuthology.orchestra.run.smithi153:> type ceph-objectstore-tool 2024-04-25T12:35:44.464 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-25T12:35:44.464 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-04-25T12:35:44.465 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:44.464+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:44.494 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:44.494 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:44.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:44.565 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:44.565+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.9s1 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.6s0 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.2s2 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.1fs2 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.1es2 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.1cs2 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.1as2 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:4.17s0 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:3.10 2024-04-25T12:35:44.623 INFO:teuthology.orchestra.run.smithi153.stdout:3.18 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:3.38 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:4.1bs1 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:4.15s2 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:2.a 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:3.1f 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:2.4 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:2.1e 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:3.17 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:2.1b 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:3.22 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:2.d 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:2.1 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:4.bs0 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:3.0 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:3.29 2024-04-25T12:35:44.624 INFO:teuthology.orchestra.run.smithi153.stdout:4.ds3 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.4 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:2.f 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:2.0 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.1a 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.23 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.1e 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.31 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.27 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.3e 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:2.e 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.28 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.3b 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:3.3d 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:4.1ds1 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:4.16s0 2024-04-25T12:35:44.625 INFO:teuthology.orchestra.run.smithi153.stdout:4.12s2 2024-04-25T12:35:44.664 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:44.664+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:44.764 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:44.764 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:44.764 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:44.765+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:44.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:44.865 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:44.865+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:44.903 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 2.1 --file /var/log/ceph/exp.2.1.5 2024-04-25T12:35:44.987 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:44.988+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:45.061 INFO:teuthology.orchestra.run.smithi153.stderr:Exporting 2.1 info 2.1( v 25'4 (0'0,25'4] local-lis/les=18/19 n=2 ec=18/18 lis/c=18/18 les/c/f=19/19/0 sis=18) 2024-04-25T12:35:45.062 INFO:teuthology.orchestra.run.smithi153.stderr:Read #2:81735765:::619.00000000:head# 2024-04-25T12:35:45.062 INFO:teuthology.orchestra.run.smithi153.stderr:Read #2:85bbe569:::202.00000000:head# 2024-04-25T12:35:45.062 INFO:teuthology.orchestra.run.smithi153.stderr:Export successful 2024-04-25T12:35:45.067 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:45.067+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:45.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:45.098 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:45.098 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:45.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:45.167 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:45.167+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:45.268 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:45.269+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:45.330 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 2.1 2024-04-25T12:35:45.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.354+0000 7f4982a00640 1 Processor -- start 2024-04-25T12:35:45.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.354+0000 7f4982a00640 1 -- start start 2024-04-25T12:35:45.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.356+0000 7f4982a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.356+0000 7f4982a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f497c0730e0 0x7f497c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.356+0000 7f4982a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c074b20 con 0x7f497c154650 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.356+0000 7f4982a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f497c0ca780 con 0x7f497c0730e0 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.356+0000 7f4982a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0caa60 con 0x7f497c0c9f00 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.356+0000 7f497be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f497c154650 0x7f497c15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38588/0 (socket says 172.21.15.123:38588) 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.356+0000 7f497be00640 1 -- 172.21.15.123:0/1096226099 learned_addr learned my addr 172.21.15.123:0/1096226099 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f4980a00640 1 --2- 172.21.15.123:0/1096226099 >> v2:172.21.15.153:3300/0 conn(0x7f497c0730e0 0x7f497c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f497b400640 1 --2- 172.21.15.123:0/1096226099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f497aa00640 1 -- 172.21.15.123:0/1096226099 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937684724 0 0) 0x7f497c074b20 con 0x7f497c154650 2024-04-25T12:35:45.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f497aa00640 1 -- 172.21.15.123:0/1096226099 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4960003520 con 0x7f497c154650 2024-04-25T12:35:45.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f497b400640 1 -- 172.21.15.123:0/1096226099 >> v2:172.21.15.153:3300/0 conn(0x7f497c0730e0 msgr2=0x7f497c0734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:45.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f497b400640 1 --2- 172.21.15.123:0/1096226099 >> v2:172.21.15.153:3300/0 conn(0x7f497c0730e0 0x7f497c0734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f497b400640 1 -- 172.21.15.123:0/1096226099 >> v1:172.21.15.123:6789/0 conn(0x7f497c154650 legacy=0x7f497c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f497b400640 1 -- 172.21.15.123:0/1096226099 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c15da50 con 0x7f497c0c9f00 2024-04-25T12:35:45.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.357+0000 7f4980a00640 1 --2- 172.21.15.123:0/1096226099 >> v2:172.21.15.153:3300/0 conn(0x7f497c0730e0 0x7f497c0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.358+0000 7f497b400640 1 --2- 172.21.15.123:0/1096226099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c0c8170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49680097a0 tx=0x7f496802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63190a61758e9294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.359+0000 7f497aa00640 1 -- 172.21.15.123:0/1096226099 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968037070 con 0x7f497c0c9f00 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.359+0000 7f497aa00640 1 -- 172.21.15.123:0/1096226099 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4968032440 con 0x7f497c0c9f00 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.359+0000 7f497aa00640 1 -- 172.21.15.123:0/1096226099 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968032810 con 0x7f497c0c9f00 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 -- 172.21.15.123:0/1096226099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 msgr2=0x7f497c0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 --2- 172.21.15.123:0/1096226099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c0c8170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49680097a0 tx=0x7f496802a210 comp rx=0 tx=0).stop 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 -- 172.21.15.123:0/1096226099 shutdown_connections 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 --2- 172.21.15.123:0/1096226099 >> v2:172.21.15.153:3300/0 conn(0x7f497c0730e0 0x7f497c0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 --2- 172.21.15.123:0/1096226099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c0c8170 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 -- 172.21.15.123:0/1096226099 >> 172.21.15.123:0/1096226099 conn(0x7f497c06f440 msgr2=0x7f497c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:45.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 -- 172.21.15.123:0/1096226099 shutdown_connections 2024-04-25T12:35:45.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.360+0000 7f4982a00640 1 -- 172.21.15.123:0/1096226099 wait complete. 2024-04-25T12:35:45.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.363+0000 7f4982a00640 1 Processor -- start 2024-04-25T12:35:45.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.363+0000 7f4982a00640 1 -- start start 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.364+0000 7f4982a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f497c0730e0 0x7f497c14f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.364+0000 7f4982a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.364+0000 7f4982a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f497c154650 0x7f497c14a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.364+0000 7f4982a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f497c15e2f0 con 0x7f497c0730e0 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.364+0000 7f4982a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f497c15e010 con 0x7f497c154650 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.364+0000 7f4982a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f497c154a60 con 0x7f497c0c9f00 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f497c0730e0 0x7f497c14f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f497c0730e0 0x7f497c14f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59202/0 (socket says 172.21.15.123:59202) 2024-04-25T12:35:45.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 -- 172.21.15.123:0/4151196484 learned_addr learned my addr 172.21.15.123:0/4151196484 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f497c154650 0x7f497c14a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497b400640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 -- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 msgr2=0x7f497c1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f497c0c9f00 0x7f497c1592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 -- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f497c154650 msgr2=0x7f497c14a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f497c154650 0x7f497c14a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f497be00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c14fc40 con 0x7f497c0730e0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.365+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f497c154650 0x7f497c14a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.366+0000 7f497be00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f497c0730e0 0x7f497c14f700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f496c00c3c0 tx=0x7f496c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.366+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c017070 con 0x7f497c0730e0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.366+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496c005ce0 con 0x7f497c0730e0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.366+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c14e660 con 0x7f497c0730e0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.366+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c0072d0 con 0x7f497c0730e0 2024-04-25T12:35:45.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.366+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c14ec20 con 0x7f497c0730e0 2024-04-25T12:35:45.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.367+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f497c14fe10 con 0x7f497c0730e0 2024-04-25T12:35:45.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.369+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f496c01a070 con 0x7f497c0730e0 2024-04-25T12:35:45.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.370+0000 7f4973e00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f495c0744e0 0x7f495c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.370+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6619+0+0 (secure 0 0 0) 0x7f496c094930 con 0x7f497c0730e0 2024-04-25T12:35:45.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.370+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v40) ==== 54+0+16 (secure 0 0 0) 0x7f496c099220 con 0x7f497c0730e0 2024-04-25T12:35:45.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.370+0000 7f497b400640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f495c0744e0 0x7f495c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.370+0000 7f4982a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f497c159f50 0x7f497c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.370+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f497c0744e0 con 0x7f497c159f50 2024-04-25T12:35:45.371 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:45.372+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:45.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.371+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f497c159f50 0x7f497c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.371+0000 7f497b400640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f495c0744e0 0x7f495c0769f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4968034000 tx=0x7f496802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.372+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f497c159f50 0x7f497c0c7e60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.372+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f497c0744e0 con 0x7f497c159f50 2024-04-25T12:35:45.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.388+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f497c073c20 con 0x7f497c159f50 2024-04-25T12:35:45.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.390+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f497c073c20 con 0x7f497c159f50 2024-04-25T12:35:45.391 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:45.391 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:45.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.391+0000 7f4982a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f497c1f6200 0x7f497c1f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.391+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f497c1f6b50 con 0x7f497c1f6200 2024-04-25T12:35:45.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.391+0000 7f497be00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f497c1f6200 0x7f497c1f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.392+0000 7f497be00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f497c1f6200 0x7f497c1f6610 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.393+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f497c1f6b50 con 0x7f497c1f6200 2024-04-25T12:35:45.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.409+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f497c0c8c70 con 0x7f497c1f6200 2024-04-25T12:35:45.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.411+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f497c0c8c70 con 0x7f497c1f6200 2024-04-25T12:35:45.411 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:45.412 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:45.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.411+0000 7f4982a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f497c0634f0 0x7f497c063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.411+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f497c063e40 con 0x7f497c0634f0 2024-04-25T12:35:45.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.412+0000 7f497b400640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f497c0634f0 0x7f497c063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.412+0000 7f497b400640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f497c0634f0 0x7f497c063900 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.413+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f497c063e40 con 0x7f497c0634f0 2024-04-25T12:35:45.417 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:45.417 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.428+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f497c150640 con 0x7f497c0634f0 2024-04-25T12:35:45.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.430+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f497c150640 con 0x7f497c0634f0 2024-04-25T12:35:45.431 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.431+0000 7f4982a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f497c15ba90 0x7f497c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.431+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f497c150640 con 0x7f497c15ba90 2024-04-25T12:35:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.431+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f497c15ba90 0x7f497c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.432+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f497c15ba90 0x7f497c15bea0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.433+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f497c150640 con 0x7f497c15ba90 2024-04-25T12:35:45.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:45.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.450+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f497c150640 con 0x7f497c15ba90 2024-04-25T12:35:45.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.452+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f497c150640 con 0x7f497c15ba90 2024-04-25T12:35:45.453 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:45.454 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:45.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.453+0000 7f4982a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f497c02cee0 0x7f497c02d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.453+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f497c150640 con 0x7f497c02cee0 2024-04-25T12:35:45.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.453+0000 7f497be00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f497c02cee0 0x7f497c02d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.454+0000 7f497be00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f497c02cee0 0x7f497c02d2f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.454+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f497c150640 con 0x7f497c02cee0 2024-04-25T12:35:45.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.469+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f497c150640 con 0x7f497c02cee0 2024-04-25T12:35:45.469 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:45.469+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:45.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.471+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f497c150640 con 0x7f497c02cee0 2024-04-25T12:35:45.471 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.471+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=41}) -- 0x7f497c14dbf0 con 0x7f497c0730e0 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.471+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f497c150640 con 0x7f497c0730e0 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.472+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f496c05e640 con 0x7f497c0730e0 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.472+0000 7f4982a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f497c207070 0x7f497c209490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.472+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f497c02de40 con 0x7f497c207070 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.472+0000 7f497b400640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f497c207070 0x7f497c209490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.473+0000 7f497b400640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f497c207070 0x7f497c209490 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.474+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f497c02de40 con 0x7f497c207070 2024-04-25T12:35:45.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.489+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f497c000950 con 0x7f497c207070 2024-04-25T12:35:45.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.492+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f497c000950 con 0x7f497c207070 2024-04-25T12:35:45.492 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:45.492 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:45.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.492+0000 7f4982a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f497c209ee0 0x7f497c20c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.492+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f497c20f260 con 0x7f497c209ee0 2024-04-25T12:35:45.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.492+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f497c209ee0 0x7f497c20c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.493+0000 7f4980a00640 1 --2- 172.21.15.123:0/4151196484 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f497c209ee0 0x7f497c20c300 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.493+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== mon.0 v2:172.21.15.123:3300/0 8 ==== osd_map(41..41 src has 1..41) ==== 344+0+0 (secure 0 0 0) 0x7f496c09b070 con 0x7f497c0730e0 2024-04-25T12:35:45.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.494+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f497c20f260 con 0x7f497c209ee0 2024-04-25T12:35:45.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:35:45.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.526+0000 7f4982a00640 1 -- 172.21.15.123:0/4151196484 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f497c20dfd0 con 0x7f497c209ee0 2024-04-25T12:35:45.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.528+0000 7f4973e00640 1 -- 172.21.15.123:0/4151196484 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f497c20dfd0 con 0x7f497c209ee0 2024-04-25T12:35:45.528 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:45.528 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:45.546 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T12:35:45.546 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-25T12:35:45.589 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:45.589+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:45.671 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:45.672+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.758017229000004, 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:35:45.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:35:45.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.757333047000003, 2024-04-25T12:35:45.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.755882506000006, 2024-04-25T12:35:45.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:45.721 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.755621691000002, 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.751771304000002, 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.751548936000006, 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:35:45.730 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:35:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.751195147000004, 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:35:45.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:35:45.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.590576198999997, 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:35:45.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:35:45.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.590368707000003, 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.745 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:35:45.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:35:45.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.590224216999999, 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:35:45.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.755 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.590092423000002, 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.590018127999997, 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:35:45.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:35:45.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:35:45.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.589918386999997, 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.589848924999998, 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:35:45.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:35:45.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:35:45.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:45.776 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.589748811, 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:45.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:35:45.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:35:45.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.781 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.589651719999999, 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:45.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.589561314999997, 2024-04-25T12:35:45.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:35:45.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.589413342, 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 55.589305936999999, 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:45.797 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 50.555054638999998, 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:45.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:35:45.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:45.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:45.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.802 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:45.803 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:45.803 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:45.804 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:45.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.764+0000 7fc002a00640 1 Processor -- start 2024-04-25T12:35:45.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.764+0000 7fc002a00640 1 -- start start 2024-04-25T12:35:45.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.766+0000 7fc002a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbffc0c4d50 0x7fbffc154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc002a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbffc15afa0 0x7fbffc0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc002a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbffc15b3b0 con 0x7fbffc074870 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc002a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0c61f0 con 0x7fbffc15afa0 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc002a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0c87d0 con 0x7fbffc0c4d50 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc000a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbffc15afa0 0x7fbffc0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fbffbe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbffc0c4d50 0x7fbffc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fbffb400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbffc074870 0x7fbffc0c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:33956/0 (socket says 172.21.15.123:33956) 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fbffbe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbffc0c4d50 0x7fbffc154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:51180/0 (socket says 172.21.15.123:51180) 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fbffb400640 1 -- 172.21.15.123:0/3227383810 learned_addr learned my addr 172.21.15.123:0/3227383810 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc000a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbffc15afa0 0x7fbffc0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59216/0 (socket says 172.21.15.123:59216) 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc000a00640 1 -- 172.21.15.123:0/3227383810 >> v2:172.21.15.153:3300/0 conn(0x7fbffc0c4d50 msgr2=0x7fbffc154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc000a00640 1 --2- 172.21.15.123:0/3227383810 >> v2:172.21.15.153:3300/0 conn(0x7fbffc0c4d50 0x7fbffc154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc000a00640 1 -- 172.21.15.123:0/3227383810 >> v1:172.21.15.153:6789/0 conn(0x7fbffc074870 legacy=0x7fbffc0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.767+0000 7fc000a00640 1 -- 172.21.15.123:0/3227383810 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc073ce0 con 0x7fbffc15afa0 2024-04-25T12:35:45.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.768+0000 7fc000a00640 1 --2- 172.21.15.123:0/3227383810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc15afa0 0x7fbffc0cda90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0ce6d0 tx=0x7fbff402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdaba55da974074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.769+0000 7fbffaa00640 1 -- 172.21.15.123:0/3227383810 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4037070 con 0x7fbffc15afa0 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.769+0000 7fbffaa00640 1 -- 172.21.15.123:0/3227383810 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff4032440 con 0x7fbffc15afa0 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.769+0000 7fbffaa00640 1 -- 172.21.15.123:0/3227383810 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4032810 con 0x7fbffc15afa0 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 -- 172.21.15.123:0/3227383810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc15afa0 msgr2=0x7fbffc0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 --2- 172.21.15.123:0/3227383810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc15afa0 0x7fbffc0cda90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0ce6d0 tx=0x7fbff402a250 comp rx=0 tx=0).stop 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 -- 172.21.15.123:0/3227383810 shutdown_connections 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 --2- 172.21.15.123:0/3227383810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc15afa0 0x7fbffc0cda90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 --2- 172.21.15.123:0/3227383810 >> v2:172.21.15.153:3300/0 conn(0x7fbffc0c4d50 0x7fbffc154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 -- 172.21.15.123:0/3227383810 >> 172.21.15.123:0/3227383810 conn(0x7fbffc06c750 msgr2=0x7fbffc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 -- 172.21.15.123:0/3227383810 shutdown_connections 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.770+0000 7fc002a00640 1 -- 172.21.15.123:0/3227383810 wait complete. 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.772+0000 7fc002a00640 1 Processor -- start 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.772+0000 7fc002a00640 1 -- start start 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fc002a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc074870 0x7fbffc1f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fc002a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0c4d50 0x7fbffc0cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fc002a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc15afa0 0x7fbffc14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fc002a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0c8460 con 0x7fbffc074870 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fc002a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0742a0 con 0x7fbffc15afa0 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fc002a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc0c9370 con 0x7fbffc0c4d50 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc074870 0x7fbffc1f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc074870 0x7fbffc1f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59218/0 (socket says 172.21.15.123:59218) 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 -- 172.21.15.123:0/4291070565 learned_addr learned my addr 172.21.15.123:0/4291070565 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fc000a00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc15afa0 0x7fbffc14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffb400640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0c4d50 0x7fbffc0cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 -- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0c4d50 msgr2=0x7fbffc0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0c4d50 0x7fbffc0cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 -- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc15afa0 msgr2=0x7fbffc14c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc15afa0 0x7fbffc14c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:45.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.774+0000 7fbffbe00640 1 -- 172.21.15.123:0/4291070565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc1f67e0 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.775+0000 7fbffbe00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc074870 0x7fbffc1f62a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009d10 tx=0x7fbfec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.775+0000 7fbff3e00640 1 -- 172.21.15.123:0/4291070565 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec018020 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.775+0000 7fbff3e00640 1 -- 172.21.15.123:0/4291070565 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec014480 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.775+0000 7fbff3e00640 1 -- 172.21.15.123:0/4291070565 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec014830 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.775+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc14b240 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.775+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc159900 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.776+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc1593b0 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.777+0000 7fbff3e00640 1 -- 172.21.15.123:0/4291070565 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbfec01b020 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.778+0000 7fbff3e00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbfdc0744b0 0x7fbfdc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.779+0000 7fbff3e00640 1 -- 172.21.15.123:0/4291070565 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6619+0+0 (secure 0 0 0) 0x7fbfec095ff0 con 0x7fbffc074870 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.779+0000 7fbffb400640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbfdc0744b0 0x7fbfdc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.779+0000 7fbffb400640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbfdc0744b0 0x7fbfdc0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbffc15eac0 tx=0x7fbfe8007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:45.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.783+0000 7fbff3e00640 1 -- 172.21.15.123:0/4291070565 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfec022030 con 0x7fbffc074870 2024-04-25T12:35:45.813 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:45.805+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:45.905 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:45.905+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:45.928 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:45.928 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:45.928 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:45.928 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:45.937 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:45.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:45.991+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fbffc076700 con 0x7fbffc074870 2024-04-25T12:35:46.005 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:46.006+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:46.050 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:46.050 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:46.051 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:46.051 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:46.051 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:46.060 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:46.144 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:46.145+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:46.211 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:46.211+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:46.215 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:46.215 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:46.215 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:46.215 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:46.215 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.215 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265598221999994, 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.216 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:35:46.217 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:35:46.218 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:46.219 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.220 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.221 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265510512999995, 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:46.222 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.223 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.224 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:35:46.225 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:35:46.226 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265384585000007, 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.227 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.228 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:35:46.229 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:35:46.230 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:35:46.231 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265322925999996, 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.232 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.233 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.234 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.235 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.236 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265266163000007, 2024-04-25T12:35:46.237 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.238 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.239 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:35:46.240 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:35:46.241 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.242 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265202693000006, 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.243 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.244 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.245 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.246 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.247 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265150743999996, 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:46.248 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.249 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.250 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.251 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.252 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.265098793999996, 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.253 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.255 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:35:46.256 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:35:46.257 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.264981684999995, 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.259 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.263 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.264943238000001, 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:46.264 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:35:46.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:35:46.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.099808185000001, 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:35:46.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:35:46.270 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:35:46.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:35:46.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.099750133999997, 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.273 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.099690441, 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.278 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.062770358999998, 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.283 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:35:46.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:35:46.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:46.286 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.062681392999998, 2024-04-25T12:35:46.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:35:46.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.291 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.292 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.293 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.062637551999998, 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.294 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:35:46.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:35:46.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:46.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:46.299 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.061123647999999, 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:46.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:35:46.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:35:46.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:35:46.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.060204431999999, 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.309 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.059749078999999, 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:35:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:35:46.312 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.313 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.059720101000003, 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:35:46.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:35:46.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.317 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:46.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:46.319 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:46.319+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:46.419 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:46.419+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:46.475 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:46.475 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:46.475 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:46.476 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:46.485 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:46.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.503+0000 7fbff3e00640 1 -- 172.21.15.123:0/4291070565 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v42) ==== 101+0+0 (secure 0 0 0) 0x7fbfec010040 con 0x7fbffc074870 2024-04-25T12:35:46.504 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.507+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbfdc0744b0 msgr2=0x7fbfdc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.507+0000 7fc002a00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbfdc0744b0 0x7fbfdc0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbffc15eac0 tx=0x7fbfe8007e80 comp rx=0 tx=0).stop 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.507+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc074870 msgr2=0x7fbffc1f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.507+0000 7fc002a00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc074870 0x7fbffc1f62a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009d10 tx=0x7fbfec00c490 comp rx=0 tx=0).stop 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 shutdown_connections 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbfdc0744b0 0x7fbfdc0769c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbffc15afa0 0x7fbffc14c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbffc0c4d50 0x7fbffc0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 --2- 172.21.15.123:0/4291070565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc074870 0x7fbffc1f62a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:46.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 >> 172.21.15.123:0/4291070565 conn(0x7fbffc06c750 msgr2=0x7fbffc0c64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:46.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 shutdown_connections 2024-04-25T12:35:46.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:46.508+0000 7fc002a00640 1 -- 172.21.15.123:0/4291070565 wait complete. 2024-04-25T12:35:46.529 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:46.530+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:46.544 INFO:teuthology.orchestra.run.smithi153.stdout: marking collection for removal 2024-04-25T12:35:46.545 INFO:teuthology.orchestra.run.smithi153.stdout:setting '_remove' omap key 2024-04-25T12:35:46.545 INFO:teuthology.orchestra.run.smithi153.stdout:finish_remove_pgs 2.1_head removing 2.1 2024-04-25T12:35:46.546 INFO:teuthology.orchestra.run.smithi153.stdout:Remove successful 2024-04-25T12:35:46.601 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:46.601 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:46.601 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:46.601 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:46.601 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:46.610 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:46.621 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:46.621+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:46.722 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:46.722+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:46.764 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:46.764 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 24/18 rep_tid 1 entries 24'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.809689417000001, 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075768199999999997, 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954840+0000", 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6869999999999999e-06 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954864+0000", 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3824000000000001e-05 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954902+0000", 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500999999999999e-05 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954919+0000", 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7618e-05 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955011+0000", 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2250000000000006e-05 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955596+0000", 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058480200000000004 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:25 2.1b e25/18 2:dbae76b5:::mds1_inotable:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.648004335000003, 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036130300000000001, 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.768 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116525+0000", 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.455e-06 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116539+0000", 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4e-05 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116563+0000", 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4111999999999999e-05 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116577+0000", 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4059999999999999e-05 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116689+0000", 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011209000000000001 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116719+0000", 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.989e-05 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120082+0000", 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033628130000000001 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120137+0000", 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4610000000000001e-05 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 4 entries 25'3 (0'0) error 2:db457a8a:::201.00000001:head by mds.1.9:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:46.772 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.646947793999999, 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030622480000000001, 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.773 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117581+0000", 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0279999999999999e-06 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117596+0000", 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5109999999999999e-05 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117631+0000", 2024-04-25T12:35:46.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4230000000000003e-05 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117655+0000", 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4987e-05 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117751+0000", 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5311999999999998e-05 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120642+0000", 2024-04-25T12:35:46.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028915809999999998 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 5 entries 25'4 (0'0) error 2:d80f8093:::201.00000002:head by mds.1.9:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.646446562999998, 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0025905469999999999, 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.776 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118082+0000", 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4000000000000001e-07 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.777 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118091+0000", 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9919999999999996e-06 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118108+0000", 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7207999999999999e-05 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118124+0000", 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5378000000000001e-05 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.778 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118203+0000", 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8929000000000003e-05 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120672+0000", 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024692999999999998 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:34 2.1b e25/18 2:db457a8a:::201.00000001:head v 25'5, mlcod=25'3)", 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:46.779 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.642972229000002, 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016516759999999999, 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.780 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121557+0000", 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.423e-06 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121568+0000", 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0499000000000001e-05 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121585+0000", 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7283e-05 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.781 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121597+0000", 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1821000000000001e-05 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121653+0000", 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6533000000000002e-05 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121671+0000", 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7285000000000001e-05 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.782 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123158+0000", 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0014869519999999999 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123207+0000", 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9879999999999997e-05 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:35 2.a 2:54ba6923:::mds1_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.783 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.620859912, 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016436459999999999, 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.784 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143670+0000", 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4040000000000002e-06 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143685+0000", 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4768e-05 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143703+0000", 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7424000000000001e-05 2024-04-25T12:35:46.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143721+0000", 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8685000000000002e-05 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143824+0000", 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010286800000000001 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144032+0000", 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020812700000000001 2024-04-25T12:35:46.786 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144205+0000", 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017276999999999999 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144982+0000", 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077719300000000002 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145196+0000", 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000213308 2024-04-25T12:35:46.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145254+0000", 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8619000000000002e-05 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145312+0000", 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7479999999999999e-05 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.788 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.1.9:35 2.a e25/18 ondisk, result = 0)", 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.619433344999997, 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00022686700000000001, 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:46.789 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145096+0000", 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2309999999999999e-06 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145102+0000", 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1639999999999998e-06 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.790 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145115+0000", 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2809000000000001e-05 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145124+0000", 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9989999999999997e-06 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145178+0000", 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4287999999999999e-05 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.791 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145189+0000", 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.084e-05 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145321+0000", 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013253599999999999 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:11 2.c 2:36924187:::618.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.612184079000002, 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070050249999999998, 2024-04-25T12:35:46.792 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.793 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152345+0000", 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6270000000000001e-06 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152355+0000", 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4069999999999999e-06 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152384+0000", 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9479e-05 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.794 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152399+0000", 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152478+0000", 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9702000000000004e-05 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152615+0000", 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013665500000000001 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.795 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152730+0000", 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011504499999999999 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156394+0000", 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036643259999999999 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028370299999999999 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.796 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159289+0000", 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7268999999999997e-05 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159349+0000", 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0161999999999998e-05 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.612097505000001, 2024-04-25T12:35:46.797 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070895239999999998, 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.798 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152431+0000", 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6899999999999996e-07 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152440+0000", 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3259999999999996e-06 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152450+0000", 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0874e-05 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.799 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152470+0000", 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9658999999999999e-05 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153265+0000", 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079454099999999995 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153349+0000", 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4381000000000004e-05 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153485+0000", 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136365 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156400+0000", 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029144990000000001 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159418+0000", 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030182299999999998 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.801 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159477+0000", 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159520+0000", 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3127000000000002e-05 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:46.802 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.612033891000003, 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0068584939999999997, 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152495+0000", 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.16e-07 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152501+0000", 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7819999999999999e-06 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152510+0000", 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9199999999999993e-06 2024-04-25T12:35:46.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152518+0000", 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9090000000000006e-06 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152651+0000", 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000132349 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152795+0000", 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014392800000000001 2024-04-25T12:35:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152930+0000", 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013561399999999999 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156365+0000", 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034348830000000001 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159266+0000", 2024-04-25T12:35:46.806 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029006959999999999 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159296+0000", 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0026e-05 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159353+0000", 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6570999999999998e-05 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.807 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.611947927999999, 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069239999999999996, 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.808 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152581+0000", 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3399999999999998e-07 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152587+0000", 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4049999999999997e-06 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.809 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152598+0000", 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1043000000000001e-05 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152607+0000", 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9409999999999996e-06 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153448+0000", 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00084054699999999997 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153563+0000", 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000115584 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153719+0000", 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015566199999999999 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156373+0000", 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002653617 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159423+0000", 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030508060000000001 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159472+0000", 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9042000000000002e-05 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159504+0000", 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1519000000000001e-05 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.611865045000002, 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0086667419999999998, 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152664+0000", 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0100000000000003e-07 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152671+0000", 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8870000000000003e-06 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152679+0000", 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2730000000000002e-06 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152696+0000", 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7373000000000001e-05 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153747+0000", 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001050647 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153967+0000", 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00021982699999999999 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154122+0000", 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015472399999999999 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156405+0000", 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022838110000000002 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161248+0000", 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048425669999999999 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161283+0000", 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5166000000000002e-05 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161330+0000", 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6566000000000001e-05 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.817 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.2 epoch 25/18 rep_tid 20 entries 25'1 (0'0) error 2:40dbbc66:::202.00000003:head by mds.2.13:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.610259163999999, 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0021562719999999999, 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154270+0000", 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.638e-06 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154290+0000", 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392e-05 2024-04-25T12:35:46.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154315+0000", 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4858000000000002e-05 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154335+0000", 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0044e-05 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154711+0000", 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037622099999999999 2024-04-25T12:35:46.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156425+0000", 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017141190000000001 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:11 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.605403269999996, 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00023418899999999999, 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.821 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159126+0000", 2024-04-25T12:35:46.822 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7699999999999992e-07 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159131+0000", 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8109999999999999e-06 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159143+0000", 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1745e-05 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159153+0000", 2024-04-25T12:35:46.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6520000000000001e-06 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159211+0000", 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8582999999999998e-05 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159225+0000", 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3088000000000001e-05 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159359+0000", 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013433299999999999 2024-04-25T12:35:46.824 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:15 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.605352873999998, 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000189507, 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159176+0000", 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6899999999999998e-07 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159180+0000", 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1320000000000004e-06 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159188+0000", 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9319999999999996e-06 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159199+0000", 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1165999999999999e-05 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5071000000000002e-05 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159262+0000", 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0160000000000008e-06 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159365+0000", 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010272099999999999 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:12 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.605313944000002, 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000315354, 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2099999999999998e-07 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159217+0000", 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.875e-06 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.830 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159223+0000", 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2999999999999998e-06 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159229+0000", 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6740000000000002e-06 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159401+0000", 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017201599999999999 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.831 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159412+0000", 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0349e-05 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159529+0000", 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000117819 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:16 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:46.832 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.605285674999998, 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00026996799999999999, 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159243+0000", 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0100000000000001e-07 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159246+0000", 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.687e-06 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.072e-06 2024-04-25T12:35:46.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159261+0000", 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2169999999999999e-06 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159408+0000", 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000147637 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159417+0000", 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999996e-06 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.835 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159512+0000", 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5163999999999999e-05 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:23 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 51.603371520000003, 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000182072, 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:46.836 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161157+0000", 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7400000000000002e-07 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161164+0000", 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161180+0000", 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6189999999999999e-05 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161191+0000", 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0981e-05 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161238+0000", 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7509999999999997e-05 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161244+0000", 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.694e-06 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161338+0000", 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4432999999999995e-05 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:46.839 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:46.840 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:46.841 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:46.841 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:46.842+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:46.941 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:46.942+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:46.994 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:46.994 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:46.994 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:46.994 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:47.004 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:47.043 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:47.044+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:47.071 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.1.5 2024-04-25T12:35:47.155 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:47.156+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:47.157 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:47.158 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:47.158 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:47.158 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:47.158 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:47.168 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:47.253 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:47.254+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:47.324 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 88.372908023999997, 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:35:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:35:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:35:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 88.372820369999999, 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:35:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:35:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 88.372475801999997, 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:35:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:35:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 88.372403332000005, 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 88.372329354000001, 2024-04-25T12:35:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 88.370291981999998, 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:35:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:35:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 88.369431845999998, 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:35:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.208746191000003, 2024-04-25T12:35:47.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:35:47.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.208670077000001, 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.373 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:35:47.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:35:47.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.379 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.208624407999999, 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:35:47.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:35:47.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:35:47.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:35:47.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.208579632999999, 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.387 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:35:47.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:35:47.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.391 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.201963208999999, 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:35:47.392 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:35:47.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.201903719000001, 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.397 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:35:47.399 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.400 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 52.170933427000001, 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.401 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.405 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:35:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:47.407 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 52.170855916000001, 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.411 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:35:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.413 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 52.170808067000003, 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.416 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:35:47.417 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.419 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 52.170766176999997, 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.420 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:47.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:35:47.423 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:35:47.424 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.425 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.426 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 52.170732004999998, 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:35:47.427 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:47.428 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.429 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.430 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.431 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:35:47.432 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 52.170449263000002, 2024-04-25T12:35:47.433 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.434 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.435 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:35:47.436 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 52.166308831999999, 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:35:47.437 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:47.438 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.439 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:35:47.440 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:47.441 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:47.443 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:47.443+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:47.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:47.544 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:47.544+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:47.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:47.644 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:47.645+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:47.723 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:47.724 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:47.724 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:47.724 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:47.745 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:47.746+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:47.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:47.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.804+0000 7f9977800640 1 Processor -- start 2024-04-25T12:35:47.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.804+0000 7f9977800640 1 -- start start 2024-04-25T12:35:47.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f9977800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f99700c2790 0x7f9970074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f9977800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99700c8170 0x7f997015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f9977800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99700c8580 con 0x7f9970152c70 2024-04-25T12:35:47.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f9977800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9970158bd0 con 0x7f99700c2790 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f9977800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9970158eb0 con 0x7f99700c8170 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f996fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f99700c2790 0x7f9970074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f996fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f99700c2790 0x7f9970074820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59222/0 (socket says 172.21.15.123:59222) 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f996fe00640 1 -- 172.21.15.123:0/68593204 learned_addr learned my addr 172.21.15.123:0/68593204 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.807+0000 7f9975800640 1 --2- 172.21.15.123:0/68593204 >> v2:172.21.15.153:3300/0 conn(0x7f99700c8170 0x7f997015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.808+0000 7f996fe00640 1 -- 172.21.15.123:0/68593204 >> v2:172.21.15.153:3300/0 conn(0x7f99700c8170 msgr2=0x7f997015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.808+0000 7f996f400640 1 -- 172.21.15.123:0/68593204 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238384187 0 0) 0x7f99700c8580 con 0x7f9970152c70 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.808+0000 7f996fe00640 1 --2- 172.21.15.123:0/68593204 >> v2:172.21.15.153:3300/0 conn(0x7f99700c8170 0x7f997015c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.808+0000 7f996fe00640 1 -- 172.21.15.123:0/68593204 >> v1:172.21.15.153:6789/0 conn(0x7f9970152c70 legacy=0x7f99700c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.808+0000 7f996fe00640 1 -- 172.21.15.123:0/68593204 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9970159190 con 0x7f99700c2790 2024-04-25T12:35:47.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.808+0000 7f9975800640 1 --2- 172.21.15.123:0/68593204 >> v2:172.21.15.153:3300/0 conn(0x7f99700c8170 0x7f997015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.809+0000 7f996fe00640 1 --2- 172.21.15.123:0/68593204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 0x7f9970074820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9970072f50 tx=0x7f995c02b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e40f610dcb32ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.809+0000 7f996f400640 1 -- 172.21.15.123:0/68593204 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00b040 con 0x7f99700c2790 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.809+0000 7f996f400640 1 -- 172.21.15.123:0/68593204 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c030710 con 0x7f99700c2790 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.809+0000 7f996f400640 1 -- 172.21.15.123:0/68593204 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c030ae0 con 0x7f99700c2790 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 -- 172.21.15.123:0/68593204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 msgr2=0x7f9970074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 --2- 172.21.15.123:0/68593204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 0x7f9970074820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9970072f50 tx=0x7f995c02b5e0 comp rx=0 tx=0).stop 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 -- 172.21.15.123:0/68593204 shutdown_connections 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 --2- 172.21.15.123:0/68593204 >> v2:172.21.15.153:3300/0 conn(0x7f99700c8170 0x7f997015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 --2- 172.21.15.123:0/68593204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 0x7f9970074820 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 -- 172.21.15.123:0/68593204 >> 172.21.15.123:0/68593204 conn(0x7f997006f440 msgr2=0x7f997006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 -- 172.21.15.123:0/68593204 shutdown_connections 2024-04-25T12:35:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.810+0000 7f9977800640 1 -- 172.21.15.123:0/68593204 wait complete. 2024-04-25T12:35:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.813+0000 7f9977800640 1 Processor -- start 2024-04-25T12:35:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.813+0000 7f9977800640 1 -- start start 2024-04-25T12:35:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.815+0000 7f9977800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 0x7f9970149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.815+0000 7f9977800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99700c8170 0x7f9970146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.815+0000 7f9977800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9970152c70 0x7f9970144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f9977800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99700c46f0 con 0x7f99700c2790 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f9977800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9970159750 con 0x7f9970152c70 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f9977800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f99700c4870 con 0x7f99700c8170 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f9974e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 0x7f9970149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f996fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99700c8170 0x7f9970146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f9975800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9970152c70 0x7f9970144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f9974e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 0x7f9970149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59234/0 (socket says 172.21.15.123:59234) 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f9974e00640 1 -- 172.21.15.123:0/3216069369 learned_addr learned my addr 172.21.15.123:0/3216069369 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f996fe00640 1 -- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9970152c70 msgr2=0x7f9970144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f996fe00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9970152c70 0x7f9970144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f996fe00640 1 -- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 msgr2=0x7f9970149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f996fe00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99700c2790 0x7f9970149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.816+0000 7f996fe00640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99701463c0 con 0x7f99700c8170 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.817+0000 7f996fe00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99700c8170 0x7f9970146db0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f995c032000 tx=0x7f995c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.817+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c032a40 con 0x7f99700c8170 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.817+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c03f070 con 0x7f99700c8170 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.817+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c032e00 con 0x7f99700c8170 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.817+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99701f3200 con 0x7f99700c8170 2024-04-25T12:35:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.817+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99701584d0 con 0x7f99700c8170 2024-04-25T12:35:47.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.818+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f9970076700 con 0x7f99700c8170 2024-04-25T12:35:47.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.819+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f995c044460 con 0x7f99700c8170 2024-04-25T12:35:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.820+0000 7f996cc00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f99500744b0 0x7f99500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.821+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 6619+0+0 (secure 0 0 0) 0x7f995c0b5a80 con 0x7f99700c8170 2024-04-25T12:35:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.821+0000 7f9974e00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f99500744b0 0x7f99500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.821+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v42) ==== 54+0+16 (secure 0 0 0) 0x7f995c0b7e20 con 0x7f99700c8170 2024-04-25T12:35:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.821+0000 7f9977800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f99701f3ce0 0x7f99700c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.821+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99701466a0 con 0x7f99701f3ce0 2024-04-25T12:35:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.821+0000 7f9974e00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f99500744b0 0x7f99500769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f99640017f0 tx=0x7f9964009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.822+0000 7f9975800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f99701f3ce0 0x7f99700c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.822+0000 7f9975800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f99701f3ce0 0x7f99700c8d40 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.823+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99701466a0 con 0x7f99701f3ce0 2024-04-25T12:35:47.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.839+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9970145400 con 0x7f99701f3ce0 2024-04-25T12:35:47.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.841+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9968003990 con 0x7f99701f3ce0 2024-04-25T12:35:47.842 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:47.842 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:47.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.842+0000 7f9977800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f99700c9280 0x7f997014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.842+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f99701484a0 con 0x7f99700c9280 2024-04-25T12:35:47.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.842+0000 7f9974e00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f99700c9280 0x7f997014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.842+0000 7f9974e00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f99700c9280 0x7f997014b2c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.843+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99701484a0 con 0x7f99700c9280 2024-04-25T12:35:47.846 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:47.846+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.860+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9970074600 con 0x7f99700c9280 2024-04-25T12:35:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.862+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9970074600 con 0x7f99700c9280 2024-04-25T12:35:47.863 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.863+0000 7f9977800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f99701f6b60 0x7f99701f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.863+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f99701f74b0 con 0x7f99701f6b60 2024-04-25T12:35:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.863+0000 7f996fe00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f99701f6b60 0x7f99701f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.864+0000 7f996fe00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f99701f6b60 0x7f99701f6f70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.865+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99701f74b0 con 0x7f99701f6b60 2024-04-25T12:35:47.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.880+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f99700632b0 con 0x7f99701f6b60 2024-04-25T12:35:47.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.884+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f99700632b0 con 0x7f99701f6b60 2024-04-25T12:35:47.884 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:47.885 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:47.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.884+0000 7f9977800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f9970150ce0 0x7f99701510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.884+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9970063230 con 0x7f9970150ce0 2024-04-25T12:35:47.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.885+0000 7f9975800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f9970150ce0 0x7f99701510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.886+0000 7f9975800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f9970150ce0 0x7f99701510f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.887+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9970063230 con 0x7f9970150ce0 2024-04-25T12:35:47.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.902+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9970063230 con 0x7f9970150ce0 2024-04-25T12:35:47.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.906+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9970063230 con 0x7f9970150ce0 2024-04-25T12:35:47.907 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:47.908 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:47.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.907+0000 7f9977800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f997002c9d0 0x7f997002cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.907+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f9970063230 con 0x7f997002c9d0 2024-04-25T12:35:47.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.908+0000 7f9974e00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f997002c9d0 0x7f997002cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.909+0000 7f9974e00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f997002c9d0 0x7f997002cde0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.910+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9970063230 con 0x7f997002c9d0 2024-04-25T12:35:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.925+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9970063230 con 0x7f997002c9d0 2024-04-25T12:35:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.927+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9970063230 con 0x7f997002c9d0 2024-04-25T12:35:47.927 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.927+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=43}) -- 0x7f99701e39b0 con 0x7f99700c8170 2024-04-25T12:35:47.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.928+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99701e3d10 con 0x7f99700c8170 2024-04-25T12:35:47.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.928+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f995c07f030 con 0x7f99700c8170 2024-04-25T12:35:47.929 INFO:teuthology.orchestra.run.smithi123.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-04-25T12:35:47.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.928+0000 7f9977800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f997002d830 0x7f9970209a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.928+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f997002dc40 con 0x7f997002d830 2024-04-25T12:35:47.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.929+0000 7f996fe00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f997002d830 0x7f9970209a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.929+0000 7f996fe00640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f997002d830 0x7f9970209a40 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.930+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f997002dc40 con 0x7f997002d830 2024-04-25T12:35:47.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.946+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f997002dc40 con 0x7f997002d830 2024-04-25T12:35:47.946 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:47.947+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:47.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.948+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f997002dc40 con 0x7f997002d830 2024-04-25T12:35:47.948 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:47.949 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:47.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.948+0000 7f9977800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f997020a490 0x7f997020c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:47.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.948+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f997020f680 con 0x7f997020a490 2024-04-25T12:35:47.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.948+0000 7f9975800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f997020a490 0x7f997020c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:47.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.949+0000 7f9975800640 1 --2- 172.21.15.123:0/3216069369 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f997020a490 0x7f997020c8b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:47.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.950+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f997020f680 con 0x7f997020a490 2024-04-25T12:35:47.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.965+0000 7f9977800640 1 -- 172.21.15.123:0/3216069369 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f997020cdf0 con 0x7f997020a490 2024-04-25T12:35:47.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:47.967+0000 7f996cc00640 1 -- 172.21.15.123:0/3216069369 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f997020cdf0 con 0x7f997020a490 2024-04-25T12:35:47.968 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:47.968 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:47.986 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T12:35:47.987 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-25T12:35:48.011 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:48.011 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:48.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:48.047 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:48.048+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:48.148 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:48.149+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:48.230 INFO:teuthology.orchestra.run.smithi153.stdout:get_pg_num_history pg_num_history pg_num_history(e39 pg_nums {1={15=1},2={18=32},3={19=64},4={23=32}} deleted_pools ) 2024-04-25T12:35:48.231 INFO:teuthology.orchestra.run.smithi153.stdout:Importing pgid 2.1 2024-04-25T12:35:48.231 INFO:teuthology.orchestra.run.smithi153.stdout:Write #2:81735765:::619.00000000:head# 2024-04-25T12:35:48.231 INFO:teuthology.orchestra.run.smithi153.stdout:snapset 0=[]:{} 2024-04-25T12:35:48.231 INFO:teuthology.orchestra.run.smithi153.stdout:Write #2:85bbe569:::202.00000000:head# 2024-04-25T12:35:48.231 INFO:teuthology.orchestra.run.smithi153.stdout:snapset 0=[]:{} 2024-04-25T12:35:48.232 INFO:teuthology.orchestra.run.smithi153.stdout:write_pg epoch 25 info 2.1( v 25'4 (0'0,25'4] local-lis/les=18/19 n=2 ec=18/18 lis/c=18/18 les/c/f=19/19/0 sis=18) 2024-04-25T12:35:48.232 INFO:teuthology.orchestra.run.smithi153.stdout:Import successful 2024-04-25T12:35:48.250 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:48.251+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:48.350 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:48.351+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:48.352 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:48.352 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:48.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:48.464 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:48.464+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:48.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:35:48.588 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:48.588+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:48.653 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:48.653+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:48.696 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:48.696 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:48.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:48.749 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /var/log/ceph/exp.2.1.5 2024-04-25T12:35:48.765 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:48.760+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:48.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.805+0000 7f0dbf200640 1 Processor -- start 2024-04-25T12:35:48.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.805+0000 7f0dbf200640 1 -- start start 2024-04-25T12:35:48.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0dbf200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0db80c4d50 0x7f0db8154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0dbf200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0db8074870 0x7f0db815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0dbf200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db8073ce0 con 0x7f0db80c8c60 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0dbf200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80c61f0 con 0x7f0db80c4d50 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0dbf200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80c87d0 con 0x7f0db8074870 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0db7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0db80c4d50 0x7f0db8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0dbd200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0db80c8c60 0x7f0db80cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38600/0 (socket says 172.21.15.123:38600) 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0db7400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0db8074870 0x7f0db815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.807+0000 7f0dbd200640 1 -- 172.21.15.123:0/4179807304 learned_addr learned my addr 172.21.15.123:0/4179807304 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.808+0000 7f0db7400640 1 -- 172.21.15.123:0/4179807304 >> v2:172.21.15.123:3300/0 conn(0x7f0db80c4d50 msgr2=0x7f0db8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.808+0000 7f0db7400640 1 --2- 172.21.15.123:0/4179807304 >> v2:172.21.15.123:3300/0 conn(0x7f0db80c4d50 0x7f0db8154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.808+0000 7f0db7400640 1 -- 172.21.15.123:0/4179807304 >> v1:172.21.15.123:6789/0 conn(0x7f0db80c8c60 legacy=0x7f0db80cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.808+0000 7f0db7400640 1 -- 172.21.15.123:0/4179807304 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db8073fc0 con 0x7f0db8074870 2024-04-25T12:35:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.808+0000 7f0db6a00640 1 -- 172.21.15.123:0/4179807304 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1395611868 0 0) 0x7f0db8073ce0 con 0x7f0db80c8c60 2024-04-25T12:35:48.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.809+0000 7f0db7400640 1 --2- 172.21.15.123:0/4179807304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db8074870 0x7f0db815afa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0db8072a70 tx=0x7f0da402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea24206e2b41fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:48.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.809+0000 7f0db6a00640 1 -- 172.21.15.123:0/4179807304 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4037070 con 0x7f0db8074870 2024-04-25T12:35:48.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.809+0000 7f0db6a00640 1 -- 172.21.15.123:0/4179807304 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da4032440 con 0x7f0db8074870 2024-04-25T12:35:48.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.809+0000 7f0db6a00640 1 -- 172.21.15.123:0/4179807304 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4032810 con 0x7f0db8074870 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 -- 172.21.15.123:0/4179807304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db8074870 msgr2=0x7f0db815afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 --2- 172.21.15.123:0/4179807304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db8074870 0x7f0db815afa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0db8072a70 tx=0x7f0da402a250 comp rx=0 tx=0).stop 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 -- 172.21.15.123:0/4179807304 shutdown_connections 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 --2- 172.21.15.123:0/4179807304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db8074870 0x7f0db815afa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 --2- 172.21.15.123:0/4179807304 >> v2:172.21.15.123:3300/0 conn(0x7f0db80c4d50 0x7f0db8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 -- 172.21.15.123:0/4179807304 >> 172.21.15.123:0/4179807304 conn(0x7f0db806c750 msgr2=0x7f0db806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 -- 172.21.15.123:0/4179807304 shutdown_connections 2024-04-25T12:35:48.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.810+0000 7f0dbf200640 1 -- 172.21.15.123:0/4179807304 wait complete. 2024-04-25T12:35:48.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.813+0000 7f0dbf200640 1 Processor -- start 2024-04-25T12:35:48.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.813+0000 7f0dbf200640 1 -- start start 2024-04-25T12:35:48.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0dbf200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0db8074870 0x7f0db81f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:48.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0dbf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db80c4d50 0x7f0db814ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:48.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0dbf200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db80c8c60 0x7f0db814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:48.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0dbf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80ce1a0 con 0x7f0db80c4d50 2024-04-25T12:35:48.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0dbf200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80c8460 con 0x7f0db80c8c60 2024-04-25T12:35:48.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0dbf200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0db80ce320 con 0x7f0db8074870 2024-04-25T12:35:48.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0db7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db80c4d50 0x7f0db814ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:48.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db80c8c60 0x7f0db814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:48.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0db7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db80c4d50 0x7f0db814ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59244/0 (socket says 172.21.15.123:59244) 2024-04-25T12:35:48.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0db7400640 1 -- 172.21.15.123:0/3403352133 learned_addr learned my addr 172.21.15.123:0/3403352133 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:48.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.815+0000 7f0db7e00640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0db8074870 0x7f0db81f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.816+0000 7f0db7400640 1 -- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0db8074870 msgr2=0x7f0db81f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.816+0000 7f0db7400640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0db8074870 0x7f0db81f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.816+0000 7f0db7400640 1 -- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db80c8c60 msgr2=0x7f0db814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.816+0000 7f0db7400640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db80c8c60 0x7f0db814fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.816+0000 7f0db7400640 1 -- 172.21.15.123:0/3403352133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db8153a20 con 0x7f0db80c4d50 2024-04-25T12:35:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.816+0000 7f0db7400640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db80c4d50 0x7f0db814ac10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000c00 tx=0x7f0da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:48.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.817+0000 7f0dafe00640 1 -- 172.21.15.123:0/3403352133 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4037050 con 0x7f0db80c4d50 2024-04-25T12:35:48.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.817+0000 7f0dafe00640 1 -- 172.21.15.123:0/3403352133 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da4038040 con 0x7f0db80c4d50 2024-04-25T12:35:48.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.817+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db81f2b10 con 0x7f0db80c4d50 2024-04-25T12:35:48.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.817+0000 7f0dafe00640 1 -- 172.21.15.123:0/3403352133 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4004470 con 0x7f0db80c4d50 2024-04-25T12:35:48.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.817+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81f30d0 con 0x7f0db80c4d50 2024-04-25T12:35:48.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.818+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db814ec50 con 0x7f0db80c4d50 2024-04-25T12:35:48.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.819+0000 7f0dafe00640 1 -- 172.21.15.123:0/3403352133 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0da4003610 con 0x7f0db80c4d50 2024-04-25T12:35:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.821+0000 7f0dafe00640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0d980744e0 0x7f0d980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.821+0000 7f0dafe00640 1 -- 172.21.15.123:0/3403352133 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6619+0+0 (secure 0 0 0) 0x7f0da40b5e00 con 0x7f0db80c4d50 2024-04-25T12:35:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.821+0000 7f0db7e00640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0d980744e0 0x7f0d980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.822+0000 7f0db7e00640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0d980744e0 0x7f0d980769f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009aa0 tx=0x7f0da8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:48.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:48.825+0000 7f0dafe00640 1 -- 172.21.15.123:0/3403352133 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0db814ec50 con 0x7f0db80c4d50 2024-04-25T12:35:48.855 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:48.855+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:48.956 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:48.956+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:49.021 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:49.022 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:49.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:49.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.041+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f0db81f2900 con 0x7f0db80c4d50 2024-04-25T12:35:49.084 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:49.084+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:49.158 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:49.158+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:49.260 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:49.259+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:49.354 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:49.354 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:49.360 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:49.360+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:49.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:49.467 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:49.467+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:49.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.517+0000 7f0dafe00640 1 -- 172.21.15.123:0/3403352133 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v43) ==== 87+0+0 (secure 0 0 0) 0x7f0da404c020 con 0x7f0db80c4d50 2024-04-25T12:35:49.517 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:35:49.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.521+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0d980744e0 msgr2=0x7f0d980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:49.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0d980744e0 0x7f0d980769f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009aa0 tx=0x7f0da8009190 comp rx=0 tx=0).stop 2024-04-25T12:35:49.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db80c4d50 msgr2=0x7f0db814ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:49.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db80c4d50 0x7f0db814ac10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000c00 tx=0x7f0da40046c0 comp rx=0 tx=0).stop 2024-04-25T12:35:49.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 shutdown_connections 2024-04-25T12:35:49.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0d980744e0 0x7f0d980769f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:49.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db80c8c60 0x7f0db814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:49.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0db80c4d50 0x7f0db814ac10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:49.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 --2- 172.21.15.123:0/3403352133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0db8074870 0x7f0db81f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:49.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 >> 172.21.15.123:0/3403352133 conn(0x7f0db806c750 msgr2=0x7f0db8073030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:49.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 shutdown_connections 2024-04-25T12:35:49.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:49.522+0000 7f0dbf200640 1 -- 172.21.15.123:0/3403352133 wait complete. 2024-04-25T12:35:49.602 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:49.601+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:49.662 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:49.663+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:49.699 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:49.699 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:49.721 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:49.763 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:49.763+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:49.863 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:49.864+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:49.964 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:49.965+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:49.988 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:50.042 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:50.042 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:50.064 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:50.065 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:50.065+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:50.166 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:50.166+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:50.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.251+0000 7fa983a00640 1 Processor -- start 2024-04-25T12:35:50.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.252+0000 7fa983a00640 1 -- start start 2024-04-25T12:35:50.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa983a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa97c0c5df0 0x7fa97c15b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa983a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0ca270 0x7fa97c0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa983a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97c06d1d0 con 0x7fa97c0711e0 2024-04-25T12:35:50.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa983a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa97c06cf80 con 0x7fa97c0c5df0 2024-04-25T12:35:50.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa983a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa97c15a4e0 con 0x7fa97c0ca270 2024-04-25T12:35:50.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa981000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa97c0c5df0 0x7fa97c15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa973e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa97c0711e0 0x7fa97c15e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38604/0 (socket says 172.21.15.123:38604) 2024-04-25T12:35:50.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa973e00640 1 -- 172.21.15.123:0/930285354 learned_addr learned my addr 172.21.15.123:0/930285354 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:50.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.254+0000 7fa981a00640 1 --2- 172.21.15.123:0/930285354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0ca270 0x7fa97c0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.255+0000 7fa981000640 1 -- 172.21.15.123:0/930285354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0ca270 msgr2=0x7fa97c0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:50.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.255+0000 7fa981000640 1 --2- 172.21.15.123:0/930285354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0ca270 0x7fa97c0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:50.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.255+0000 7fa981000640 1 -- 172.21.15.123:0/930285354 >> v1:172.21.15.123:6789/0 conn(0x7fa97c0711e0 legacy=0x7fa97c15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:50.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.255+0000 7fa981000640 1 -- 172.21.15.123:0/930285354 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c0708b0 con 0x7fa97c0c5df0 2024-04-25T12:35:50.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.255+0000 7fa97be00640 1 -- 172.21.15.123:0/930285354 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335654611 0 0) 0x7fa97c06d1d0 con 0x7fa97c0711e0 2024-04-25T12:35:50.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.256+0000 7fa981000640 1 --2- 172.21.15.123:0/930285354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 0x7fa97c15b100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0097a0 tx=0x7fa96c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a28b22b62f8d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.256+0000 7fa97be00640 1 -- 172.21.15.123:0/930285354 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c037070 con 0x7fa97c0c5df0 2024-04-25T12:35:50.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.256+0000 7fa97be00640 1 -- 172.21.15.123:0/930285354 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c032440 con 0x7fa97c0c5df0 2024-04-25T12:35:50.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.256+0000 7fa97be00640 1 -- 172.21.15.123:0/930285354 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c032810 con 0x7fa97c0c5df0 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 -- 172.21.15.123:0/930285354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 msgr2=0x7fa97c15b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 --2- 172.21.15.123:0/930285354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 0x7fa97c15b100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0097a0 tx=0x7fa96c02a210 comp rx=0 tx=0).stop 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 -- 172.21.15.123:0/930285354 shutdown_connections 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 --2- 172.21.15.123:0/930285354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0ca270 0x7fa97c0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 --2- 172.21.15.123:0/930285354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 0x7fa97c15b100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 -- 172.21.15.123:0/930285354 >> 172.21.15.123:0/930285354 conn(0x7fa97c071b90 msgr2=0x7fa97c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 -- 172.21.15.123:0/930285354 shutdown_connections 2024-04-25T12:35:50.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.257+0000 7fa983a00640 1 -- 172.21.15.123:0/930285354 wait complete. 2024-04-25T12:35:50.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.260+0000 7fa983a00640 1 Processor -- start 2024-04-25T12:35:50.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.260+0000 7fa983a00640 1 -- start start 2024-04-25T12:35:50.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.261+0000 7fa983a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0711e0 0x7fa97c1f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.261+0000 7fa983a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 0x7fa97c1f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.261+0000 7fa983a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa97c0ca270 0x7fa97c1e9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.261+0000 7fa983a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c15e820 con 0x7fa97c0c5df0 2024-04-25T12:35:50.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.261+0000 7fa983a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c15dbf0 con 0x7fa97c0ca270 2024-04-25T12:35:50.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.261+0000 7fa983a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa97c15e9a0 con 0x7fa97c0711e0 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 0x7fa97c1f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 0x7fa97c1f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59264/0 (socket says 172.21.15.123:59264) 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 -- 172.21.15.123:0/1808272812 learned_addr learned my addr 172.21.15.123:0/1808272812 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0711e0 0x7fa97c1f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa981a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa97c0ca270 0x7fa97c1e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 -- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0711e0 msgr2=0x7fa97c1f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0711e0 0x7fa97c1f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 -- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa97c0ca270 msgr2=0x7fa97c1e9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:50.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa97c0ca270 0x7fa97c1e9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c15b170 con 0x7fa97c0c5df0 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa97c0711e0 0x7fa97c1f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.262+0000 7fa973e00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa97c0c5df0 0x7fa97c1f2df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c410 tx=0x7fa96400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.263+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9640074e0 con 0x7fa97c0c5df0 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.263+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9640047e0 con 0x7fa97c0c5df0 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.263+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c15bc60 con 0x7fa97c0c5df0 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.263+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964004bd0 con 0x7fa97c0c5df0 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.263+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c1f2770 con 0x7fa97c0c5df0 2024-04-25T12:35:50.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.264+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa97c076700 con 0x7fa97c0c5df0 2024-04-25T12:35:50.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.266+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa964011070 con 0x7fa97c0c5df0 2024-04-25T12:35:50.266 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:50.267+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:50.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.267+0000 7fa979600640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9580744b0 0x7fa9580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.267+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6619+0+0 (secure 0 0 0) 0x7fa9640950b0 con 0x7fa97c0c5df0 2024-04-25T12:35:50.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.268+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9580744b0 0x7fa9580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.268+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v43) ==== 54+0+16 (secure 0 0 0) 0x7fa964097650 con 0x7fa97c0c5df0 2024-04-25T12:35:50.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.268+0000 7fa983a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa97c0c7600 0x7fa97c1e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.268+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa97c070fc0 con 0x7fa97c0c7600 2024-04-25T12:35:50.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.268+0000 7fa981a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa97c0c7600 0x7fa97c1e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.269+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa9580744b0 0x7fa9580769c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa96c034000 tx=0x7fa96c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.269+0000 7fa981a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa97c0c7600 0x7fa97c1e7ac0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.270+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c070fc0 con 0x7fa97c0c7600 2024-04-25T12:35:50.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.288+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa97c072140 con 0x7fa97c0c7600 2024-04-25T12:35:50.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.290+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa97c072140 con 0x7fa97c0c7600 2024-04-25T12:35:50.291 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.291+0000 7fa983a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa97c1e8510 0x7fa97c062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.291+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa97c1e8920 con 0x7fa97c1e8510 2024-04-25T12:35:50.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.291+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa97c1e8510 0x7fa97c062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.292+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa97c1e8510 0x7fa97c062fe0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.293+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c1e8920 con 0x7fa97c1e8510 2024-04-25T12:35:50.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.309+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa97c15aa60 con 0x7fa97c1e8510 2024-04-25T12:35:50.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.311+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa97c15aa60 con 0x7fa97c1e8510 2024-04-25T12:35:50.312 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:50.312 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:50.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.312+0000 7fa983a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa97c063a30 0x7fa97c0ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.312+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa97c063e40 con 0x7fa97c063a30 2024-04-25T12:35:50.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.312+0000 7fa973e00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa97c063a30 0x7fa97c0ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.313+0000 7fa973e00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa97c063a30 0x7fa97c0ca680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.314+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c063e40 con 0x7fa97c063a30 2024-04-25T12:35:50.320 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:50.320 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.329+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa97c15c210 con 0x7fa97c063a30 2024-04-25T12:35:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.332+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa97c15c210 con 0x7fa97c063a30 2024-04-25T12:35:50.332 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.332+0000 7fa983a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa97c0cb0d0 0x7fa97c02c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.332+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa97c15c420 con 0x7fa97c0cb0d0 2024-04-25T12:35:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.333+0000 7fa981a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa97c0cb0d0 0x7fa97c02c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.334+0000 7fa981a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa97c0cb0d0 0x7fa97c02c9d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.335+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c15c420 con 0x7fa97c0cb0d0 2024-04-25T12:35:50.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:50.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.351+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa97c1f3330 con 0x7fa97c0cb0d0 2024-04-25T12:35:50.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.353+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa97c1f3330 con 0x7fa97c0cb0d0 2024-04-25T12:35:50.354 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:50.354 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:50.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.354+0000 7fa983a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa97c02d420 0x7fa97c02d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.354+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa97c1f3330 con 0x7fa97c02d420 2024-04-25T12:35:50.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.354+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa97c02d420 0x7fa97c02d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.355+0000 7fa981000640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa97c02d420 0x7fa97c02d830 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.356+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c1f3330 con 0x7fa97c02d420 2024-04-25T12:35:50.368 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:50.368+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:50.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.371+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa97c1f3330 con 0x7fa97c02d420 2024-04-25T12:35:50.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.373+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa97c1f3330 con 0x7fa97c02d420 2024-04-25T12:35:50.373 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:50.374 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:50.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.373+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=44}) -- 0x7fa97c02dd70 con 0x7fa97c0c5df0 2024-04-25T12:35:50.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.373+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa97c1f3330 con 0x7fa97c0c5df0 2024-04-25T12:35:50.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.373+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa96409c070 con 0x7fa97c0c5df0 2024-04-25T12:35:50.374 INFO:teuthology.orchestra.run.smithi123.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-04-25T12:35:50.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.374+0000 7fa983a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa97c207540 0x7fa97c209960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.374+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fa97c20c730 con 0x7fa97c207540 2024-04-25T12:35:50.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.374+0000 7fa973e00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa97c207540 0x7fa97c209960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.375+0000 7fa973e00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa97c207540 0x7fa97c209960 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.376+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c20c730 con 0x7fa97c207540 2024-04-25T12:35:50.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.391+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa97c209ea0 con 0x7fa97c207540 2024-04-25T12:35:50.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.393+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa97c209ea0 con 0x7fa97c207540 2024-04-25T12:35:50.393 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:50.394 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:50.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.393+0000 7fa983a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa97c20a520 0x7fa97c20c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:50.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.393+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fa97c20f820 con 0x7fa97c20a520 2024-04-25T12:35:50.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.394+0000 7fa981a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa97c20a520 0x7fa97c20c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:50.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.394+0000 7fa981a00640 1 --2- 172.21.15.123:0/1808272812 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa97c20a520 0x7fa97c20c940 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:50.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.395+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c20f820 con 0x7fa97c20a520 2024-04-25T12:35:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.411+0000 7fa983a00640 1 -- 172.21.15.123:0/1808272812 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa97c20f7f0 con 0x7fa97c20a520 2024-04-25T12:35:50.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:50.414+0000 7fa979600640 1 -- 172.21.15.123:0/1808272812 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa97c20f7f0 con 0x7fa97c20a520 2024-04-25T12:35:50.414 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:50.414 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:50.428 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T12:35:50.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-25T12:35:50.469 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:50.470+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:50.570 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:50.570+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:50.594 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:50.594 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:50.594 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:50.594 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:50.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:50.671 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:50.671+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:50.771 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:50.771+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:50.872 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:50.872+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:50.933 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:50.933 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:50.933 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:50.934 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:50.934 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:50.956 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:50.973 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:50.974+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:51.074 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:51.074+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:51.175 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:51.175+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:51.276 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:51.276+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:51.278 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.325642670999997, 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:35:51.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:35:51.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.324958488999997, 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:35:51.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.286 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.323507948, 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:35:51.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.323247132999995, 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:35:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:35:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.319396745999995, 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:35:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:35:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.319174378, 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:35:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:35:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.318820588999998, 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.158201640999998, 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:35:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157994148999997, 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:35:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157849659, 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:35:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157717865000002, 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:35:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157643569999998, 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:35:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157543828999998, 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:35:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157474366999999, 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:35:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:35:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157374253, 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:35:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:35:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157277162, 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157186756999998, 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.157038784000001, 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:35:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:35:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:35:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.156931379, 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.122680080999999, 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:35:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:35:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:35:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:51.370 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:51.371 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:51.412 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:51.412+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:51.478 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:51.478+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:51.496 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:51.497 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:51.497 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:51.497 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:51.506 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:51.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:35:51.611 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:51.611+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:51.619 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:51.619 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:51.619 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:51.619 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:51.628 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:51.679 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:51.680+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:51.741 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.792046459999995, 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:35:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.744 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:35:51.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:51.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791958750999996, 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.747 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.748 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:35:51.750 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.751 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.752 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791832822999993, 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.753 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.754 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:35:51.755 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.756 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.757 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791771163999996, 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.758 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:35:51.759 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:35:51.760 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:35:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.762 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791714400999993, 2024-04-25T12:35:51.763 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.764 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.765 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:35:51.766 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.767 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:51.768 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791650931000007, 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.769 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.770 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.771 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.772 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:51.773 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791598981999996, 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:35:51.775 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.776 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.777 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.778 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791547031999997, 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.779 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:35:51.780 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:35:51.781 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.782 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.783 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791429922999995, 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.784 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:35:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:35:51.786 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:51.787 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.788 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.791391476000001, 2024-04-25T12:35:51.789 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.790 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:35:51.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:35:51.792 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 61.626256423000001, 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:35:51.793 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.794 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.795 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.796 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.797 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.798 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 61.626198371999998, 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.799 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.800 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.802 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.803 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 61.626138679, 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:51.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:35:51.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.589218596999999, 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:35:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.589129630999999, 2024-04-25T12:35:51.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.817 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.818 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:51.819 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.589085789999999, 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:51.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:35:51.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:35:51.822 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:35:51.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.587571885999999, 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.825 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.829 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.586652669999999, 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:51.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:35:51.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:35:51.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:35:51.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:35:51.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.586197317, 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.835 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.836 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.838 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 56.586168338999997, 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:51.839 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:51.840 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.841 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.842 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.843 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:51.843 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:51.843 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:51.844 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:51.845 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:51.844+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:51.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.855+0000 7f83db200640 1 Processor -- start 2024-04-25T12:35:51.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.855+0000 7f83db200640 1 -- start start 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83db200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83dc072d70 0x7f83dc152a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83db200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f83dc073bc0 0x7f83dc158900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83db200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc0c3d40 0x7f83dc0c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83db200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f83dc06eb60 con 0x7f83dc072d70 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83db200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f83dc06fc80 con 0x7f83dc073bc0 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83db200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f83dc06ff40 con 0x7f83dc0c3d40 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83d9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83dc072d70 0x7f83dc152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83d9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83dc072d70 0x7f83dc152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59272/0 (socket says 172.21.15.123:59272) 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83da800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc0c3d40 0x7f83dc0c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83d9e00640 1 -- 172.21.15.123:0/538310281 learned_addr learned my addr 172.21.15.123:0/538310281 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.857+0000 7f83d3e00640 1 --2- 172.21.15.123:0/538310281 >> v2:172.21.15.153:3300/0 conn(0x7f83dc073bc0 0x7f83dc158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.858+0000 7f83d9e00640 1 -- 172.21.15.123:0/538310281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc0c3d40 msgr2=0x7f83dc0c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.858+0000 7f83d9e00640 1 --2- 172.21.15.123:0/538310281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc0c3d40 0x7f83dc0c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.858+0000 7f83d9e00640 1 -- 172.21.15.123:0/538310281 >> v2:172.21.15.153:3300/0 conn(0x7f83dc073bc0 msgr2=0x7f83dc158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.858+0000 7f83d9e00640 1 --2- 172.21.15.123:0/538310281 >> v2:172.21.15.153:3300/0 conn(0x7f83dc073bc0 0x7f83dc158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.858+0000 7f83d9e00640 1 -- 172.21.15.123:0/538310281 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc152fd0 con 0x7f83dc072d70 2024-04-25T12:35:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.859+0000 7f83d9e00640 1 --2- 172.21.15.123:0/538310281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc072d70 0x7f83dc152a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83dc071ca0 tx=0x7f83cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff401d7597b9a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.859+0000 7f83d9400640 1 -- 172.21.15.123:0/538310281 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc037070 con 0x7f83dc072d70 2024-04-25T12:35:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.859+0000 7f83d9400640 1 -- 172.21.15.123:0/538310281 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc032440 con 0x7f83dc072d70 2024-04-25T12:35:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.859+0000 7f83d9400640 1 -- 172.21.15.123:0/538310281 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc032810 con 0x7f83dc072d70 2024-04-25T12:35:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 -- 172.21.15.123:0/538310281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc072d70 msgr2=0x7f83dc152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 --2- 172.21.15.123:0/538310281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc072d70 0x7f83dc152a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83dc071ca0 tx=0x7f83cc02a140 comp rx=0 tx=0).stop 2024-04-25T12:35:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 -- 172.21.15.123:0/538310281 shutdown_connections 2024-04-25T12:35:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 --2- 172.21.15.123:0/538310281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc0c3d40 0x7f83dc0c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 --2- 172.21.15.123:0/538310281 >> v2:172.21.15.153:3300/0 conn(0x7f83dc073bc0 0x7f83dc158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 --2- 172.21.15.123:0/538310281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc072d70 0x7f83dc152a90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 -- 172.21.15.123:0/538310281 >> 172.21.15.123:0/538310281 conn(0x7f83dc0cd600 msgr2=0x7f83dc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 -- 172.21.15.123:0/538310281 shutdown_connections 2024-04-25T12:35:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.860+0000 7f83db200640 1 -- 172.21.15.123:0/538310281 wait complete. 2024-04-25T12:35:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.863+0000 7f83db200640 1 Processor -- start 2024-04-25T12:35:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.863+0000 7f83db200640 1 -- start start 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.864+0000 7f83db200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc072d70 0x7f83dc1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.864+0000 7f83db200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc073bc0 0x7f83dc1e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.864+0000 7f83db200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc0c3d40 0x7f83dc14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.864+0000 7f83db200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc06f0e0 con 0x7f83dc0c3d40 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.864+0000 7f83db200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc06ee00 con 0x7f83dc073bc0 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.864+0000 7f83db200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f83dc0c9d60 con 0x7f83dc072d70 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc0c3d40 0x7f83dc14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc0c3d40 0x7f83dc14bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59280/0 (socket says 172.21.15.123:59280) 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 -- 172.21.15.123:0/1155082639 learned_addr learned my addr 172.21.15.123:0/1155082639 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83d3e00640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc073bc0 0x7f83dc1e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83d9e00640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc072d70 0x7f83dc1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 -- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc072d70 msgr2=0x7f83dc1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc072d70 0x7f83dc1f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 -- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc073bc0 msgr2=0x7f83dc1e22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc073bc0 0x7f83dc1e22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 -- 172.21.15.123:0/1155082639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc1f3cd0 con 0x7f83dc0c3d40 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.865+0000 7f83da800640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc0c3d40 0x7f83dc14bf20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009d30 tx=0x7f83d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.866+0000 7f83d2000640 1 -- 172.21.15.123:0/1155082639 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4018020 con 0x7f83dc0c3d40 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.866+0000 7f83d2000640 1 -- 172.21.15.123:0/1155082639 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d4014480 con 0x7f83dc0c3d40 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.866+0000 7f83d2000640 1 -- 172.21.15.123:0/1155082639 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4014870 con 0x7f83dc0c3d40 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.866+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc1e27f0 con 0x7f83dc0c3d40 2024-04-25T12:35:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.866+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc14f810 con 0x7f83dc0c3d40 2024-04-25T12:35:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.867+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83dc150340 con 0x7f83dc0c3d40 2024-04-25T12:35:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.869+0000 7f83d2000640 1 -- 172.21.15.123:0/1155082639 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83d401b020 con 0x7f83dc0c3d40 2024-04-25T12:35:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.870+0000 7f83d2000640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f83b80744e0 0x7f83b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.870+0000 7f83d2000640 1 -- 172.21.15.123:0/1155082639 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6619+0+0 (secure 0 0 0) 0x7f83d4096270 con 0x7f83dc0c3d40 2024-04-25T12:35:51.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.870+0000 7f83d9e00640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f83b80744e0 0x7f83b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:51.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.871+0000 7f83d9e00640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f83b80744e0 0x7f83b80769f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f83cc034000 tx=0x7f83cc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:51.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:51.876+0000 7f83d2000640 1 -- 172.21.15.123:0/1155082639 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83dc150340 con 0x7f83dc0c3d40 2024-04-25T12:35:51.945 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:51.945+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:51.997 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:51.997 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:51.997 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:51.997 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:52.006 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:52.046 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:52.047+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:52.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.099+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f83dc14f600 con 0x7f83dc0c3d40 2024-04-25T12:35:52.146 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:52.147+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:52.158 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:52.158 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:52.158 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:52.158 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:52.158 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:52.167 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:52.248 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:52.249+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 24/18 rep_tid 1 entries 24'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.366850420000006, 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075768199999999997, 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.322 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954840+0000", 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6869999999999999e-06 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954864+0000", 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3824000000000001e-05 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954902+0000", 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500999999999999e-05 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954919+0000", 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7618e-05 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955011+0000", 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2250000000000006e-05 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955596+0000", 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058480200000000004 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:25 2.1b e25/18 2:dbae76b5:::mds1_inotable:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.205165338, 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036130300000000001, 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116525+0000", 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.455e-06 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116539+0000", 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4e-05 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116563+0000", 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4111999999999999e-05 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116577+0000", 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4059999999999999e-05 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116689+0000", 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011209000000000001 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116719+0000", 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.989e-05 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120082+0000", 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033628130000000001 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120137+0000", 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4610000000000001e-05 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.329 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 4 entries 25'3 (0'0) error 2:db457a8a:::201.00000001:head by mds.1.9:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.204108797000004, 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030622480000000001, 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117581+0000", 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0279999999999999e-06 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117596+0000", 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5109999999999999e-05 2024-04-25T12:35:52.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117631+0000", 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4230000000000003e-05 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117655+0000", 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4987e-05 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117751+0000", 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5311999999999998e-05 2024-04-25T12:35:52.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120642+0000", 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028915809999999998 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 5 entries 25'4 (0'0) error 2:d80f8093:::201.00000002:head by mds.1.9:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.203607566000002, 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0025905469999999999, 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.333 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118082+0000", 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4000000000000001e-07 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118091+0000", 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9919999999999996e-06 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118108+0000", 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7207999999999999e-05 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118124+0000", 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5378000000000001e-05 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118203+0000", 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8929000000000003e-05 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120672+0000", 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024692999999999998 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.336 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:34 2.1b e25/18 2:db457a8a:::201.00000001:head v 25'5, mlcod=25'3)", 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.200133231999999, 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016516759999999999, 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:52.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121557+0000", 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.423e-06 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121568+0000", 2024-04-25T12:35:52.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0499000000000001e-05 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121585+0000", 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7283e-05 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121597+0000", 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1821000000000001e-05 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121653+0000", 2024-04-25T12:35:52.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6533000000000002e-05 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121671+0000", 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7285000000000001e-05 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123158+0000", 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0014869519999999999 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123207+0000", 2024-04-25T12:35:52.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9879999999999997e-05 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:35 2.a 2:54ba6923:::mds1_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.178020914999998, 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016436459999999999, 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.341 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143670+0000", 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4040000000000002e-06 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143685+0000", 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4768e-05 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143703+0000", 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7424000000000001e-05 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143721+0000", 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8685000000000002e-05 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143824+0000", 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010286800000000001 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144032+0000", 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020812700000000001 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144205+0000", 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017276999999999999 2024-04-25T12:35:52.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144982+0000", 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077719300000000002 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145196+0000", 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000213308 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145254+0000", 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8619000000000002e-05 2024-04-25T12:35:52.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145312+0000", 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7479999999999999e-05 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.1.9:35 2.a e25/18 ondisk, result = 0)", 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.176594348000002, 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00022686700000000001, 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.346 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145096+0000", 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2309999999999999e-06 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145102+0000", 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1639999999999998e-06 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145115+0000", 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2809000000000001e-05 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145124+0000", 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9989999999999997e-06 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145178+0000", 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4287999999999999e-05 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145189+0000", 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.084e-05 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145321+0000", 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013253599999999999 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:11 2.c 2:36924187:::618.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.169345082, 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070050249999999998, 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.350 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152345+0000", 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6270000000000001e-06 2024-04-25T12:35:52.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152355+0000", 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4069999999999999e-06 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152384+0000", 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9479e-05 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152399+0000", 2024-04-25T12:35:52.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152478+0000", 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9702000000000004e-05 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152615+0000", 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013665500000000001 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152730+0000", 2024-04-25T12:35:52.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011504499999999999 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156394+0000", 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036643259999999999 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028370299999999999 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159289+0000", 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7268999999999997e-05 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159349+0000", 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0161999999999998e-05 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.169258507999999, 2024-04-25T12:35:52.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070895239999999998, 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152431+0000", 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6899999999999996e-07 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152440+0000", 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3259999999999996e-06 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152450+0000", 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0874e-05 2024-04-25T12:35:52.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152470+0000", 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9658999999999999e-05 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153265+0000", 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079454099999999995 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153349+0000", 2024-04-25T12:35:52.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4381000000000004e-05 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153485+0000", 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136365 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156400+0000", 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029144990000000001 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159418+0000", 2024-04-25T12:35:52.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030182299999999998 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159477+0000", 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159520+0000", 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3127000000000002e-05 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.169194894, 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0068584939999999997, 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152495+0000", 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.16e-07 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152501+0000", 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7819999999999999e-06 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152510+0000", 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9199999999999993e-06 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152518+0000", 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9090000000000006e-06 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152651+0000", 2024-04-25T12:35:52.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000132349 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152795+0000", 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014392800000000001 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152930+0000", 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013561399999999999 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156365+0000", 2024-04-25T12:35:52.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034348830000000001 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159266+0000", 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029006959999999999 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159296+0000", 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0026e-05 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159353+0000", 2024-04-25T12:35:52.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6570999999999998e-05 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.169108930999997, 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069239999999999996, 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152581+0000", 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3399999999999998e-07 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152587+0000", 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4049999999999997e-06 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152598+0000", 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1043000000000001e-05 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152607+0000", 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9409999999999996e-06 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153448+0000", 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00084054699999999997 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153563+0000", 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000115584 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153719+0000", 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015566199999999999 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156373+0000", 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002653617 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159423+0000", 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030508060000000001 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159472+0000", 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9042000000000002e-05 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159504+0000", 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1519000000000001e-05 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.169026047999999, 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0086667419999999998, 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.371 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152664+0000", 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0100000000000003e-07 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152671+0000", 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8870000000000003e-06 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152679+0000", 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2730000000000002e-06 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152696+0000", 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7373000000000001e-05 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153747+0000", 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001050647 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153967+0000", 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00021982699999999999 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154122+0000", 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015472399999999999 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156405+0000", 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022838110000000002 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161248+0000", 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048425669999999999 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161283+0000", 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5166000000000002e-05 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161330+0000", 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6566000000000001e-05 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.2 epoch 25/18 rep_tid 20 entries 25'1 (0'0) error 2:40dbbc66:::202.00000003:head by mds.2.13:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:52.376 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.167420167000003, 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0021562719999999999, 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:35:52.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154270+0000", 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.638e-06 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154290+0000", 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392e-05 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154315+0000", 2024-04-25T12:35:52.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4858000000000002e-05 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154335+0000", 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0044e-05 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154711+0000", 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037622099999999999 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156425+0000", 2024-04-25T12:35:52.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017141190000000001 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:11 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.162564273000001, 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00023418899999999999, 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159126+0000", 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7699999999999992e-07 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159131+0000", 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8109999999999999e-06 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159143+0000", 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1745e-05 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159153+0000", 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6520000000000001e-06 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159211+0000", 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8582999999999998e-05 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159225+0000", 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3088000000000001e-05 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159359+0000", 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013433299999999999 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.383 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:15 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.162513877000002, 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000189507, 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159176+0000", 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6899999999999998e-07 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159180+0000", 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1320000000000004e-06 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159188+0000", 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9319999999999996e-06 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159199+0000", 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1165999999999999e-05 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.386 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5071000000000002e-05 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159262+0000", 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0160000000000008e-06 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159365+0000", 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010272099999999999 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:12 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.162474947, 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000315354, 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2099999999999998e-07 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159217+0000", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.875e-06 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159223+0000", 2024-04-25T12:35:52.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2999999999999998e-06 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159229+0000", 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6740000000000002e-06 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159401+0000", 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017201599999999999 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159412+0000", 2024-04-25T12:35:52.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0349e-05 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159529+0000", 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000117819 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:16 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.162446678000002, 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00026996799999999999, 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.391 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.392 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159243+0000", 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0100000000000001e-07 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159246+0000", 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.687e-06 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.072e-06 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159261+0000", 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2169999999999999e-06 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159408+0000", 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000147637 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159417+0000", 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999996e-06 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159512+0000", 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5163999999999999e-05 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:23 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.160532523000001, 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000182072, 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161157+0000", 2024-04-25T12:35:52.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7400000000000002e-07 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161164+0000", 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161180+0000", 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6189999999999999e-05 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161191+0000", 2024-04-25T12:35:52.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0981e-05 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161238+0000", 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7509999999999997e-05 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161244+0000", 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.694e-06 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161338+0000", 2024-04-25T12:35:52.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4432999999999995e-05 2024-04-25T12:35:52.399 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.399 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.399 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.399 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.399 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.399 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:52.400 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:52.401 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:52.401+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:52.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:52.501 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:52.502+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:52.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.525+0000 7f83d2000640 1 -- 172.21.15.123:0/1155082639 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v44) ==== 97+0+0 (secure 0 0 0) 0x7f83d4022020 con 0x7f83dc0c3d40 2024-04-25T12:35:52.524 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:35:52.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.531+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f83b80744e0 msgr2=0x7f83b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:52.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.531+0000 7f83db200640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f83b80744e0 0x7f83b80769f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f83cc034000 tx=0x7f83cc02e000 comp rx=0 tx=0).stop 2024-04-25T12:35:52.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.531+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc0c3d40 msgr2=0x7f83dc14bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:52.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.531+0000 7f83db200640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc0c3d40 0x7f83dc14bf20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009d30 tx=0x7f83d400c490 comp rx=0 tx=0).stop 2024-04-25T12:35:52.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.532+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 shutdown_connections 2024-04-25T12:35:52.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.532+0000 7f83db200640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f83b80744e0 0x7f83b80769f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.532+0000 7f83db200640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83dc0c3d40 0x7f83dc14bf20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.532+0000 7f83db200640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc073bc0 0x7f83dc1e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.532+0000 7f83db200640 1 --2- 172.21.15.123:0/1155082639 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83dc072d70 0x7f83dc1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.532+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 >> 172.21.15.123:0/1155082639 conn(0x7f83dc0cd600 msgr2=0x7f83dc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:52.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.532+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 shutdown_connections 2024-04-25T12:35:52.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.533+0000 7f83db200640 1 -- 172.21.15.123:0/1155082639 wait complete. 2024-04-25T12:35:52.554 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:52.554 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:52.554 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:52.554 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:52.564 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:52.605 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:52.606+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:52.676 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:52.677 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:52.677 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:52.677 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:52.677 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:52.686 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:52.703 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:52.704+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:52.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.724+0000 7fd035e00640 1 Processor -- start 2024-04-25T12:35:52.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.724+0000 7fd035e00640 1 -- start start 2024-04-25T12:35:52.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd035e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd03015d1d0 0x7fd030156560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd035e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd0300c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd035e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0300c8790 con 0x7fd0300c5180 2024-04-25T12:35:52.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd035e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd0300c8a30 con 0x7fd03015d1d0 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd035e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd0300c6820 con 0x7fd030074780 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd027e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd0300c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd034a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd03015d1d0 0x7fd030156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd027e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd0300c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:41344/0 (socket says 172.21.15.123:41344) 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.726+0000 7fd027e00640 1 -- 172.21.15.123:0/3142788524 learned_addr learned my addr 172.21.15.123:0/3142788524 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.727+0000 7fd027e00640 1 -- 172.21.15.123:0/3142788524 >> v2:172.21.15.123:3300/0 conn(0x7fd03015d1d0 msgr2=0x7fd030156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.727+0000 7fd027e00640 1 --2- 172.21.15.123:0/3142788524 >> v2:172.21.15.123:3300/0 conn(0x7fd03015d1d0 0x7fd030156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.727+0000 7fd027e00640 1 -- 172.21.15.123:0/3142788524 >> v1:172.21.15.123:6789/0 conn(0x7fd0300c5180 legacy=0x7fd0300c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:52.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.727+0000 7fd027e00640 1 -- 172.21.15.123:0/3142788524 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd030074020 con 0x7fd030074780 2024-04-25T12:35:52.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.728+0000 7fd027e00640 1 --2- 172.21.15.123:0/3142788524 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd0300c8f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd018009870 tx=0x7fd01802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b82f5e367ae9e95f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.728+0000 7fd02fe00640 1 -- 172.21.15.123:0/3142788524 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018037070 con 0x7fd030074780 2024-04-25T12:35:52.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.728+0000 7fd02fe00640 1 -- 172.21.15.123:0/3142788524 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01802ae50 con 0x7fd030074780 2024-04-25T12:35:52.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.728+0000 7fd02fe00640 1 -- 172.21.15.123:0/3142788524 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018032860 con 0x7fd030074780 2024-04-25T12:35:52.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 -- 172.21.15.123:0/3142788524 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 msgr2=0x7fd0300c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:52.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 --2- 172.21.15.123:0/3142788524 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd0300c8f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd018009870 tx=0x7fd01802a300 comp rx=0 tx=0).stop 2024-04-25T12:35:52.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 -- 172.21.15.123:0/3142788524 shutdown_connections 2024-04-25T12:35:52.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 --2- 172.21.15.123:0/3142788524 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd0300c8f80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 --2- 172.21.15.123:0/3142788524 >> v2:172.21.15.123:3300/0 conn(0x7fd03015d1d0 0x7fd030156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 -- 172.21.15.123:0/3142788524 >> 172.21.15.123:0/3142788524 conn(0x7fd0300ca780 msgr2=0x7fd0300cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:52.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 -- 172.21.15.123:0/3142788524 shutdown_connections 2024-04-25T12:35:52.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.729+0000 7fd035e00640 1 -- 172.21.15.123:0/3142788524 wait complete. 2024-04-25T12:35:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.732+0000 7fd035e00640 1 Processor -- start 2024-04-25T12:35:52.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.732+0000 7fd035e00640 1 -- start start 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.733+0000 7fd035e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd030152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd035e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0300c5180 0x7fd030151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd035e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd03015d1d0 0x7fd030151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd035e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd03015ccb0 con 0x7fd03015d1d0 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd035e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd03015c9d0 con 0x7fd0300c5180 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd035e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd0300c94c0 con 0x7fd030074780 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd030152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd030152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:41346/0 (socket says 172.21.15.123:41346) 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 -- 172.21.15.123:0/4200938381 learned_addr learned my addr 172.21.15.123:0/4200938381 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd035400640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd03015d1d0 0x7fd030151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0300c5180 0x7fd030151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 -- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0300c5180 msgr2=0x7fd030151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0300c5180 0x7fd030151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 -- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd03015d1d0 msgr2=0x7fd030151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd03015d1d0 0x7fd030151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd034a00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd030153220 con 0x7fd030074780 2024-04-25T12:35:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.734+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0300c5180 0x7fd030151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:52.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.735+0000 7fd034a00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd030074780 0x7fd030152ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd020009d60 tx=0x7fd02000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.735+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020018020 con 0x7fd030074780 2024-04-25T12:35:52.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.735+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd020014480 con 0x7fd030074780 2024-04-25T12:35:52.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.735+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd030159d80 con 0x7fd030074780 2024-04-25T12:35:52.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.735+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020014850 con 0x7fd030074780 2024-04-25T12:35:52.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.735+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd030151010 con 0x7fd030074780 2024-04-25T12:35:52.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.736+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd030076da0 con 0x7fd030074780 2024-04-25T12:35:52.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.738+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd02001b020 con 0x7fd030074780 2024-04-25T12:35:52.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.739+0000 7fd02d600640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd00c0744e0 0x7fd00c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.739+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 6619+0+0 (secure 0 0 0) 0x7fd020095f80 con 0x7fd030074780 2024-04-25T12:35:52.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.739+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd00c0744e0 0x7fd00c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.739+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v44) ==== 54+0+16 (secure 0 0 0) 0x7fd020022030 con 0x7fd030074780 2024-04-25T12:35:52.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.739+0000 7fd035e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd030157c00 0x7fd03006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.739+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd030160970 con 0x7fd030157c00 2024-04-25T12:35:52.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.740+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd00c0744e0 0x7fd00c0769f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd018034000 tx=0x7fd01802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.740+0000 7fd035400640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd030157c00 0x7fd03006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.740+0000 7fd035400640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd030157c00 0x7fd03006dd30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.741+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd030160970 con 0x7fd030157c00 2024-04-25T12:35:52.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.758+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd0300c9100 con 0x7fd030157c00 2024-04-25T12:35:52.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.761+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd0300c9100 con 0x7fd030157c00 2024-04-25T12:35:52.760 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:52.761 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:52.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.761+0000 7fd035e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fd0301533f0 0x7fd0301fa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.761+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd0301e5eb0 con 0x7fd0301533f0 2024-04-25T12:35:52.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.762+0000 7fd034a00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fd0301533f0 0x7fd0301fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.762+0000 7fd034a00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fd0301533f0 0x7fd0301fa310 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.763+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0301e5eb0 con 0x7fd0301533f0 2024-04-25T12:35:52.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.780+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd030152720 con 0x7fd0301533f0 2024-04-25T12:35:52.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.782+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd030152720 con 0x7fd0301533f0 2024-04-25T12:35:52.781 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.782+0000 7fd035e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fd030148320 0x7fd030148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.782+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd030152930 con 0x7fd030148320 2024-04-25T12:35:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.783+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fd030148320 0x7fd030148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.784+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fd030148320 0x7fd030148730 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.784+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd030152930 con 0x7fd030148320 2024-04-25T12:35:52.801 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.850161295999996, 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:35:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.803 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:35:52.804 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:35:52.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:52.806 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.850073641999998, 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.807 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.808 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.809 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:35:52.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:35:52.811 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.849729073999995, 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.812 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:35:52.813 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:35:52.814 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.815 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:35:52.816 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.849656604000003, 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:52.817 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:35:52.818 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:35:52.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:35:52.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:35:52.821 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.849582626, 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.822 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:35:52.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:35:52.824 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:35:52.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:35:52.826 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.847545253999996, 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.827 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.828 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.829 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.830 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.846685117999996, 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.831 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.832 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.834 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.685999463000002, 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.835 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.836 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.837 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.838 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.839 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.685923348999999, 2024-04-25T12:35:52.840 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.841 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:35:52.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.843 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.844 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:52.845 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.685877679999997, 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.846 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.847 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.848 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.849 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.850 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.685832904999998, 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:52.851 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:35:52.852 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:35:52.853 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.679216480999997, 2024-04-25T12:35:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.855 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.856 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.857 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:52.858 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.679156990999999, 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.859 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.860 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.861 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.648186699, 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:52.862 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:35:52.863 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.864 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.865 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.866 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.648109187999999, 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:35:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:35:52.869 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:35:52.870 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.871 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.648061339000002, 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:35:52.873 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:35:52.874 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:35:52.875 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:35:52.876 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.648019449000003, 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.877 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.878 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.879 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.880 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.881 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.647985276999997, 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.882 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.883 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.884 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:35:52.885 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:35:52.886 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.647702535000001, 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:35:52.887 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.888 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.890 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 57.643562103999997, 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:52.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:52.894 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.805+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd030148ec0 con 0x7fd030148320 2024-04-25T12:35:52.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.810+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd030148ec0 con 0x7fd030148320 2024-04-25T12:35:52.896 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:52.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:52.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.811+0000 7fd035e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fd0300634f0 0x7fd030063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.811+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd030063e40 con 0x7fd0300634f0 2024-04-25T12:35:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.811+0000 7fd035400640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fd0300634f0 0x7fd030063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.812+0000 7fd035400640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fd0300634f0 0x7fd030063900 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.813+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd030063e40 con 0x7fd0300634f0 2024-04-25T12:35:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.830+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd030063e40 con 0x7fd0300634f0 2024-04-25T12:35:52.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.834+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd030063e40 con 0x7fd0300634f0 2024-04-25T12:35:52.898 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:52.899 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:52.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.834+0000 7fd035e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fd03015aef0 0x7fd03002c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.835+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd030063e40 con 0x7fd03015aef0 2024-04-25T12:35:52.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.835+0000 7fd034a00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fd03015aef0 0x7fd03002c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.836+0000 7fd034a00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fd03015aef0 0x7fd03002c9d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.837+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd030063e40 con 0x7fd03015aef0 2024-04-25T12:35:52.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.853+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd030063e40 con 0x7fd03015aef0 2024-04-25T12:35:52.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.855+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd030063e40 con 0x7fd03015aef0 2024-04-25T12:35:52.900 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.856+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=45}) -- 0x7fd03002d1e0 con 0x7fd030074780 2024-04-25T12:35:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.856+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd030063e40 con 0x7fd030074780 2024-04-25T12:35:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.856+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd02005f6b0 con 0x7fd030074780 2024-04-25T12:35:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-04-25T12:35:52.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.856+0000 7fd035e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fd03020b8a0 0x7fd03020dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.856+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fd030210b10 con 0x7fd03020b8a0 2024-04-25T12:35:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.857+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fd03020b8a0 0x7fd03020dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.857+0000 7fd027e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fd03020b8a0 0x7fd03020dcc0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.858+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd030210b10 con 0x7fd03020b8a0 2024-04-25T12:35:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.874+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd03020e200 con 0x7fd03020b8a0 2024-04-25T12:35:52.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.876+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd03020e200 con 0x7fd03020b8a0 2024-04-25T12:35:52.902 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:52.903 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:52.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.876+0000 7fd035e00640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fd03020e920 0x7fd03020ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:52.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.876+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fd030213ed0 con 0x7fd03020e920 2024-04-25T12:35:52.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.877+0000 7fd035400640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fd03020e920 0x7fd03020ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:52.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.877+0000 7fd035400640 1 --2- 172.21.15.123:0/4200938381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fd03020e920 0x7fd03020ed30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:52.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.878+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd030213ed0 con 0x7fd03020e920 2024-04-25T12:35:52.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.893+0000 7fd035e00640 1 -- 172.21.15.123:0/4200938381 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd03020f380 con 0x7fd03020e920 2024-04-25T12:35:52.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:52.896+0000 7fd02d600640 1 -- 172.21.15.123:0/4200938381 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd03020f380 con 0x7fd03020e920 2024-04-25T12:35:52.904 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:52.904 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:52.904 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:52.896+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:52.925 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T12:35:52.925 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-25T12:35:52.996 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:52.997+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:53.097 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:53.098+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:53.166 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:53.166 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:53.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:53.198 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:53.198+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:53.298 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:53.299+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:53.399 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:53.400+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:53.500 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:53.500+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:53.524 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:53.524 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:53.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:53.638 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:53.639+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:53.702 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:53.703+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:53.802 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:53.803+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:53.819 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-04-25T12:35:53.820 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:35:53.820 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-04-25T12:35:53.820 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T12:35:53.820 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T12:35:53.822 INFO:tasks.ceph.osd.5:Started 2024-04-25T12:35:53.823 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:35:53.849 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:53.849 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:53.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:53.913 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:53.914+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:no valid command found; 10 closest matches: 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:0 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:1 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:2 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:abort 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:assert 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:config diff 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:config diff get 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:config get 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:config help [] 2024-04-25T12:35:53.936 INFO:teuthology.orchestra.run.smithi153.stderr:config set ... 2024-04-25T12:35:53.937 INFO:teuthology.orchestra.run.smithi153.stderr:admin_socket: invalid command 2024-04-25T12:35:53.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:35:53.939 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-04-25T12:35:54.003 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:54.004+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:54.105 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:54.106+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:54.177 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:54.177 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:54.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:54.243 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:54.244+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:54.306 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:54.307+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:54.407 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:54.408+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:54.425 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:54.425+0000 7f2d71415640 -1 Falling back to public interface 2024-04-25T12:35:54.453 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:54.453 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:54.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:54.507 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:54.508+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:54.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:35:54.608 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:54.609+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:54.709 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:54.710+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:54.769 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:54.769 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:54.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:35:54.810 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:54.810+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:54.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.821+0000 7f2c78e00640 1 Processor -- start 2024-04-25T12:35:54.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.821+0000 7f2c78e00640 1 -- start start 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.823+0000 7f2c78e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2c74074870 0x7f2c74154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c78e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7415afa0 0x7f2c740cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c78e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7415b3b0 con 0x7f2c740c4d50 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c78e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f2c740c61f0 con 0x7f2c74074870 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c78e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2c740c87d0 con 0x7f2c7415afa0 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c72800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7415afa0 0x7f2c740cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c71400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2c74074870 0x7f2c74154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:54.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c72800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7415afa0 0x7f2c740cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:40456/0 (socket says 172.21.15.123:40456) 2024-04-25T12:35:54.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c72800640 1 -- 172.21.15.123:0/3058791521 learned_addr learned my addr 172.21.15.123:0/3058791521 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:54.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c71e00640 1 --1- 172.21.15.123:0/3058791521 >> v1:172.21.15.153:6789/0 conn(0x7f2c740c4d50 0x7f2c740c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:49912/0 (socket says 172.21.15.123:49912) 2024-04-25T12:35:54.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c72800640 1 -- 172.21.15.123:0/3058791521 >> v2:172.21.15.153:3300/0 conn(0x7f2c74074870 msgr2=0x7f2c74154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:54.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c72800640 1 --2- 172.21.15.123:0/3058791521 >> v2:172.21.15.153:3300/0 conn(0x7f2c74074870 0x7f2c74154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:54.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c72800640 1 -- 172.21.15.123:0/3058791521 >> v1:172.21.15.153:6789/0 conn(0x7f2c740c4d50 legacy=0x7f2c740c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:54.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.825+0000 7f2c72800640 1 -- 172.21.15.123:0/3058791521 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c74073ce0 con 0x7f2c7415afa0 2024-04-25T12:35:54.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.824+0000 7f2c70a00640 1 -- 172.21.15.123:0/3058791521 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2967351465 0 0) 0x7f2c7415b3b0 con 0x7f2c740c4d50 2024-04-25T12:35:54.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.825+0000 7f2c71400640 1 --2- 172.21.15.123:0/3058791521 >> v2:172.21.15.153:3300/0 conn(0x7f2c74074870 0x7f2c74154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:54.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.826+0000 7f2c72800640 1 --2- 172.21.15.123:0/3058791521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7415afa0 0x7f2c740cda90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009e50 tx=0x7f2c6c02a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e527e27aa66fe41a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:54.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.826+0000 7f2c70a00640 1 -- 172.21.15.123:0/3058791521 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c036070 con 0x7f2c7415afa0 2024-04-25T12:35:54.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.826+0000 7f2c70a00640 1 -- 172.21.15.123:0/3058791521 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c6c031600 con 0x7f2c7415afa0 2024-04-25T12:35:54.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.826+0000 7f2c70a00640 1 -- 172.21.15.123:0/3058791521 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0319d0 con 0x7f2c7415afa0 2024-04-25T12:35:54.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 -- 172.21.15.123:0/3058791521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7415afa0 msgr2=0x7f2c740cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:54.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 --2- 172.21.15.123:0/3058791521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7415afa0 0x7f2c740cda90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009e50 tx=0x7f2c6c02a7a0 comp rx=0 tx=0).stop 2024-04-25T12:35:54.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 -- 172.21.15.123:0/3058791521 shutdown_connections 2024-04-25T12:35:54.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 --2- 172.21.15.123:0/3058791521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7415afa0 0x7f2c740cda90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:54.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 --2- 172.21.15.123:0/3058791521 >> v2:172.21.15.153:3300/0 conn(0x7f2c74074870 0x7f2c74154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:54.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 -- 172.21.15.123:0/3058791521 >> 172.21.15.123:0/3058791521 conn(0x7f2c7406c750 msgr2=0x7f2c7406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:54.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 -- 172.21.15.123:0/3058791521 shutdown_connections 2024-04-25T12:35:54.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.827+0000 7f2c78e00640 1 -- 172.21.15.123:0/3058791521 wait complete. 2024-04-25T12:35:54.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.829+0000 7f2c78e00640 1 Processor -- start 2024-04-25T12:35:54.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.829+0000 7f2c78e00640 1 -- start start 2024-04-25T12:35:54.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c78e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c74074870 0x7f2c74154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c78e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c740c4d50 0x7f2c74151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c78e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7415afa0 0x7f2c7415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c78e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c740c8460 con 0x7f2c740c4d50 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c78e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c740742a0 con 0x7f2c7415afa0 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c78e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2c740c9370 con 0x7f2c74074870 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c71400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c740c4d50 0x7f2c74151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c72800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7415afa0 0x7f2c7415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c71e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c74074870 0x7f2c74154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c71400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c740c4d50 0x7f2c74151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41480/0 (socket says 172.21.15.123:41480) 2024-04-25T12:35:54.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.831+0000 7f2c71400640 1 -- 172.21.15.123:0/2665950839 learned_addr learned my addr 172.21.15.123:0/2665950839 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:54.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.832+0000 7f2c72800640 1 -- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c74074870 msgr2=0x7f2c74154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:54.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.832+0000 7f2c72800640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c74074870 0x7f2c74154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:54.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.832+0000 7f2c72800640 1 -- 172.21.15.123:0/2665950839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c740c4d50 msgr2=0x7f2c74151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.832+0000 7f2c72800640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c740c4d50 0x7f2c74151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.832+0000 7f2c72800640 1 -- 172.21.15.123:0/2665950839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c74151a80 con 0x7f2c7415afa0 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.832+0000 7f2c72800640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7415afa0 0x7f2c7415c290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c030000 tx=0x7f2c6c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.833+0000 7f2c6a000640 1 -- 172.21.15.123:0/2665950839 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c036050 con 0x7f2c7415afa0 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.833+0000 7f2c6a000640 1 -- 172.21.15.123:0/2665950839 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c6c042070 con 0x7f2c7415afa0 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.833+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7414e7c0 con 0x7f2c7415afa0 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.833+0000 7f2c6a000640 1 -- 172.21.15.123:0/2665950839 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c009510 con 0x7f2c7415afa0 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.833+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7414ea60 con 0x7f2c7415afa0 2024-04-25T12:35:54.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.834+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c74076700 con 0x7f2c7415afa0 2024-04-25T12:35:54.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.835+0000 7f2c6a000640 1 -- 172.21.15.123:0/2665950839 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2c6c007c10 con 0x7f2c7415afa0 2024-04-25T12:35:54.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.836+0000 7f2c6a000640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c540744b0 0x7f2c540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:54.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.836+0000 7f2c6a000640 1 -- 172.21.15.123:0/2665950839 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6619+0+0 (secure 0 0 0) 0x7f2c6c0b6250 con 0x7f2c7415afa0 2024-04-25T12:35:54.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.836+0000 7f2c71e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c540744b0 0x7f2c540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:54.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.837+0000 7f2c71e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c540744b0 0x7f2c540769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c64007fd0 tx=0x7f2c64007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:54.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:54.841+0000 7f2c6a000640 1 -- 172.21.15.123:0/2665950839 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c6c07ff60 con 0x7f2c7415afa0 2024-04-25T12:35:54.910 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:54.911+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:54.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:55.011 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:55.012+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:55.076 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:55.076 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:55.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.091+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f2c74069920 con 0x7f2c7415afa0 2024-04-25T12:35:55.101 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:35:55.148 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:55.149+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:55.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.211+0000 7fc839a00640 1 Processor -- start 2024-04-25T12:35:55.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.212+0000 7fc839a00640 1 -- start start 2024-04-25T12:35:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.213+0000 7fc839a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc834152c70 0x7fc834074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.213+0000 7fc839a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc8340c2790 0x7fc8340c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.213+0000 7fc839a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc834158bd0 con 0x7fc8340c3fe0 2024-04-25T12:35:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.213+0000 7fc839a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc834158eb0 con 0x7fc834152c70 2024-04-25T12:35:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.213+0000 7fc839a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc834159190 con 0x7fc8340c2790 2024-04-25T12:35:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc834152c70 0x7fc834074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc834152c70 0x7fc834074820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41484/0 (socket says 172.21.15.123:41484) 2024-04-25T12:35:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832a00640 1 -- 172.21.15.123:0/1091368429 learned_addr learned my addr 172.21.15.123:0/1091368429 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:55.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832000640 1 --2- 172.21.15.123:0/1091368429 >> v2:172.21.15.153:3300/0 conn(0x7fc8340c2790 0x7fc8340c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.215 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:55.214+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832a00640 1 -- 172.21.15.123:0/1091368429 >> v2:172.21.15.153:3300/0 conn(0x7fc8340c2790 msgr2=0x7fc8340c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832a00640 1 --2- 172.21.15.123:0/1091368429 >> v2:172.21.15.153:3300/0 conn(0x7fc8340c2790 0x7fc8340c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832a00640 1 -- 172.21.15.123:0/1091368429 >> v1:172.21.15.153:6789/0 conn(0x7fc8340c3fe0 legacy=0x7fc83415c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.214+0000 7fc832a00640 1 -- 172.21.15.123:0/1091368429 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc834159470 con 0x7fc834152c70 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.215+0000 7fc832a00640 1 --2- 172.21.15.123:0/1091368429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc834152c70 0x7fc834074820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc8340c7e30 tx=0x7fc82402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66d7e7e43651afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.216+0000 7fc831600640 1 -- 172.21.15.123:0/1091368429 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824037070 con 0x7fc834152c70 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.216+0000 7fc831600640 1 -- 172.21.15.123:0/1091368429 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc824032440 con 0x7fc834152c70 2024-04-25T12:35:55.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.216+0000 7fc831600640 1 -- 172.21.15.123:0/1091368429 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824032810 con 0x7fc834152c70 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 -- 172.21.15.123:0/1091368429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc834152c70 msgr2=0x7fc834074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 --2- 172.21.15.123:0/1091368429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc834152c70 0x7fc834074820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc8340c7e30 tx=0x7fc82402a1e0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 -- 172.21.15.123:0/1091368429 shutdown_connections 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 --2- 172.21.15.123:0/1091368429 >> v2:172.21.15.153:3300/0 conn(0x7fc8340c2790 0x7fc8340c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 --2- 172.21.15.123:0/1091368429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc834152c70 0x7fc834074820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 -- 172.21.15.123:0/1091368429 >> 172.21.15.123:0/1091368429 conn(0x7fc83406f440 msgr2=0x7fc83406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 -- 172.21.15.123:0/1091368429 shutdown_connections 2024-04-25T12:35:55.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.217+0000 7fc839a00640 1 -- 172.21.15.123:0/1091368429 wait complete. 2024-04-25T12:35:55.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.219+0000 7fc839a00640 1 Processor -- start 2024-04-25T12:35:55.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.219+0000 7fc839a00640 1 -- start start 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc839a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8340c2790 0x7fc834159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc839a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8340c3fe0 0x7fc8341460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc839a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc834152c70 0x7fc834143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc839a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc83415cc40 con 0x7fc8340c3fe0 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc839a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc83415cac0 con 0x7fc834152c70 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc839a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc83415a7d0 con 0x7fc8340c2790 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc832000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8340c3fe0 0x7fc8341460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc832000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8340c3fe0 0x7fc8341460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41494/0 (socket says 172.21.15.123:41494) 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc832000640 1 -- 172.21.15.123:0/1920710053 learned_addr learned my addr 172.21.15.123:0/1920710053 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc834152c70 0x7fc834143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.221+0000 7fc832a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8340c2790 0x7fc834159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc833400640 1 -- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8340c2790 msgr2=0x7fc834159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8340c2790 0x7fc834159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc833400640 1 -- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8340c3fe0 msgr2=0x7fc8341460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8340c3fe0 0x7fc8341460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc833400640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83415a300 con 0x7fc834152c70 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc832000640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8340c3fe0 0x7fc8341460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:55.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc834152c70 0x7fc834143ea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00c700 tx=0x7fc82c00cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c003b70 con 0x7fc834152c70 2024-04-25T12:35:55.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c003d10 con 0x7fc834152c70 2024-04-25T12:35:55.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c005310 con 0x7fc834152c70 2024-04-25T12:35:55.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.222+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8341f2ce0 con 0x7fc834152c70 2024-04-25T12:35:55.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.223+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8341f32a0 con 0x7fc834152c70 2024-04-25T12:35:55.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.224+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc834076700 con 0x7fc834152c70 2024-04-25T12:35:55.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.225+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc82c0054b0 con 0x7fc834152c70 2024-04-25T12:35:55.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.226+0000 7fc82aa00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8140744b0 0x7fc8140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.226+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6619+0+0 (secure 0 0 0) 0x7fc82c095250 con 0x7fc834152c70 2024-04-25T12:35:55.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.227+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v44) ==== 54+0+16 (secure 0 0 0) 0x7fc82c0977f0 con 0x7fc834152c70 2024-04-25T12:35:55.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.227+0000 7fc832a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8140744b0 0x7fc8140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.227+0000 7fc839a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc8341443e0 0x7fc8341478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.227+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8340c8d40 con 0x7fc8341443e0 2024-04-25T12:35:55.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.227+0000 7fc832000640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc8341443e0 0x7fc8341478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.227+0000 7fc832a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8140744b0 0x7fc8140769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc824034000 tx=0x7fc82402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.228+0000 7fc832000640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc8341443e0 0x7fc8341478b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.229+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8340c8d40 con 0x7fc8341443e0 2024-04-25T12:35:55.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.251+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc834074600 con 0x7fc8341443e0 2024-04-25T12:35:55.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.254+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc834074600 con 0x7fc8341443e0 2024-04-25T12:35:55.254 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:55.254 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:55.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.255+0000 7fc839a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc8341f6550 0x7fc8341f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.255+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc8341f6ea0 con 0x7fc8341f6550 2024-04-25T12:35:55.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.255+0000 7fc832a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc8341f6550 0x7fc8341f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.256+0000 7fc832a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc8341f6550 0x7fc8341f6960 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.257+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8341f6ea0 con 0x7fc8341f6550 2024-04-25T12:35:55.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.279+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc8341f3980 con 0x7fc8341f6550 2024-04-25T12:35:55.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.281+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc8341f3980 con 0x7fc8341f6550 2024-04-25T12:35:55.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:55.281 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:55.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.281+0000 7fc839a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc8340634f0 0x7fc834063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.282+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc834164f20 con 0x7fc8340634f0 2024-04-25T12:35:55.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.282+0000 7fc832000640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc8340634f0 0x7fc834063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.283+0000 7fc832000640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc8340634f0 0x7fc834063900 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.284+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc834164f20 con 0x7fc8340634f0 2024-04-25T12:35:55.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.301+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc834156f40 con 0x7fc8340634f0 2024-04-25T12:35:55.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.304+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc834156f40 con 0x7fc8340634f0 2024-04-25T12:35:55.303 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:55.304 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:55.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.305+0000 7fc839a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc834150a30 0x7fc834150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.305+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc834156600 con 0x7fc834150a30 2024-04-25T12:35:55.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.305+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc834150a30 0x7fc834150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.306+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc834150a30 0x7fc834150e40 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.306+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc834156600 con 0x7fc834150a30 2024-04-25T12:35:55.314 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:55.315+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:55.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.321+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc8341447f0 con 0x7fc834150a30 2024-04-25T12:35:55.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.323+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc8341447f0 con 0x7fc834150a30 2024-04-25T12:35:55.322 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:55.323 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:55.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.324+0000 7fc839a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc83402cee0 0x7fc83402d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.324+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc834146600 con 0x7fc83402cee0 2024-04-25T12:35:55.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.324+0000 7fc832a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc83402cee0 0x7fc83402d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.325+0000 7fc832a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc83402cee0 0x7fc83402d2f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.325+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc834146600 con 0x7fc83402cee0 2024-04-25T12:35:55.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.340+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc834146600 con 0x7fc83402cee0 2024-04-25T12:35:55.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.343+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc834146600 con 0x7fc83402cee0 2024-04-25T12:35:55.342 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.343+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=45}) -- 0x7fc834146600 con 0x7fc834152c70 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.343+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc83415bc40 con 0x7fc834152c70 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.343+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc82c09c070 con 0x7fc834152c70 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.344+0000 7fc839a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc8342073c0 0x7fc8342097e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.344+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fc83420c630 con 0x7fc8342073c0 2024-04-25T12:35:55.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.344+0000 7fc832000640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc8342073c0 0x7fc8342097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.345+0000 7fc832000640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc8342073c0 0x7fc8342097e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.346+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc83420c630 con 0x7fc8342073c0 2024-04-25T12:35:55.357 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:55.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:55.357 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:55.357 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:55.357 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.362+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc834209da0 con 0x7fc8342073c0 2024-04-25T12:35:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.364+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc834209da0 con 0x7fc8342073c0 2024-04-25T12:35:55.364 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.365+0000 7fc839a00640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc83420a420 0x7fc83420c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.365+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fc83420f8b0 con 0x7fc83420a420 2024-04-25T12:35:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.365+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc83420a420 0x7fc83420c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.366+0000 7fc833400640 1 --2- 172.21.15.123:0/1920710053 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc83420a420 0x7fc83420c870 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.366+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc83420f8b0 con 0x7fc83420a420 2024-04-25T12:35:55.377 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:35:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.383+0000 7fc839a00640 1 -- 172.21.15.123:0/1920710053 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc834000950 con 0x7fc83420a420 2024-04-25T12:35:55.387 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:55.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.385+0000 7fc82aa00640 1 -- 172.21.15.123:0/1920710053 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc834000950 con 0x7fc83420a420 2024-04-25T12:35:55.387 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:55.400 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T12:35:55.400 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-25T12:35:55.415 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:55.416+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:55.516 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:55.517+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:55.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.564+0000 7f2c6a000640 1 -- 172.21.15.123:0/2665950839 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v45) ==== 91+0+0 (secure 0 0 0) 0x7f2c6c084260 con 0x7f2c7415afa0 2024-04-25T12:35:55.563 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:35:55.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.566+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c540744b0 msgr2=0x7f2c540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:55.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.566+0000 7f2c78e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c540744b0 0x7f2c540769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c64007fd0 tx=0x7f2c64007de0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.566+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7415afa0 msgr2=0x7f2c7415c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:55.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.566+0000 7f2c78e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7415afa0 0x7f2c7415c290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c030000 tx=0x7f2c6c004a40 comp rx=0 tx=0).stop 2024-04-25T12:35:55.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 shutdown_connections 2024-04-25T12:35:55.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c540744b0 0x7f2c540769c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7415afa0 0x7f2c7415c290 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c740c4d50 0x7f2c74151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 --2- 172.21.15.123:0/2665950839 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c74074870 0x7f2c74154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:55.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 >> 172.21.15.123:0/2665950839 conn(0x7f2c7406c750 msgr2=0x7f2c740c7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:55.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 shutdown_connections 2024-04-25T12:35:55.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:55.567+0000 7f2c78e00640 1 -- 172.21.15.123:0/2665950839 wait complete. 2024-04-25T12:35:55.642 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:55.643+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:55.644 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.692097399999994, 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:35:55.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.691985457000001, 2024-04-25T12:35:55.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:35:55.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:35:55.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:35:55.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.654 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.691936656999999, 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:35:55.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.691865496999995, 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.664 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.691817084999997, 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.683621157000005, 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:35:55.668 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.577377158999994, 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:35:55.672 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:35:55.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.577195157999995, 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:35:55.677 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.682 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.525161170999993, 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.525085215000004, 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:35:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:35:55.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.692 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.520656474999996, 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:35:55.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:35:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:35:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.520474411999999, 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:35:55.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:35:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.520370048000004, 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:35:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.704 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.519094491000004, 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:55.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:35:55.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:35:55.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:35:55.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.518040267000003, 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:35:55.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:35:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.489174626999997, 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:55.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:35:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:35:55.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.716 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.767663132000003, 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:55.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:35:55.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:35:55.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:35:55.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:35:55.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.725102579999998, 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:55.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:35:55.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:35:55.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:48 2.14 2:2e2fa760:::200.00000001:head [write 5505~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.8527106490000005, 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010227916, 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790256+0000", 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7570000000000002e-06 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790280+0000", 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4456000000000001e-05 2024-04-25T12:35:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790319+0000", 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9103000000000002e-05 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790344+0000", 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4998999999999999e-05 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790557+0000", 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021251199999999999 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790572+0000", 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5003e-05 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.796879+0000", 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063072359999999999 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797129+0000", 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025002800000000001 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797609+0000", 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047965200000000001 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800188+0000", 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002578922 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800281+0000", 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3561999999999996e-05 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800387+0000", 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010545800000000001 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800480+0000", 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3227999999999996e-05 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:49 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:35:55.732 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.8524410679999992, 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011186097000000001, 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:35:55.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790524+0000", 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999998e-06 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790541+0000", 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7567e-05 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790566+0000", 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4338999999999999e-05 2024-04-25T12:35:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790581+0000", 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5123000000000001e-05 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790708+0000", 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012676699999999999 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790718+0000", 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-04-25T12:35:55.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.798785+0000", 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080674500000000003 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799024+0000", 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000239044 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799376+0000", 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035158299999999998 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800980+0000", 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016040939999999999 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801537+0000", 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055677400000000001 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801639+0000", 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102388 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:55.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801708+0000", 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8504999999999995e-05 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:55.738 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:55.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:35:55.740 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:55.740+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:55.840 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:55.841+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:55.941 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:55.942+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:56.042 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:56.043+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:56.084 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:56.085 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:56.085 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:35:56.085 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:56.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:35:56.149 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:56.150+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:56.244 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:56.245+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:35:56.344 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:56.345+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:56.397 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:56.397 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:35:56.398 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:35:56.398 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:35:56.398 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:56.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:35:56.458 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:56.459+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:56.547 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:56.548+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:56.648 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:56.649+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.764754674000002, 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:35:56.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:35:56.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:56.722 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.764070492000002, 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:35:56.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:35:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.762619951000005, 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.730 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.762359136000001, 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:56.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:35:56.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:35:56.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.758508749000001, 2024-04-25T12:35:56.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:35:56.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:35:56.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:35:56.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.758286381000005, 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:35:56.739 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.757932592000003, 2024-04-25T12:35:56.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.597313643999996, 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.748 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:35:56.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:35:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:35:56.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:35:56.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.597106151999995, 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.753 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:56.758 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596961661999998, 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:35:56.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:35:56.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:35:56.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596829868, 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:35:56.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:35:56.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:35:56.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:35:56.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.768 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.769 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:56.769 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596755572999996, 2024-04-25T12:35:56.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:35:56.769 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.769 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.769 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.769 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:35:56.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:35:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:35:56.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596655831999996, 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:35:56.775 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:56.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:35:56.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596586369999997, 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:35:56.780 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:35:56.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:35:56.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596486256000006, 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.785 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:56.790 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596389165000005, 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:35:56.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:35:56.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:35:56.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.795 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596298759999996, 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:56.799 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596150786999999, 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:35:56.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:35:56.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.596043382000005, 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.803 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:35:56.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:35:56.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:35:56.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.561792083999997, 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:35:56.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:35:56.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:35:56.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:35:56.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:35:56.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:35:56.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:35:56.812 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.812 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.812 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.812 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:35:56.812 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:35:56.812 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:35:56.812 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:35:56.813 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:56.771+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:56.871 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:56.872+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:56.940 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:56.940 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:56.941 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:56.941 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:56.950 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:35:56.972 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:56.973+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:57.073 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:57.074+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:57.102 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:57.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:57.102 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:57.102 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:57.102 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:57.111 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:35:57.174 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:57.175+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:57.235 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:57.235 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:35:57.235 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:35:57.235 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:35:57.235 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.235 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.235 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.285435133999997, 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:35:57.236 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:35:57.237 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:35:57.238 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:35:57.239 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.240 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.285347424999998, 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.241 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.242 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.244 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.245 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.285221496999995, 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:57.246 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:35:57.247 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:35:57.248 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:35:57.249 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.250 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.285159837999998, 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.251 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:35:57.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.253 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:35:57.254 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.255 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.285103074999995, 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.256 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.257 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.258 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:35:57.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:35:57.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.285039604999994, 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:35:57.261 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:57.264 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.284987655999998, 2024-04-25T12:35:57.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:35:57.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:35:57.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:35:57.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.284935705999999, 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.271 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.273 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:57.276 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.284818596999997, 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:35:57.278 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:35:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:35:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.284780150000003, 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:35:57.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.119645097000003, 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:35:57.285 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:57.290 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.119587046000007, 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:35:57.291 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.292 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:35:57.294 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.295 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.119527352999995, 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.297 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.300 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.082607271000001, 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:57.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:35:57.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:35:57.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:35:57.305 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.082518305000001, 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.306 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.310 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.082474464000001, 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.311 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:35:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:35:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:57.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.080960560000001, 2024-04-25T12:35:57.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.317 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:35:57.318 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.319 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.320 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:57.321 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.080041344000001, 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.322 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:35:57.323 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:35:57.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.326 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.079585991000002, 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:57.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:35:57.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 62.079557012999999, 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:35:57.330 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:35:57.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.334 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.334 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:35:57.334 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:35:57.334 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:57.335 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:35:57.336 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:57.336+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:57.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:35:57.436 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:57.437+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:57.487 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:57.488 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:57.488 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:57.488 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:57.497 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:35:57.538 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:57.539+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:57.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:35:57.636 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:57.637+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:57.651 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:57.652 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:57.652 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:35:57.652 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:35:57.652 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.655+0000 7f9a2ba00640 1 Processor -- start 2024-04-25T12:35:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.656+0000 7f9a2ba00640 1 -- start start 2024-04-25T12:35:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.657+0000 7f9a2ba00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a241426a0 0x7f9a24159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.657+0000 7f9a2ba00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a24074400 con 0x7f9a240c8170 2024-04-25T12:35:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.657+0000 7f9a2ba00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a241595e0 con 0x7f9a240c8fd0 2024-04-25T12:35:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a2ba00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9a24074670 con 0x7f9a241426a0 2024-04-25T12:35:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a29000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9a240c8fd0 0x7f9a24153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:49934/0 (socket says 172.21.15.123:49934) 2024-04-25T12:35:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a29000640 1 -- 172.21.15.123:0/46102057 learned_addr learned my addr 172.21.15.123:0/46102057 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:35:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a23e00640 1 --2- 172.21.15.123:0/46102057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a241426a0 0x7f9a24159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a23400640 1 -- 172.21.15.123:0/46102057 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 684509294 0 0) 0x7f9a241595e0 con 0x7f9a240c8fd0 2024-04-25T12:35:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a23400640 1 -- 172.21.15.123:0/46102057 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a0c005730 con 0x7f9a240c8fd0 2024-04-25T12:35:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a23e00640 1 -- 172.21.15.123:0/46102057 >> v1:172.21.15.153:6789/0 conn(0x7f9a240c8fd0 legacy=0x7f9a24153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a23400640 1 -- 172.21.15.123:0/46102057 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597537883 0 0) 0x7f9a24074400 con 0x7f9a240c8170 2024-04-25T12:35:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a23e00640 1 -- 172.21.15.123:0/46102057 >> v1:172.21.15.123:6789/0 conn(0x7f9a240c8170 legacy=0x7f9a240ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.658+0000 7f9a23e00640 1 -- 172.21.15.123:0/46102057 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a24074950 con 0x7f9a241426a0 2024-04-25T12:35:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.659+0000 7f9a23e00640 1 --2- 172.21.15.123:0/46102057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a241426a0 0x7f9a24159dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097d0 tx=0x7f9a1002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efdcfd496b819bec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.659+0000 7f9a23400640 1 -- 172.21.15.123:0/46102057 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10037070 con 0x7f9a241426a0 2024-04-25T12:35:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.660+0000 7f9a23400640 1 -- 172.21.15.123:0/46102057 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10032440 con 0x7f9a241426a0 2024-04-25T12:35:57.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.660+0000 7f9a23400640 1 -- 172.21.15.123:0/46102057 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100327f0 con 0x7f9a241426a0 2024-04-25T12:35:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.660+0000 7f9a2ba00640 1 -- 172.21.15.123:0/46102057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a241426a0 msgr2=0x7f9a24159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.660+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/46102057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a241426a0 0x7f9a24159dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097d0 tx=0x7f9a1002a210 comp rx=0 tx=0).stop 2024-04-25T12:35:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.660+0000 7f9a2ba00640 1 -- 172.21.15.123:0/46102057 shutdown_connections 2024-04-25T12:35:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.660+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/46102057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a241426a0 0x7f9a24159dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.660+0000 7f9a2ba00640 1 -- 172.21.15.123:0/46102057 >> 172.21.15.123:0/46102057 conn(0x7f9a2406f440 msgr2=0x7f9a2406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.661+0000 7f9a2ba00640 1 -- 172.21.15.123:0/46102057 shutdown_connections 2024-04-25T12:35:57.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.661+0000 7f9a2ba00640 1 -- 172.21.15.123:0/46102057 wait complete. 2024-04-25T12:35:57.661 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:35:57.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.663+0000 7f9a2ba00640 1 Processor -- start 2024-04-25T12:35:57.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.663+0000 7f9a2ba00640 1 -- start start 2024-04-25T12:35:57.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a2ba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a240c8170 0x7f9a240c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a2ba00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a240c8fd0 0x7f9a240cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a2ba00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a241426a0 0x7f9a24140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a2ba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a240c4880 con 0x7f9a240c8170 2024-04-25T12:35:57.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a2ba00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2415a300 con 0x7f9a241426a0 2024-04-25T12:35:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a2ba00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9a240ca910 con 0x7f9a240c8fd0 2024-04-25T12:35:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a29000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a240c8170 0x7f9a240c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a29000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a240c8170 0x7f9a240c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41508/0 (socket says 172.21.15.123:41508) 2024-04-25T12:35:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a29000640 1 -- 172.21.15.123:0/3936103223 learned_addr learned my addr 172.21.15.123:0/3936103223 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a23e00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a240c8fd0 0x7f9a240cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.665+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a241426a0 0x7f9a24140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a29000640 1 -- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a240c8fd0 msgr2=0x7f9a240cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a29000640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a240c8fd0 0x7f9a240cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a29000640 1 -- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a241426a0 msgr2=0x7f9a24140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a29000640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a241426a0 0x7f9a24140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a29000640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a240ccd60 con 0x7f9a240c8170 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a241426a0 0x7f9a24140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a29000640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a240c8170 0x7f9a240c7760 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800c440 tx=0x7f9a1800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18017070 con 0x7f9a240c8170 2024-04-25T12:35:57.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a18005ce0 con 0x7f9a240c8170 2024-04-25T12:35:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a240cd700 con 0x7f9a240c8170 2024-04-25T12:35:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a180072b0 con 0x7f9a240c8170 2024-04-25T12:35:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.666+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a240cc020 con 0x7f9a240c8170 2024-04-25T12:35:57.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.668+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f9a240c7e60 con 0x7f9a240c8170 2024-04-25T12:35:57.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.669+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9a1801a070 con 0x7f9a240c8170 2024-04-25T12:35:57.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.670+0000 7f9a20c00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9a040744b0 0x7f9a040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.670+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6619+0+0 (secure 0 0 0) 0x7f9a18094950 con 0x7f9a240c8170 2024-04-25T12:35:57.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.670+0000 7f9a23e00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9a040744b0 0x7f9a040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.670+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v45) ==== 54+0+16 (secure 0 0 0) 0x7f9a240c7e60 con 0x7f9a240c8170 2024-04-25T12:35:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.671+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f9a24158460 0x7f9a241f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.671+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a240ccf30 con 0x7f9a24158460 2024-04-25T12:35:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.671+0000 7f9a23e00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9a040744b0 0x7f9a040769c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9a10034000 tx=0x7f9a1002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.671+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f9a24158460 0x7f9a241f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.672+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f9a24158460 0x7f9a241f2140 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.672+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a1c001e50 con 0x7f9a24158460 2024-04-25T12:35:57.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.690+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9a240706c0 con 0x7f9a24158460 2024-04-25T12:35:57.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.692+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9a240706c0 con 0x7f9a24158460 2024-04-25T12:35:57.691 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:35:57.691 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:35:57.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.692+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f9a241f5710 0x7f9a241f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.692+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f9a240c4200 con 0x7f9a241f5710 2024-04-25T12:35:57.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.692+0000 7f9a29000640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f9a241f5710 0x7f9a241f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.693+0000 7f9a29000640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f9a241f5710 0x7f9a241f5b20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.694+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a240c4200 con 0x7f9a241f5710 2024-04-25T12:35:57.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.710+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9a240cdbe0 con 0x7f9a241f5710 2024-04-25T12:35:57.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.712+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9a240cdbe0 con 0x7f9a241f5710 2024-04-25T12:35:57.711 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:35:57.712 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:35:57.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.713+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f9a24062fe0 0x7f9a240633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.713+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f9a241f6570 con 0x7f9a24062fe0 2024-04-25T12:35:57.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.713+0000 7f9a23e00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f9a24062fe0 0x7f9a240633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.714+0000 7f9a23e00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f9a24062fe0 0x7f9a240633f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.715+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a241f6570 con 0x7f9a24062fe0 2024-04-25T12:35:57.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.730+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9a24063930 con 0x7f9a24062fe0 2024-04-25T12:35:57.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.732+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9a24063930 con 0x7f9a24062fe0 2024-04-25T12:35:57.731 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:35:57.732 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:35:57.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.732+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f9a24070ee0 0x7f9a240712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.732+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9a24063f00 con 0x7f9a24070ee0 2024-04-25T12:35:57.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.733+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f9a24070ee0 0x7f9a240712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.733+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f9a24070ee0 0x7f9a240712f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.735+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a24063f00 con 0x7f9a24070ee0 2024-04-25T12:35:57.745 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:57.746+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:57.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.756+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9a240cdd90 con 0x7f9a24070ee0 2024-04-25T12:35:57.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.759+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9a240cdd90 con 0x7f9a24070ee0 2024-04-25T12:35:57.759 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:35:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:35:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.760+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f9a2402cee0 0x7f9a2402d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.760+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f9a2402d830 con 0x7f9a2402cee0 2024-04-25T12:35:57.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.760+0000 7f9a29000640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f9a2402cee0 0x7f9a2402d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.760+0000 7f9a29000640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f9a2402cee0 0x7f9a2402d2f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.761+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a2402d830 con 0x7f9a2402cee0 2024-04-25T12:35:57.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.776+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9a2402d830 con 0x7f9a2402cee0 2024-04-25T12:35:57.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.778+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9a2402d830 con 0x7f9a2402cee0 2024-04-25T12:35:57.777 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:35:57.778 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:35:57.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.779+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=46}) -- 0x7f9a2402d830 con 0x7f9a240c8170 2024-04-25T12:35:57.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.779+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a2402dc00 con 0x7f9a240c8170 2024-04-25T12:35:57.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.779+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9a18090060 con 0x7f9a240c8170 2024-04-25T12:35:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-04-25T12:35:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.779+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f9a24207170 0x7f9a24209590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.779+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f9a2420c360 con 0x7f9a24207170 2024-04-25T12:35:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.780+0000 7f9a23e00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f9a24207170 0x7f9a24209590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.780+0000 7f9a23e00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f9a24207170 0x7f9a24209590 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.781+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a2420c360 con 0x7f9a24207170 2024-04-25T12:35:57.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.796+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9a24209ad0 con 0x7f9a24207170 2024-04-25T12:35:57.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.798+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9a24209ad0 con 0x7f9a24207170 2024-04-25T12:35:57.797 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:35:57.798 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:35:57.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.798+0000 7f9a2ba00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f9a2420a150 0x7f9a2420c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.798+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f9a2420f570 con 0x7f9a2420a150 2024-04-25T12:35:57.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.799+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f9a2420a150 0x7f9a2420c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.799+0000 7f9a29a00640 1 --2- 172.21.15.123:0/3936103223 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f9a2420a150 0x7f9a2420c570 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.800+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a2420f570 con 0x7f9a2420a150 2024-04-25T12:35:57.814 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:57.815 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:57.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.816+0000 7f9a2ba00640 1 -- 172.21.15.123:0/3936103223 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9a2402de30 con 0x7f9a2420a150 2024-04-25T12:35:57.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.818+0000 7f9a20c00640 1 -- 172.21.15.123:0/3936103223 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9a2402de30 con 0x7f9a2420a150 2024-04-25T12:35:57.817 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:35:57.817 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:35:57.824 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:35:57.833 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T12:35:57.833 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-25T12:35:57.865 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:57.866+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:57.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.875+0000 7f682b600640 1 Processor -- start 2024-04-25T12:35:57.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.876+0000 7f682b600640 1 -- start start 2024-04-25T12:35:57.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f682b600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c9b80 0x7f68240742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f682b600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6824159f30 0x7f68240c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f682b600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6824156e60 con 0x7f68240c8250 2024-04-25T12:35:57.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f682b600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f68240c8ba0 con 0x7f6824159f30 2024-04-25T12:35:57.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f682b600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f682415ca70 con 0x7f68240c9b80 2024-04-25T12:35:57.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f6829600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6824159f30 0x7f68240c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f6828c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c9b80 0x7f68240742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f6829600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6824159f30 0x7f68240c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:60558/0 (socket says 172.21.15.123:60558) 2024-04-25T12:35:57.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f6829600640 1 -- 172.21.15.123:0/3679660445 learned_addr learned my addr 172.21.15.123:0/3679660445 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:57.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.878+0000 7f6829600640 1 -- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c9b80 msgr2=0x7f68240742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:35:57.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.879+0000 7f6829600640 1 --2- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c9b80 0x7f68240742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.879+0000 7f6829600640 1 -- 172.21.15.123:0/3679660445 >> v1:172.21.15.123:6789/0 conn(0x7f68240c8250 legacy=0x7f6824159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.879+0000 7f6829600640 1 -- 172.21.15.123:0/3679660445 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68240c7130 con 0x7f6824159f30 2024-04-25T12:35:57.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.879+0000 7f6828c00640 1 --2- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c9b80 0x7f68240742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:57.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.880+0000 7f6829600640 1 --2- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6824159f30 0x7f68240c8660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f681c00bb10 tx=0x7f681c02c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1318b93f2c185ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.880+0000 7f6823400640 1 -- 172.21.15.123:0/3679660445 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c036020 con 0x7f6824159f30 2024-04-25T12:35:57.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.880+0000 7f6823400640 1 -- 172.21.15.123:0/3679660445 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681c032820 con 0x7f6824159f30 2024-04-25T12:35:57.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.880+0000 7f6823400640 1 -- 172.21.15.123:0/3679660445 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c032bf0 con 0x7f6824159f30 2024-04-25T12:35:57.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 -- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6824159f30 msgr2=0x7f68240c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 --2- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6824159f30 0x7f68240c8660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f681c00bb10 tx=0x7f681c02c920 comp rx=0 tx=0).stop 2024-04-25T12:35:57.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 -- 172.21.15.123:0/3679660445 shutdown_connections 2024-04-25T12:35:57.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 --2- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6824159f30 0x7f68240c8660 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 --2- 172.21.15.123:0/3679660445 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c9b80 0x7f68240742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 -- 172.21.15.123:0/3679660445 >> 172.21.15.123:0/3679660445 conn(0x7f682406c750 msgr2=0x7f682406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:57.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 -- 172.21.15.123:0/3679660445 shutdown_connections 2024-04-25T12:35:57.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.881+0000 7f682b600640 1 -- 172.21.15.123:0/3679660445 wait complete. 2024-04-25T12:35:57.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.884+0000 7f682b600640 1 Processor -- start 2024-04-25T12:35:57.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.884+0000 7f682b600640 1 -- start start 2024-04-25T12:35:57.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.886+0000 7f682b600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c8250 0x7f68241e2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.886+0000 7f682b600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68240c9b80 0x7f682414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.886+0000 7f6828c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c8250 0x7f68241e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.886+0000 7f6823e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68240c9b80 0x7f682414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.886+0000 7f6828c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c8250 0x7f68241e2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:40522/0 (socket says 172.21.15.123:40522) 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6823e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68240c9b80 0x7f682414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:60560/0 (socket says 172.21.15.123:60560) 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f682b600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6824159f30 0x7f68240cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f682415d310 con 0x7f6824159f30 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6828c00640 1 -- 172.21.15.123:0/3867372804 learned_addr learned my addr 172.21.15.123:0/3867372804 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f682415d030 con 0x7f68240c9b80 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f682415d5f0 con 0x7f68240c8250 2024-04-25T12:35:57.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6829600640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6824159f30 0x7f68240cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6823e00640 1 -- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c8250 msgr2=0x7f68241e2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6823e00640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c8250 0x7f68241e2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6823e00640 1 -- 172.21.15.123:0/3867372804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6824159f30 msgr2=0x7f68240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6823e00640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6824159f30 0x7f68240cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6823e00640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68241e2e90 con 0x7f68240c9b80 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.887+0000 7f6828c00640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c8250 0x7f68241e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.888+0000 7f6823e00640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68240c9b80 0x7f682414e050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68240ce8a0 tx=0x7f681000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.888+0000 7f6820c00640 1 -- 172.21.15.123:0/3867372804 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100052d0 con 0x7f68240c9b80 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.888+0000 7f6820c00640 1 -- 172.21.15.123:0/3867372804 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6810005490 con 0x7f68240c9b80 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.888+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68241f6a00 con 0x7f68240c9b80 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.888+0000 7f6820c00640 1 -- 172.21.15.123:0/3867372804 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810007260 con 0x7f68240c9b80 2024-04-25T12:35:57.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.888+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68241f6fc0 con 0x7f68240c9b80 2024-04-25T12:35:57.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.889+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68240c6d70 con 0x7f68240c9b80 2024-04-25T12:35:57.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.890+0000 7f6820c00640 1 -- 172.21.15.123:0/3867372804 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6810007400 con 0x7f68240c9b80 2024-04-25T12:35:57.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.891+0000 7f6820c00640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68040744b0 0x7f68040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:35:57.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.891+0000 7f6820c00640 1 -- 172.21.15.123:0/3867372804 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6619+0+0 (secure 0 0 0) 0x7f6810094860 con 0x7f68240c9b80 2024-04-25T12:35:57.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.892+0000 7f6828c00640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68040744b0 0x7f68040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:35:57.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.892+0000 7f6828c00640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68040744b0 0x7f68040769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6814003c60 tx=0x7f681400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:35:57.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:57.896+0000 7f6820c00640 1 -- 172.21.15.123:0/3867372804 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68240c6d70 con 0x7f68240c9b80 2024-04-25T12:35:57.939 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:35:57.940+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:35:57.945 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:57.946+0000 7f2d71415640 -1 osd.5 39 log_to_monitors true 2024-04-25T12:35:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:57.992 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:35:58.041 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:58.042+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:58.103 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:58.104 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:58.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:35:58.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.114+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f68240ce380 con 0x7f68240c9b80 2024-04-25T12:35:58.156 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:58.157+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:58.224 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:58.224 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:58.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:35:58.242 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:58.243+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:58.343 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:58.344+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:58.444 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:58.445+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:58.494 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:58.494 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:58.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:35:58.544 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:58.544+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:58.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.567+0000 7f6820c00640 1 -- 172.21.15.123:0/3867372804 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v46) ==== 101+0+0 (secure 0 0 0) 0x7f681005e600 con 0x7f68240c9b80 2024-04-25T12:35:58.566 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:35:58.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.571+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68040744b0 msgr2=0x7f68040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:58.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.571+0000 7f682b600640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68040744b0 0x7f68040769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6814003c60 tx=0x7f681400b040 comp rx=0 tx=0).stop 2024-04-25T12:35:58.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.571+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68240c9b80 msgr2=0x7f682414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:35:58.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.571+0000 7f682b600640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68240c9b80 0x7f682414e050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68240ce8a0 tx=0x7f681000c820 comp rx=0 tx=0).stop 2024-04-25T12:35:58.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 shutdown_connections 2024-04-25T12:35:58.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f68040744b0 0x7f68040769c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:58.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6824159f30 0x7f68240cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:58.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68240c9b80 0x7f682414e050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:58.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 --2- 172.21.15.123:0/3867372804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f68240c8250 0x7f68241e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:35:58.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 >> 172.21.15.123:0/3867372804 conn(0x7f682406c750 msgr2=0x7f68240742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:35:58.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 shutdown_connections 2024-04-25T12:35:58.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:35:58.572+0000 7f682b600640 1 -- 172.21.15.123:0/3867372804 wait complete. 2024-04-25T12:35:58.600 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:35:58.600+0000 7f2d64af6640 -1 osd.5 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:35:58.645 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:58.646+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:58.746 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:58.747+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:58.814 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:58.814 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:58.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:35:58.846 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:58.847+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:58.940 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:35:58.984 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:58.985+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:59.048 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:59.049+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:59.054 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:35:59.055 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:35:59.055 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:35:59.055 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:35:59.149 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:59.150+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:59.163 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:59.163 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:59.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:35:59.249 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:59.250+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:59.350 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:35:59.351+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:35:59.451 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:35:59.451+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:35:59.485 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:59.485 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:59.507 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:35:59.552 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:59.553+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:59.652 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:35:59.653+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:35:59.753 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:35:59.754+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:35:59.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:35:59.841 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:35:59.841 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:35:59.854 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:35:59.855+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:35:59.863 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:35:59.955 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:35:59.956+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:00.056 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:00.057+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:00.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.113+0000 7f4a9b000640 1 Processor -- start 2024-04-25T12:36:00.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.113+0000 7f4a9b000640 1 -- start start 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.115+0000 7f4a9b000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a94153640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.115+0000 7f4a9b000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4a940c8170 0x7f4a940ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.115+0000 7f4a9b000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a94074400 con 0x7f4a941426a0 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.115+0000 7f4a9b000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4a94153b80 con 0x7f4a940c8170 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.115+0000 7f4a9b000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a941595e0 con 0x7f4a940c8fd0 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.115+0000 7f4a93400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4a941426a0 0x7f4a94159dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42088/0 (socket says 172.21.15.123:42088) 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.115+0000 7f4a93400640 1 -- 172.21.15.123:0/833476121 learned_addr learned my addr 172.21.15.123:0/833476121 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a99000640 1 --2- 172.21.15.123:0/833476121 >> v2:172.21.15.153:3300/0 conn(0x7f4a940c8170 0x7f4a940ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a93e00640 1 --2- 172.21.15.123:0/833476121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a94153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a92a00640 1 -- 172.21.15.123:0/833476121 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704759050 0 0) 0x7f4a94074400 con 0x7f4a941426a0 2024-04-25T12:36:00.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a92a00640 1 -- 172.21.15.123:0/833476121 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a78003520 con 0x7f4a941426a0 2024-04-25T12:36:00.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a93e00640 1 -- 172.21.15.123:0/833476121 >> v2:172.21.15.153:3300/0 conn(0x7f4a940c8170 msgr2=0x7f4a940ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:00.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a93e00640 1 --2- 172.21.15.123:0/833476121 >> v2:172.21.15.153:3300/0 conn(0x7f4a940c8170 0x7f4a940ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a93e00640 1 -- 172.21.15.123:0/833476121 >> v1:172.21.15.123:6789/0 conn(0x7f4a941426a0 legacy=0x7f4a94159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.116+0000 7f4a93e00640 1 -- 172.21.15.123:0/833476121 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a94074670 con 0x7f4a940c8fd0 2024-04-25T12:36:00.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.118+0000 7f4a93e00640 1 --2- 172.21.15.123:0/833476121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a94153640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097a0 tx=0x7f4a8402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509eb43304178909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.118+0000 7f4a92a00640 1 -- 172.21.15.123:0/833476121 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84037070 con 0x7f4a940c8fd0 2024-04-25T12:36:00.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.118+0000 7f4a92a00640 1 -- 172.21.15.123:0/833476121 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a84032440 con 0x7f4a940c8fd0 2024-04-25T12:36:00.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.118+0000 7f4a92a00640 1 -- 172.21.15.123:0/833476121 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a840327f0 con 0x7f4a940c8fd0 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 -- 172.21.15.123:0/833476121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 msgr2=0x7f4a94153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 --2- 172.21.15.123:0/833476121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a94153640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097a0 tx=0x7f4a8402a180 comp rx=0 tx=0).stop 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 -- 172.21.15.123:0/833476121 shutdown_connections 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 --2- 172.21.15.123:0/833476121 >> v2:172.21.15.153:3300/0 conn(0x7f4a940c8170 0x7f4a940ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 --2- 172.21.15.123:0/833476121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a94153640 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 -- 172.21.15.123:0/833476121 >> 172.21.15.123:0/833476121 conn(0x7f4a9406f440 msgr2=0x7f4a9406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 -- 172.21.15.123:0/833476121 shutdown_connections 2024-04-25T12:36:00.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.119+0000 7f4a9b000640 1 -- 172.21.15.123:0/833476121 wait complete. 2024-04-25T12:36:00.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.122+0000 7f4a9b000640 1 Processor -- start 2024-04-25T12:36:00.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.122+0000 7f4a9b000640 1 -- start start 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.123+0000 7f4a9b000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940c8170 0x7f4a9413f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.123+0000 7f4a9b000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a940cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.123+0000 7f4a9b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a941426a0 0x7f4a9413e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.123+0000 7f4a9b000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a940c4880 con 0x7f4a940c8170 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.123+0000 7f4a9b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941598a0 con 0x7f4a941426a0 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.123+0000 7f4a9b000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a940c8580 con 0x7f4a940c8fd0 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a93e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940c8170 0x7f4a9413f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a93e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940c8170 0x7f4a9413f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41524/0 (socket says 172.21.15.123:41524) 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a93e00640 1 -- 172.21.15.123:0/3566614384 learned_addr learned my addr 172.21.15.123:0/3566614384 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a941426a0 0x7f4a9413e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a93400640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a940cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a99000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 msgr2=0x7f4a940cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a940c8fd0 0x7f4a940cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a99000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940c8170 msgr2=0x7f4a9413f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940c8170 0x7f4a9413f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a99000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a940cdf80 con 0x7f4a941426a0 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.124+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940c8170 0x7f4a9413f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.125+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a941426a0 0x7f4a9413e030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00eb70 tx=0x7f4a8c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.125+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c018020 con 0x7f4a941426a0 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.125+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c00cd20 con 0x7f4a941426a0 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.125+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0146c0 con 0x7f4a941426a0 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.125+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a941f1270 con 0x7f4a941426a0 2024-04-25T12:36:00.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.125+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a940cc080 con 0x7f4a941426a0 2024-04-25T12:36:00.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.126+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4a94076700 con 0x7f4a941426a0 2024-04-25T12:36:00.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.128+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a8c005d10 con 0x7f4a941426a0 2024-04-25T12:36:00.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.129+0000 7f4a8be00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a740744e0 0x7f4a740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.129+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6619+0+0 (secure 0 0 0) 0x7f4a8c096040 con 0x7f4a941426a0 2024-04-25T12:36:00.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.130+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a740744e0 0x7f4a740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.130+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v47) ==== 54+0+16 (secure 0 0 0) 0x7f4a8c022030 con 0x7f4a941426a0 2024-04-25T12:36:00.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.130+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4a940ce150 0x7f4a9413cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.130+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a94165d00 con 0x7f4a940ce150 2024-04-25T12:36:00.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.130+0000 7f4a93400640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4a940ce150 0x7f4a9413cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.130+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a740744e0 0x7f4a740769f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097a0 tx=0x7f4a8402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.131+0000 7f4a93400640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4a940ce150 0x7f4a9413cc50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.132+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a94165d00 con 0x7f4a940ce150 2024-04-25T12:36:00.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.149+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a94159f60 con 0x7f4a940ce150 2024-04-25T12:36:00.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.152+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4a94159f60 con 0x7f4a940ce150 2024-04-25T12:36:00.151 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:00.151 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:00.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.152+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f4a9413a380 0x7f4a941f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.152+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4a9413a790 con 0x7f4a9413a380 2024-04-25T12:36:00.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.152+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f4a9413a380 0x7f4a941f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.153+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f4a9413a380 0x7f4a941f39d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.154+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a9413a790 con 0x7f4a9413a380 2024-04-25T12:36:00.157 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:00.158+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:00.161 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:00.161 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:00.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.176+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a940c4200 con 0x7f4a9413a380 2024-04-25T12:36:00.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.179+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4a940c4200 con 0x7f4a9413a380 2024-04-25T12:36:00.442 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:00.442 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:00.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.180+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f4a941f4420 0x7f4a94062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.180+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4a940cc3c0 con 0x7f4a941f4420 2024-04-25T12:36:00.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.180+0000 7f4a93400640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f4a941f4420 0x7f4a94062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.181+0000 7f4a93400640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f4a941f4420 0x7f4a94062fe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.181+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a940cc3c0 con 0x7f4a941f4420 2024-04-25T12:36:00.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.204+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a941f4830 con 0x7f4a941f4420 2024-04-25T12:36:00.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.207+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4a941f4830 con 0x7f4a941f4420 2024-04-25T12:36:00.445 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.207+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f4a940706c0 0x7f4a94070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.207+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4a94063df0 con 0x7f4a940706c0 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.207+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f4a940706c0 0x7f4a94070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.208+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f4a940706c0 0x7f4a94070ad0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.209+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a94063df0 con 0x7f4a940706c0 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.229+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a940712e0 con 0x7f4a940706c0 2024-04-25T12:36:00.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.232+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4a940712e0 con 0x7f4a940706c0 2024-04-25T12:36:00.447 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.232+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f4a9402c9d0 0x7f4a9402cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.232+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4a9402d320 con 0x7f4a9402c9d0 2024-04-25T12:36:00.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.232+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f4a9402c9d0 0x7f4a9402cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.233+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f4a9402c9d0 0x7f4a9402cde0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.234+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a9402d320 con 0x7f4a9402c9d0 2024-04-25T12:36:00.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.250+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a9402d320 con 0x7f4a9402c9d0 2024-04-25T12:36:00.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.253+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4a9402d320 con 0x7f4a9402c9d0 2024-04-25T12:36:00.449 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.253+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f4a9402da40 0x7f4a942091b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.253+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f4a9402de50 con 0x7f4a9402da40 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.253+0000 7f4a93400640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f4a9402da40 0x7f4a942091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.254+0000 7f4a93400640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f4a9402da40 0x7f4a942091b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.255+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a9402de50 con 0x7f4a9402da40 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.270+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a9402de50 con 0x7f4a9402da40 2024-04-25T12:36:00.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.272+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' ) ==== 503+0+3 (crc 0 0 0) 0x7f4a9402de50 con 0x7f4a9402da40 2024-04-25T12:36:00.452 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:00.452 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:00.442+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-04-25T12:36:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.272+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f4a94209c00 0x7f4a9420c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.273+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4a9420f1a0 con 0x7f4a94209c00 2024-04-25T12:36:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.273+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f4a94209c00 0x7f4a9420c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.274+0000 7f4a99000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f4a94209c00 0x7f4a9420c3d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.275+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a9420f1a0 con 0x7f4a94209c00 2024-04-25T12:36:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.291+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a9420c910 con 0x7f4a94209c00 2024-04-25T12:36:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.293+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4a9420c910 con 0x7f4a94209c00 2024-04-25T12:36:00.454 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:00.458 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.293+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f4a9420cf90 0x7f4a9420f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.293+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4a942124b0 con 0x7f4a9420cf90 2024-04-25T12:36:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.294+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f4a9420cf90 0x7f4a9420f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.294+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f4a9420cf90 0x7f4a9420f3d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.295+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a942124b0 con 0x7f4a9420cf90 2024-04-25T12:36:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.310+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4a94212440 con 0x7f4a9420cf90 2024-04-25T12:36:00.459 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:00.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.312+0000 7f4a8be00640 1 -- 172.21.15.123:0/3566614384 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4a94212440 con 0x7f4a9420cf90 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.312+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4a940ce150 msgr2=0x7f4a9413cc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.312+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4a940ce150 0x7f4a9413cc50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f4a9413a380 msgr2=0x7f4a941f39d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f4a9413a380 0x7f4a941f39d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f4a941f4420 msgr2=0x7f4a94062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f4a941f4420 0x7f4a94062fe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f4a940706c0 msgr2=0x7f4a94070ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a99000640 1 -- 172.21.15.123:0/3566614384 reap_dead start 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f4a940706c0 0x7f4a94070ad0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f4a9402c9d0 msgr2=0x7f4a9402cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f4a9402c9d0 0x7f4a9402cde0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f4a9402da40 msgr2=0x7f4a942091b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f4a9402da40 0x7f4a942091b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f4a94209c00 msgr2=0x7f4a9420c3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f4a94209c00 0x7f4a9420c3d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f4a9420cf90 msgr2=0x7f4a9420f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f4a9420cf90 0x7f4a9420f3d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a99000640 1 -- 172.21.15.123:0/3566614384 reap_dead start 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a740744e0 msgr2=0x7f4a740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a740744e0 0x7f4a740769f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097a0 tx=0x7f4a8402e000 comp rx=0 tx=0).stop 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a941426a0 msgr2=0x7f4a9413e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a941426a0 0x7f4a9413e030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00eb70 tx=0x7f4a8c00c490 comp rx=0 tx=0).stop 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 shutdown_connections 2024-04-25T12:36:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4a740744e0 0x7f4a740769f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 --2- 172.21.15.123:0/3566614384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a941426a0 0x7f4a9413e030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 >> 172.21.15.123:0/3566614384 conn(0x7f4a9406f440 msgr2=0x7f4a94158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:00.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.313+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 shutdown_connections 2024-04-25T12:36:00.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.314+0000 7f4a9b000640 1 -- 172.21.15.123:0/3566614384 wait complete. 2024-04-25T12:36:00.543 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:00.543+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:00.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:36:00.652 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:00.652+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:00.746 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:00.747+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:00.757 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:00.757 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:00.757 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:00.757 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:00.777 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:00.845 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:00.846+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:00.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.880+0000 7f2a8f800640 1 Processor -- start 2024-04-25T12:36:00.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.880+0000 7f2a8f800640 1 -- start start 2024-04-25T12:36:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.882+0000 7f2a8f800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2a88074870 0x7f2a88154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a8f800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8815b3b0 con 0x7f2a880c4d50 2024-04-25T12:36:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a8f800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a88073ce0 con 0x7f2a8815afa0 2024-04-25T12:36:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a8f800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f2a880c61f0 con 0x7f2a88074870 2024-04-25T12:36:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a8ce00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2a880c4d50 0x7f2a880c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42090/0 (socket says 172.21.15.123:42090) 2024-04-25T12:36:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a8ce00640 1 -- 172.21.15.123:0/992935043 learned_addr learned my addr 172.21.15.123:0/992935043 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a87e00640 1 --2- 172.21.15.123:0/992935043 >> v2:172.21.15.153:3300/0 conn(0x7f2a88074870 0x7f2a88154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3956081152 0 0) 0x7f2a8815b3b0 con 0x7f2a880c4d50 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.883+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a70003520 con 0x7f2a880c4d50 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089175573 0 0) 0x7f2a88073ce0 con 0x7f2a8815afa0 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a8815b3b0 con 0x7f2a8815afa0 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 156760977 0 0) 0x7f2a70003520 con 0x7f2a880c4d50 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 >> v2:172.21.15.153:3300/0 conn(0x7f2a88074870 msgr2=0x7f2a88154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 --2- 172.21.15.123:0/992935043 >> v2:172.21.15.153:3300/0 conn(0x7f2a88074870 0x7f2a88154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 >> v1:172.21.15.153:6789/0 conn(0x7f2a8815afa0 legacy=0x7f2a880cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a88073fc0 con 0x7f2a880c4d50 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f2a7c001670 con 0x7f2a880c4d50 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87e00640 1 --2- 172.21.15.123:0/992935043 >> v2:172.21.15.153:3300/0 conn(0x7f2a88074870 0x7f2a88154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 >> v1:172.21.15.123:6789/0 conn(0x7f2a880c4d50 legacy=0x7f2a880c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70003b40 0x7f2a70003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a70004800 0x7f2a70004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a700051a0 0x7f2a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7000c090 con 0x7f2a700051a0 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7000c210 con 0x7f2a70004800 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7000c410 con 0x7f2a70003b40 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a8ce00640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70003b40 0x7f2a70003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a87e00640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a70004800 0x7f2a70004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.884+0000 7f2a8d800640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a700051a0 0x7f2a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a87e00640 1 -- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70003b40 msgr2=0x7f2a70003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a87e00640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70003b40 0x7f2a70003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a87e00640 1 -- 172.21.15.123:0/992935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a700051a0 msgr2=0x7f2a7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a87e00640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a700051a0 0x7f2a7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a87e00640 1 -- 172.21.15.123:0/992935043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a70003520 con 0x7f2a70004800 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a8ce00640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70003b40 0x7f2a70003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.885+0000 7f2a8d800640 1 -- 172.21.15.123:0/992935043 reap_dead start 2024-04-25T12:36:00.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.886+0000 7f2a87e00640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a70004800 0x7f2a70004c10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a880b3270 tx=0x7f2a70005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4845bd5c941062f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.886+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74009860 con 0x7f2a70004800 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.886+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a74004020 con 0x7f2a70004800 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.886+0000 7f2a87400640 1 -- 172.21.15.123:0/992935043 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a740043f0 con 0x7f2a70004800 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.887+0000 7f2a8f800640 1 -- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a70004800 msgr2=0x7f2a70004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.887+0000 7f2a8f800640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a70004800 0x7f2a70004c10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a880b3270 tx=0x7f2a70005830 comp rx=0 tx=0).stop 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.888+0000 7f2a8f800640 1 -- 172.21.15.123:0/992935043 shutdown_connections 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.888+0000 7f2a8f800640 1 --2- 172.21.15.123:0/992935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a70004800 0x7f2a70004c10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.888+0000 7f2a8f800640 1 -- 172.21.15.123:0/992935043 >> 172.21.15.123:0/992935043 conn(0x7f2a8806c750 msgr2=0x7f2a8806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.888+0000 7f2a8f800640 1 -- 172.21.15.123:0/992935043 shutdown_connections 2024-04-25T12:36:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.888+0000 7f2a8f800640 1 -- 172.21.15.123:0/992935043 wait complete. 2024-04-25T12:36:00.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.890+0000 7f2a8f800640 1 Processor -- start 2024-04-25T12:36:00.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.890+0000 7f2a8f800640 1 -- start start 2024-04-25T12:36:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8f800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70004800 0x7f2a8814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8f800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a880c7460 0x7f2a8814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8f800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a88154170 0x7f2a8814a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8f800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a880c8460 con 0x7f2a88154170 2024-04-25T12:36:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8f800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a880c87d0 con 0x7f2a880c7460 2024-04-25T12:36:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8f800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2a88154890 con 0x7f2a70004800 2024-04-25T12:36:00.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a88154170 0x7f2a8814a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.892+0000 7f2a8ce00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70004800 0x7f2a8814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a88154170 0x7f2a8814a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41542/0 (socket says 172.21.15.123:41542) 2024-04-25T12:36:00.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8ce00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70004800 0x7f2a8814d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:40554/0 (socket says 172.21.15.123:40554) 2024-04-25T12:36:00.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8d800640 1 -- 172.21.15.123:0/1135376658 learned_addr learned my addr 172.21.15.123:0/1135376658 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:00.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a87e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a880c7460 0x7f2a8814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8ce00640 1 -- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a880c7460 msgr2=0x7f2a8814a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8ce00640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a880c7460 0x7f2a8814a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8ce00640 1 -- 172.21.15.123:0/1135376658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a88154170 msgr2=0x7f2a8814a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8ce00640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a88154170 0x7f2a8814a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8ce00640 1 -- 172.21.15.123:0/1135376658 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a88153e10 con 0x7f2a70004800 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8d800640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a88154170 0x7f2a8814a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.893+0000 7f2a8ce00640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70004800 0x7f2a8814d2b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a8806b760 tx=0x7f2a7c007660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.894+0000 7f2a84c00640 1 -- 172.21.15.123:0/1135376658 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c007ca0 con 0x7f2a70004800 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.894+0000 7f2a84c00640 1 -- 172.21.15.123:0/1135376658 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7c018070 con 0x7f2a70004800 2024-04-25T12:36:00.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.894+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a880ccc00 con 0x7f2a70004800 2024-04-25T12:36:00.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.894+0000 7f2a84c00640 1 -- 172.21.15.123:0/1135376658 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00fab0 con 0x7f2a70004800 2024-04-25T12:36:00.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.894+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881f1210 con 0x7f2a70004800 2024-04-25T12:36:00.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.896+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a88076700 con 0x7f2a70004800 2024-04-25T12:36:00.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.897+0000 7f2a84c00640 1 -- 172.21.15.123:0/1135376658 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a7c00fc50 con 0x7f2a70004800 2024-04-25T12:36:00.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.897+0000 7f2a84c00640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2a68074500 0x7f2a68076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:00.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.898+0000 7f2a84c00640 1 -- 172.21.15.123:0/1135376658 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 6619+0+0 (secure 0 0 0) 0x7f2a7c095310 con 0x7f2a70004800 2024-04-25T12:36:00.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.898+0000 7f2a87e00640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2a68074500 0x7f2a68076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:00.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.899+0000 7f2a87e00640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2a68074500 0x7f2a68076a10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a74032000 tx=0x7f2a74008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:00.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:00.903+0000 7f2a84c00640 1 -- 172.21.15.123:0/1135376658 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a7c05f7f0 con 0x7f2a70004800 2024-04-25T12:36:00.947 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:00.948+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:01.047 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:01.048+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:01.074 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:01.074 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:01.075 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:01.075 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:01.075 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:01.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:01.148 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:01.149+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:01.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.154+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f2a880b3610 con 0x7f2a70004800 2024-04-25T12:36:01.248 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:01.250+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:01.348 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:01.350+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:01.352 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.398175026, 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:36:01.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:36:01.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:36:01.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.398063083, 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:36:01.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:36:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.39801428299999, 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:36:01.367 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.397943123, 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:36:01.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:36:01.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:01.373 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.39789471100001, 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:01.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:36:01.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.389698783, 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:36:01.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 85.283454785000004, 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:36:01.381 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:36:01.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:36:01.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 85.283272784000005, 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.231238797000003, 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:01.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:36:01.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:36:01.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.231162841, 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:36:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:36:01.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:36:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.226734101000005, 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.226552037999994, 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:36:01.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:36:01.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:36:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.226447673999999, 2024-04-25T12:36:01.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.225172117, 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.421 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 71.224117892999999, 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:36:01.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:36:01.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.195252253000007, 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:36:01.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.473740757999998, 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.430 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:36:01.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 48.431180206000001, 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.435 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:36:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:36:01.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:36:01.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:48 2.14 2:2e2fa760:::200.00000001:head [write 5505~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.558788275, 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010227916, 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790256+0000", 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7570000000000002e-06 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790280+0000", 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4456000000000001e-05 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790319+0000", 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9103000000000002e-05 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790344+0000", 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4998999999999999e-05 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790557+0000", 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021251199999999999 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790572+0000", 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5003e-05 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.796879+0000", 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063072359999999999 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797129+0000", 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025002800000000001 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797609+0000", 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047965200000000001 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800188+0000", 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002578922 2024-04-25T12:36:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800281+0000", 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3561999999999996e-05 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800387+0000", 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010545800000000001 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800480+0000", 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3227999999999996e-05 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.445 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:49 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.558518694, 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011186097000000001, 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:01.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790524+0000", 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999998e-06 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790541+0000", 2024-04-25T12:36:01.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7567e-05 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790566+0000", 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4338999999999999e-05 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790581+0000", 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5123000000000001e-05 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790708+0000", 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012676699999999999 2024-04-25T12:36:01.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790718+0000", 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.798785+0000", 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080674500000000003 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799024+0000", 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000239044 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799376+0000", 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035158299999999998 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800980+0000", 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016040939999999999 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801537+0000", 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055677400000000001 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801639+0000", 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102388 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801708+0000", 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8504999999999995e-05 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:01.451 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:01.453 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:01.453 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:01.453+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:01.554 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:01.555+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:01.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.613+0000 7f2a84c00640 1 -- 172.21.15.123:0/1135376658 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v49) ==== 87+0+0 (secure 0 0 0) 0x7f2a7c05f0f0 con 0x7f2a70004800 2024-04-25T12:36:01.613 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:36:01.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.617+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2a68074500 msgr2=0x7f2a68076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:01.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.617+0000 7f2a8f800640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2a68074500 0x7f2a68076a10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a74032000 tx=0x7f2a74008030 comp rx=0 tx=0).stop 2024-04-25T12:36:01.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.617+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70004800 msgr2=0x7f2a8814d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:01.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.617+0000 7f2a8f800640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70004800 0x7f2a8814d2b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a8806b760 tx=0x7f2a7c007660 comp rx=0 tx=0).stop 2024-04-25T12:36:01.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 shutdown_connections 2024-04-25T12:36:01.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2a68074500 0x7f2a68076a10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:01.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a88154170 0x7f2a8814a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:01.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a880c7460 0x7f2a8814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:01.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 --2- 172.21.15.123:0/1135376658 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a70004800 0x7f2a8814d2b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:01.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 >> 172.21.15.123:0/1135376658 conn(0x7f2a8806c750 msgr2=0x7f2a8814f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:01.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 shutdown_connections 2024-04-25T12:36:01.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:01.618+0000 7f2a8f800640 1 -- 172.21.15.123:0/1135376658 wait complete. 2024-04-25T12:36:01.655 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:01.656+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:01.756 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:01.757+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:01.765 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:01.765 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:01.766 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:01.766 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:01.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:01.874 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:01.875+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:01.958 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:01.960+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:02.059 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:02.060+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:02.103 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:02.103 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:02.103 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:02.103 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:02.103 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:02.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:02.166 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:02.167+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:02.261 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:02.262+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:02.362 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:02.363+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.48234372100001, 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:36:02.435 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:36:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:36:02.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.48165953900001, 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:02.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:36:02.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.48020899799999, 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:36:02.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.448 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.479948183, 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:36:02.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:36:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.476097796, 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:36:02.452 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:36:02.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:36:02.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:36:02.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.457 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.47587542799999, 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:36:02.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:36:02.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.47552163899999, 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:36:02.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:36:02.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:36:02.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.314902691, 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:36:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:36:02.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:02.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.472 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.314695198999999, 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:36:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:36:02.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:36:02.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.314550709000002, 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:36:02.478 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:36:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:36:02.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.483 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.314418915000005, 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.31434462, 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.489 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:36:02.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:36:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:36:02.498 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.314244879, 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:36:02.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:36:02.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.314175417000001, 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:36:02.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:36:02.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:02.510 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.314075302999996, 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:36:02.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:02.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.313978211999995, 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:36:02.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:36:02.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.313887807, 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:36:02.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.313739834000003, 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.529 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.313632428999995, 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.533 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 67.279381130999994, 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:36:02.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:02.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:02.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:02.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:02.539 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:02.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:02.541 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:02.541+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:02.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:02.641 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:02.642+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:02.669 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:02.669 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:02.669 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:02.669 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:02.678 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:02.743 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:02.744+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:02.830 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:02.831 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:02.831 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:02.831 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:02.831 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:02.840 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:02.843 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:02.844+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:02.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.845+0000 7fcfd4600640 1 Processor -- start 2024-04-25T12:36:02.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.845+0000 7fcfd4600640 1 -- start start 2024-04-25T12:36:02.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.847+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.847+0000 7fcfd4600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfcc074b20 con 0x7fcfcc154650 2024-04-25T12:36:02.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.847+0000 7fcfd4600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfcc0ca780 con 0x7fcfcc0730e0 2024-04-25T12:36:02.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.847+0000 7fcfd4600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcfcc0caa60 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.848+0000 7fcfd1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.848+0000 7fcfd2600640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcfcc0730e0 0x7fcfcc164d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:52542/0 (socket says 172.21.15.123:52542) 2024-04-25T12:36:02.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.848+0000 7fcfd2600640 1 -- 172.21.15.123:0/1694035211 learned_addr learned my addr 172.21.15.123:0/1694035211 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:02.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.848+0000 7fcfcbe00640 1 -- 172.21.15.123:0/1694035211 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 406287859 0 0) 0x7fcfcc0ca780 con 0x7fcfcc0730e0 2024-04-25T12:36:02.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.848+0000 7fcfd1200640 1 -- 172.21.15.123:0/1694035211 >> v1:172.21.15.153:6789/0 conn(0x7fcfcc0730e0 legacy=0x7fcfcc164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:02.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.848+0000 7fcfd1200640 1 -- 172.21.15.123:0/1694035211 >> v1:172.21.15.123:6789/0 conn(0x7fcfcc154650 legacy=0x7fcfcc15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:02.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.848+0000 7fcfd1200640 1 -- 172.21.15.123:0/1694035211 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc15da50 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.850+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1694035211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c8170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80097d0 tx=0x7fcfb802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49daf23de3411247 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.850+0000 7fcfcbe00640 1 -- 172.21.15.123:0/1694035211 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8037070 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.850+0000 7fcfcbe00640 1 -- 172.21.15.123:0/1694035211 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb802ae30 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.850+0000 7fcfcbe00640 1 -- 172.21.15.123:0/1694035211 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8032640 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.851+0000 7fcfd4600640 1 -- 172.21.15.123:0/1694035211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 msgr2=0x7fcfcc0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:02.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.851+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1694035211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c8170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80097d0 tx=0x7fcfb802a1a0 comp rx=0 tx=0).stop 2024-04-25T12:36:02.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.851+0000 7fcfd4600640 1 -- 172.21.15.123:0/1694035211 shutdown_connections 2024-04-25T12:36:02.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.851+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1694035211 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c8170 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:02.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.851+0000 7fcfd4600640 1 -- 172.21.15.123:0/1694035211 >> 172.21.15.123:0/1694035211 conn(0x7fcfcc06f440 msgr2=0x7fcfcc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:02.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.851+0000 7fcfd4600640 1 -- 172.21.15.123:0/1694035211 shutdown_connections 2024-04-25T12:36:02.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.851+0000 7fcfd4600640 1 -- 172.21.15.123:0/1694035211 wait complete. 2024-04-25T12:36:02.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.853+0000 7fcfd4600640 1 Processor -- start 2024-04-25T12:36:02.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.853+0000 7fcfd4600640 1 -- start start 2024-04-25T12:36:02.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfcc0730e0 0x7fcfcc1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfcc154650 0x7fcfcc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd4600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc15e2f0 con 0x7fcfcc154650 2024-04-25T12:36:02.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd4600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc15e010 con 0x7fcfcc0730e0 2024-04-25T12:36:02.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd4600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcfcc154a60 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfcc154650 0x7fcfcc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd1c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfcc0730e0 0x7fcfcc1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:51438/0 (socket says 172.21.15.123:51438) 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.856+0000 7fcfd1200640 1 -- 172.21.15.123:0/1849698277 learned_addr learned my addr 172.21.15.123:0/1849698277 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd1200640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfcc0730e0 msgr2=0x7fcfcc1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfcc0730e0 0x7fcfcc1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd1200640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfcc154650 msgr2=0x7fcfcc0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfcc154650 0x7fcfcc0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd1200640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc1539a0 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd2600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfcc154650 0x7fcfcc0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:02.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c6b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8037040 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb8004030 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8004420 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.857+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfcc153c80 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.858+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfcc0c29f0 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.859+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fcfcc076700 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.860+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcfb803a040 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.861+0000 7fcfc9600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcfac0744b0 0x7fcfac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.862+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 6619+0+0 (secure 0 0 0) 0x7fcfb80b5ae0 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.862+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcfac0744b0 0x7fcfac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.862+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v49) ==== 54+0+16 (secure 0 0 0) 0x7fcfb8042020 con 0x7fcfcc0c9f00 2024-04-25T12:36:02.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.862+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fcfcc0c41d0 0x7fcfcc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.862+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc153e50 con 0x7fcfcc0c41d0 2024-04-25T12:36:02.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.862+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcfac0744b0 0x7fcfac0769c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0000f60 tx=0x7fcfc0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.862+0000 7fcfd2600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fcfcc0c41d0 0x7fcfcc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.863+0000 7fcfd2600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fcfcc0c41d0 0x7fcfcc0b28c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.864+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc153e50 con 0x7fcfcc0c41d0 2024-04-25T12:36:02.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.880+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc158010 con 0x7fcfcc0c41d0 2024-04-25T12:36:02.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.882+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc158010 con 0x7fcfcc0c41d0 2024-04-25T12:36:02.881 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:02.882 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:02.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.882+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fcfcc15c7e0 0x7fcfcc15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.882+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc0744e0 con 0x7fcfcc15c7e0 2024-04-25T12:36:02.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.883+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fcfcc15c7e0 0x7fcfcc15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.883+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fcfcc15c7e0 0x7fcfcc15cbf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.884+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc0744e0 con 0x7fcfcc15c7e0 2024-04-25T12:36:02.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.900+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc073c20 con 0x7fcfcc15c7e0 2024-04-25T12:36:02.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.902+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc073c20 con 0x7fcfcc15c7e0 2024-04-25T12:36:02.902 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:02.902 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:02.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.903+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcfcc062fe0 0x7fcfcc0633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.903+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc15d640 con 0x7fcfcc062fe0 2024-04-25T12:36:02.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.903+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcfcc062fe0 0x7fcfcc0633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.904+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcfcc062fe0 0x7fcfcc0633f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.905+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc15d640 con 0x7fcfcc062fe0 2024-04-25T12:36:02.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.921+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc0cb960 con 0x7fcfcc062fe0 2024-04-25T12:36:02.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.923+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc0cb960 con 0x7fcfcc062fe0 2024-04-25T12:36:02.922 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:02.923 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:02.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.923+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fcfcc063cf0 0x7fcfcc0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.923+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc0cb680 con 0x7fcfcc063cf0 2024-04-25T12:36:02.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.924+0000 7fcfd2600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fcfcc063cf0 0x7fcfcc0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.924+0000 7fcfd2600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fcfcc063cf0 0x7fcfcc0c8370 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.925+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc0cb680 con 0x7fcfcc063cf0 2024-04-25T12:36:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.941+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc0c8b80 con 0x7fcfcc063cf0 2024-04-25T12:36:02.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.944+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc0c8b80 con 0x7fcfcc063cf0 2024-04-25T12:36:02.943 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:02.943 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.944+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fcfcc0c8b00 0x7fcfcc02cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.944+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc02d420 con 0x7fcfcc0c8b00 2024-04-25T12:36:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.944+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fcfcc0c8b00 0x7fcfcc02cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.945+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fcfcc0c8b00 0x7fcfcc02cee0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:02.945 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:02.946+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.947+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc02d420 con 0x7fcfcc0c8b00 2024-04-25T12:36:02.954 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:02.954 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:02.954 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:02.954 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:02.954 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.954 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.004750966, 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.955 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:36:02.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:36:02.959 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.004663257, 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.960 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.004537329, 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.965 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.968 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.970 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.00447567000001, 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:36:02.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:36:02.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.004418907, 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:36:02.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:36:02.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:36:02.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:36:02.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.004355437, 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:36:02.981 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.986 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.00430348800001, 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:02.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.990 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.00425153800001, 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.993 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:36:02.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:36:02.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:02.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.004134429, 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:02.997 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:36:02.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:36:02.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:36:03.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:36:03.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 104.004095982, 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:36:03.002 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:36:03.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:36:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.006 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 72.838960928999995, 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:36:03.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:36:03.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 72.838902877999999, 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.012 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:36:03.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 72.838843185000002, 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:36:03.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:36:03.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:36:03.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.801923103000007, 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.023 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:36:03.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:36:03.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:03.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:36:03.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.801834137, 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:36:03.028 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.033 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.801790295999993, 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:36:03.038 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.800276392000001, 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.799357176000001, 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:03.044 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:36:03.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:36:03.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:36:03.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:36:03.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.798901822999994, 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.049 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 67.798872845000005, 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:03.053 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:03.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:36:03.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:36:03.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:36:03.057 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.057 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.057 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.057 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:03.057 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:03.057 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:03.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.963+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc02d420 con 0x7fcfcc0c8b00 2024-04-25T12:36:03.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.966+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc02d420 con 0x7fcfcc0c8b00 2024-04-25T12:36:03.058 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:03.059 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:03.060 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:03.059+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:03.060 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.966+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fcfcc02db70 0x7fcfcc209570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.966+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc20c340 con 0x7fcfcc02db70 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.966+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fcfcc02db70 0x7fcfcc209570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.967+0000 7fcfd1200640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fcfcc02db70 0x7fcfcc209570 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.968+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc20c340 con 0x7fcfcc02db70 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.984+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc209ab0 con 0x7fcfcc02db70 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.986+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc209ab0 con 0x7fcfcc02db70 2024-04-25T12:36:03.061 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:03.062 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:03.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.986+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcfcc20a130 0x7fcfcc20c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.986+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc20f320 con 0x7fcfcc20a130 2024-04-25T12:36:03.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.986+0000 7fcfd2600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcfcc20a130 0x7fcfcc20c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.987+0000 7fcfd2600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcfcc20a130 0x7fcfcc20c550 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:03.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:02.988+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc20f320 con 0x7fcfcc20a130 2024-04-25T12:36:03.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.003+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc20ca90 con 0x7fcfcc20a130 2024-04-25T12:36:03.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.005+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc20ca90 con 0x7fcfcc20a130 2024-04-25T12:36:03.063 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:03.064 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:03.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.005+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fcfcc20d110 0x7fcfcc20f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.005+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fcfcc2123d0 con 0x7fcfcc20d110 2024-04-25T12:36:03.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.006+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fcfcc20d110 0x7fcfcc20f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.006+0000 7fcfd1c00640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fcfcc20d110 0x7fcfcc20f530 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:03.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.007+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfcc2123d0 con 0x7fcfcc20d110 2024-04-25T12:36:03.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.022+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcfcc20fa70 con 0x7fcfcc20d110 2024-04-25T12:36:03.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfc9600640 1 -- 172.21.15.123:0/1849698277 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfcc20fa70 con 0x7fcfcc20d110 2024-04-25T12:36:03.065 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fcfcc0c41d0 msgr2=0x7fcfcc0b28c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fcfcc0c41d0 0x7fcfcc0b28c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fcfcc15c7e0 msgr2=0x7fcfcc15cbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fcfcc15c7e0 0x7fcfcc15cbf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcfcc062fe0 msgr2=0x7fcfcc0633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcfcc062fe0 0x7fcfcc0633f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fcfcc063cf0 msgr2=0x7fcfcc0c8370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fcfcc063cf0 0x7fcfcc0c8370 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fcfcc0c8b00 msgr2=0x7fcfcc02cee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fcfcc0c8b00 0x7fcfcc02cee0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fcfcc02db70 msgr2=0x7fcfcc209570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fcfcc02db70 0x7fcfcc209570 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcfcc20a130 msgr2=0x7fcfcc20c550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcfcc20a130 0x7fcfcc20c550 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fcfcc20d110 msgr2=0x7fcfcc20f530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fcfcc20d110 0x7fcfcc20f530 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd2600640 1 -- 172.21.15.123:0/1849698277 reap_dead start 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd2600640 1 -- 172.21.15.123:0/1849698277 reap_dead start 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd2600640 1 -- 172.21.15.123:0/1849698277 reap_dead start 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd2600640 1 -- 172.21.15.123:0/1849698277 reap_dead start 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.024+0000 7fcfd2600640 1 -- 172.21.15.123:0/1849698277 reap_dead start 2024-04-25T12:36:03.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcfac0744b0 msgr2=0x7fcfac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcfac0744b0 0x7fcfac0769c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0000f60 tx=0x7fcfc0009040 comp rx=0 tx=0).stop 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 msgr2=0x7fcfcc0c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c6b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb80046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 shutdown_connections 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcfac0744b0 0x7fcfac0769c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 --2- 172.21.15.123:0/1849698277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcfcc0c9f00 0x7fcfcc0c6b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 >> 172.21.15.123:0/1849698277 conn(0x7fcfcc06f440 msgr2=0x7fcfcc145cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 shutdown_connections 2024-04-25T12:36:03.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.025+0000 7fcfd4600640 1 -- 172.21.15.123:0/1849698277 wait complete. 2024-04-25T12:36:03.159 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:03.160+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:03.212 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:03.212 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:03.221 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:03.262 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:03.263+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:03.362 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:03.362+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:03.372 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:03.372 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:03.381 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:03.462 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:03.463+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:03.493 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:03.493 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:03.502 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:03.563 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:03.564+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:03.616 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:03.617 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:03.627 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:03.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:36:03.664 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:03.665+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:03.737 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:03.746 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:03.764 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:03.765+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:03.858 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:03.858 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:03.864 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:03.865+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:03.867 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:03.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.916+0000 7fb221800640 1 Processor -- start 2024-04-25T12:36:03.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.916+0000 7fb221800640 1 -- start start 2024-04-25T12:36:03.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb221800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb221800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb21c15c060 0x7fb21c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb221800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb21c0ccf10 0x7fb21c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb221800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb21c15d6b0 con 0x7fb21c0ccf10 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb221800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb21c0d2760 con 0x7fb21c15c060 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb221800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c0ceb90 con 0x7fb21c0d1d80 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb21be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb220e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb21c0ccf10 0x7fb21c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.918+0000 7fb21b400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb21c15c060 0x7fb21c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c15d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:51442/0 (socket says 172.21.15.123:51442) 2024-04-25T12:36:03.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21be00640 1 -- 172.21.15.123:0/3887491957 learned_addr learned my addr 172.21.15.123:0/3887491957 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:03.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21be00640 1 -- 172.21.15.123:0/3887491957 >> v2:172.21.15.153:3300/0 conn(0x7fb21c15c060 msgr2=0x7fb21c15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:03.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21be00640 1 --2- 172.21.15.123:0/3887491957 >> v2:172.21.15.153:3300/0 conn(0x7fb21c15c060 0x7fb21c15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21be00640 1 -- 172.21.15.123:0/3887491957 >> v2:172.21.15.123:3300/0 conn(0x7fb21c0ccf10 msgr2=0x7fb21c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21be00640 1 --2- 172.21.15.123:0/3887491957 >> v2:172.21.15.123:3300/0 conn(0x7fb21c0ccf10 0x7fb21c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21be00640 1 -- 172.21.15.123:0/3887491957 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c0d2480 con 0x7fb21c0d1d80 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb220e00640 1 --2- 172.21.15.123:0/3887491957 >> v2:172.21.15.123:3300/0 conn(0x7fb21c0ccf10 0x7fb21c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.919+0000 7fb21b400640 1 --2- 172.21.15.123:0/3887491957 >> v2:172.21.15.153:3300/0 conn(0x7fb21c15c060 0x7fb21c15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.920+0000 7fb21be00640 1 --2- 172.21.15.123:0/3887491957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c15d170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb21c08e5c0 tx=0x7fb20c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de59af762ab9d052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.920+0000 7fb21aa00640 1 -- 172.21.15.123:0/3887491957 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c037070 con 0x7fb21c0d1d80 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.920+0000 7fb21aa00640 1 -- 172.21.15.123:0/3887491957 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c032440 con 0x7fb21c0d1d80 2024-04-25T12:36:03.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb21aa00640 1 -- 172.21.15.123:0/3887491957 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c032810 con 0x7fb21c0d1d80 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 -- 172.21.15.123:0/3887491957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 msgr2=0x7fb21c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 --2- 172.21.15.123:0/3887491957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c15d170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb21c08e5c0 tx=0x7fb20c02a270 comp rx=0 tx=0).stop 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 -- 172.21.15.123:0/3887491957 shutdown_connections 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 --2- 172.21.15.123:0/3887491957 >> v2:172.21.15.123:3300/0 conn(0x7fb21c0ccf10 0x7fb21c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 --2- 172.21.15.123:0/3887491957 >> v2:172.21.15.153:3300/0 conn(0x7fb21c15c060 0x7fb21c15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 --2- 172.21.15.123:0/3887491957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c15d170 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 -- 172.21.15.123:0/3887491957 >> 172.21.15.123:0/3887491957 conn(0x7fb21c08da20 msgr2=0x7fb21c08de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 -- 172.21.15.123:0/3887491957 shutdown_connections 2024-04-25T12:36:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.921+0000 7fb221800640 1 -- 172.21.15.123:0/3887491957 wait complete. 2024-04-25T12:36:03.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.924+0000 7fb221800640 1 Processor -- start 2024-04-25T12:36:03.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.924+0000 7fb221800640 1 -- start start 2024-04-25T12:36:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb221800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb21c0ccf10 0x7fb21c1e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb221800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c1e2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb221800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb21c15c060 0x7fb21c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb221800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c0ce820 con 0x7fb21c15c060 2024-04-25T12:36:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb221800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c0ce540 con 0x7fb21c0ccf10 2024-04-25T12:36:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb221800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c0d2190 con 0x7fb21c0d1d80 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb21be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb21c0ccf10 0x7fb21c1e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb21b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c1e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb21be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb21c0ccf10 0x7fb21c1e5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41478/0 (socket says 172.21.15.123:41478) 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb21b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c1e2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:51444/0 (socket says 172.21.15.123:51444) 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb21be00640 1 -- 172.21.15.123:0/4294345588 learned_addr learned my addr 172.21.15.123:0/4294345588 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.926+0000 7fb220e00640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb21c15c060 0x7fb21c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb21be00640 1 -- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 msgr2=0x7fb21c1e2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb21be00640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c1e2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb21be00640 1 -- 172.21.15.123:0/4294345588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb21c15c060 msgr2=0x7fb21c157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb21be00640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb21c15c060 0x7fb21c157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb21be00640 1 -- 172.21.15.123:0/4294345588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c1e56d0 con 0x7fb21c0ccf10 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb21b400640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c1e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb220e00640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb21c15c060 0x7fb21c157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.927+0000 7fb21be00640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb21c0ccf10 0x7fb21c1e5190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb20c02a2b0 tx=0x7fb20c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.928+0000 7fb213e00640 1 -- 172.21.15.123:0/4294345588 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c041020 con 0x7fb21c0ccf10 2024-04-25T12:36:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.928+0000 7fb213e00640 1 -- 172.21.15.123:0/4294345588 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c03d640 con 0x7fb21c0ccf10 2024-04-25T12:36:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.928+0000 7fb213e00640 1 -- 172.21.15.123:0/4294345588 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c037040 con 0x7fb21c0ccf10 2024-04-25T12:36:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.928+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c1e4150 con 0x7fb21c0ccf10 2024-04-25T12:36:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.928+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c1e46d0 con 0x7fb21c0ccf10 2024-04-25T12:36:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.929+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21c076da0 con 0x7fb21c0ccf10 2024-04-25T12:36:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.930+0000 7fb213e00640 1 -- 172.21.15.123:0/4294345588 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb20c044020 con 0x7fb21c0ccf10 2024-04-25T12:36:03.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.931+0000 7fb213e00640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb1f80744e0 0x7fb1f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:03.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.931+0000 7fb213e00640 1 -- 172.21.15.123:0/4294345588 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6619+0+0 (secure 0 0 0) 0x7fb20c0b5bb0 con 0x7fb21c0ccf10 2024-04-25T12:36:03.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.932+0000 7fb21b400640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb1f80744e0 0x7fb1f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:03.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.932+0000 7fb21b400640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb1f80744e0 0x7fb1f80769f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb204009560 tx=0x7fb204009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:03.936+0000 7fb213e00640 1 -- 172.21.15.123:0/4294345588 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb20c04b030 con 0x7fb21c0ccf10 2024-04-25T12:36:03.966 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:03.966+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:04.020 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:04.020 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:04.029 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:04.064 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:04.064 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:04.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:36:04.066 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:04.067+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:04.166 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:04.167+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:04.180 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:04.180 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:04.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.184+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fb21c0691e0 con 0x7fb21c0ccf10 2024-04-25T12:36:04.189 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:04.273 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:04.274+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:04.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.318+0000 7f9d8e600640 1 Processor -- start 2024-04-25T12:36:04.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.318+0000 7f9d8e600640 1 -- start start 2024-04-25T12:36:04.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8e600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9d8806db80 0x7f9d880c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8e600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9d88156980 0x7f9d880c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8e600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d880c9a20 0x7f9d880c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8e600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9d880c8be0 con 0x7f9d8806db80 2024-04-25T12:36:04.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8e600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9d880c9570 con 0x7f9d88156980 2024-04-25T12:36:04.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8e600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9d880c8900 con 0x7f9d880c9a20 2024-04-25T12:36:04.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9d8806db80 0x7f9d880c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9d8806db80 0x7f9d880c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53744/0 (socket says 172.21.15.123:53744) 2024-04-25T12:36:04.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d87e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9d88156980 0x7f9d880c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8d200640 1 -- 172.21.15.123:0/2747163673 learned_addr learned my addr 172.21.15.123:0/2747163673 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:04.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.321+0000 7f9d8dc00640 1 --2- 172.21.15.123:0/2747163673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d880c9a20 0x7f9d880c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.322+0000 7f9d8d200640 1 -- 172.21.15.123:0/2747163673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d880c9a20 msgr2=0x7f9d880c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:04.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.322+0000 7f9d8d200640 1 --2- 172.21.15.123:0/2747163673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d880c9a20 0x7f9d880c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.322+0000 7f9d8d200640 1 -- 172.21.15.123:0/2747163673 >> v2:172.21.15.153:3300/0 conn(0x7f9d88156980 msgr2=0x7f9d880c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:04.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.322+0000 7f9d8d200640 1 --2- 172.21.15.123:0/2747163673 >> v2:172.21.15.153:3300/0 conn(0x7f9d88156980 0x7f9d880c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.322+0000 7f9d8d200640 1 -- 172.21.15.123:0/2747163673 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8815cfd0 con 0x7f9d8806db80 2024-04-25T12:36:04.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.322+0000 7f9d8dc00640 1 --2- 172.21.15.123:0/2747163673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d880c9a20 0x7f9d880c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:04.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.323+0000 7f9d8d200640 1 --2- 172.21.15.123:0/2747163673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d8806db80 0x7f9d880c5410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9d8806db10 tx=0x7f9d7c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16ad11a71e4caee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:04.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.323+0000 7f9d87400640 1 -- 172.21.15.123:0/2747163673 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c037070 con 0x7f9d8806db80 2024-04-25T12:36:04.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.323+0000 7f9d87400640 1 -- 172.21.15.123:0/2747163673 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c032440 con 0x7f9d8806db80 2024-04-25T12:36:04.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.323+0000 7f9d87400640 1 -- 172.21.15.123:0/2747163673 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c032810 con 0x7f9d8806db80 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 -- 172.21.15.123:0/2747163673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d8806db80 msgr2=0x7f9d880c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2747163673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d8806db80 0x7f9d880c5410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9d8806db10 tx=0x7f9d7c02a1d0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 -- 172.21.15.123:0/2747163673 shutdown_connections 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2747163673 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d880c9a20 0x7f9d880c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2747163673 >> v2:172.21.15.153:3300/0 conn(0x7f9d88156980 0x7f9d880c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2747163673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d8806db80 0x7f9d880c5410 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 -- 172.21.15.123:0/2747163673 >> 172.21.15.123:0/2747163673 conn(0x7f9d880ca860 msgr2=0x7f9d880cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 -- 172.21.15.123:0/2747163673 shutdown_connections 2024-04-25T12:36:04.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.324+0000 7f9d8e600640 1 -- 172.21.15.123:0/2747163673 wait complete. 2024-04-25T12:36:04.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.327+0000 7f9d8e600640 1 Processor -- start 2024-04-25T12:36:04.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.327+0000 7f9d8e600640 1 -- start start 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.329+0000 7f9d8e600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d8806db80 0x7f9d8815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.329+0000 7f9d8e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880c9a20 0x7f9d88151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.329+0000 7f9d8e600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d88156980 0x7f9d8814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.329+0000 7f9d8e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880c90f0 con 0x7f9d880c9a20 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.329+0000 7f9d8e600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880c8f70 con 0x7f9d8806db80 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.329+0000 7f9d8e600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9d880c9270 con 0x7f9d88156980 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8d200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d8806db80 0x7f9d8815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d88156980 0x7f9d8814c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d87e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880c9a20 0x7f9d88151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8d200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d8806db80 0x7f9d8815ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41504/0 (socket says 172.21.15.123:41504) 2024-04-25T12:36:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8d200640 1 -- 172.21.15.123:0/2083958191 learned_addr learned my addr 172.21.15.123:0/2083958191 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:04.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8dc00640 1 -- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d8806db80 msgr2=0x7f9d8815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8dc00640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d8806db80 0x7f9d8815ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8dc00640 1 -- 172.21.15.123:0/2083958191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880c9a20 msgr2=0x7f9d88151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8dc00640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880c9a20 0x7f9d88151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8dc00640 1 -- 172.21.15.123:0/2083958191 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d88154650 con 0x7f9d88156980 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8d200640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d8806db80 0x7f9d8815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.330+0000 7f9d8dc00640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d88156980 0x7f9d8814c950 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000eea0 tx=0x7f9d8000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.331+0000 7f9d84c00640 1 -- 172.21.15.123:0/2083958191 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80018020 con 0x7f9d88156980 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.331+0000 7f9d84c00640 1 -- 172.21.15.123:0/2083958191 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d80014400 con 0x7f9d88156980 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.331+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8815f140 con 0x7f9d88156980 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.331+0000 7f9d84c00640 1 -- 172.21.15.123:0/2083958191 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d800147f0 con 0x7f9d88156980 2024-04-25T12:36:04.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.331+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8815e000 con 0x7f9d88156980 2024-04-25T12:36:04.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.333+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d88155b70 con 0x7f9d88156980 2024-04-25T12:36:04.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.334+0000 7f9d84c00640 1 -- 172.21.15.123:0/2083958191 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9d80005d10 con 0x7f9d88156980 2024-04-25T12:36:04.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.335+0000 7f9d84c00640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9d680744e0 0x7f9d680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.335+0000 7f9d84c00640 1 -- 172.21.15.123:0/2083958191 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 6619+0+0 (secure 0 0 0) 0x7f9d80095d10 con 0x7f9d88156980 2024-04-25T12:36:04.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.335+0000 7f9d8d200640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9d680744e0 0x7f9d680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.336+0000 7f9d8d200640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9d680744e0 0x7f9d680769f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c035040 tx=0x7f9d7c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:04.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.339+0000 7f9d84c00640 1 -- 172.21.15.123:0/2083958191 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d8009b020 con 0x7f9d88156980 2024-04-25T12:36:04.343 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:04.343 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:04.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:04.400 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:04.401+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:04.470 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:04.471+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:04.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.551+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d88076da0 con 0x7f9d680744e0 2024-04-25T12:36:04.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.561+0000 7f9d84c00640 1 -- 172.21.15.123:0/2083958191 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333515 (secure 0 0 0) 0x7f9d88076da0 con 0x7f9d680744e0 2024-04-25T12:36:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:36:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.566+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9d680744e0 msgr2=0x7f9d680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.566+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9d680744e0 0x7f9d680769f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c035040 tx=0x7f9d7c02e000 comp rx=0 tx=0).stop 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.566+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d88156980 msgr2=0x7f9d8814c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.566+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d88156980 0x7f9d8814c950 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000eea0 tx=0x7f9d8000ef90 comp rx=0 tx=0).stop 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 shutdown_connections 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9d680744e0 0x7f9d680769f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d88156980 0x7f9d8814c950 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d880c9a20 0x7f9d88151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 --2- 172.21.15.123:0/2083958191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d8806db80 0x7f9d8815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 >> 172.21.15.123:0/2083958191 conn(0x7f9d880ca860 msgr2=0x7f9d880cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:04.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 shutdown_connections 2024-04-25T12:36:04.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.567+0000 7f9d8e600640 1 -- 172.21.15.123:0/2083958191 wait complete. 2024-04-25T12:36:04.570 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:04.571+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:04.589 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T12:36:02.820640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601885,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":242,"num_write_kb":1682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":301,"acting":301,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221724,"kb_used_data":4336,"kb_used_omap":20,"kb_used_meta":217259,"kb_avail":754752996,"statfs":{"total":773094113280,"available":772867067904,"internally_reserved":0,"allocated":4440064,"data_stored":3205448,"data_compressed":66734,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":20525,"internal_metadata":222474195},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.255418"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":76,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.618575+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:36:00.618575+0000","last_peered":"2024-04-25T12:36:00.618575+0000","last_clean":"2024-04-25T12:36:00.618575+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:36:00.618575+0000","last_undegraded":"2024-04-25T12:36:00.618575+0000","last_fullsized":"2024-04-25T12:36:00.618575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.686165+0000","last_change":"2024-04-25T12:35:45.507223+0000","last_active":"2024-04-25T12:35:56.686165+0000","last_peered":"2024-04-25T12:35:56.686165+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.507223+0000","last_became_peered":"2024-04-25T12:35:45.507223+0000","last_unstale":"2024-04-25T12:35:56.686165+0000","last_undegraded":"2024-04-25T12:35:56.686165+0000","last_fullsized":"2024-04-25T12:35:45.500102+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:48.667371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":60,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.607820+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:36:00.607820+0000","last_peered":"2024-04-25T12:36:00.607820+0000","last_clean":"2024-04-25T12:36:00.607820+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:36:00.607820+0000","last_undegraded":"2024-04-25T12:36:00.607820+0000","last_fullsized":"2024-04-25T12:36:00.607820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.564903+0000","last_change":"2024-04-25T12:35:45.509602+0000","last_active":"2024-04-25T12:35:55.564903+0000","last_peered":"2024-04-25T12:35:55.564903+0000","last_clean":"2024-04-25T12:34:11.863765+0000","last_became_active":"2024-04-25T12:35:45.509602+0000","last_became_peered":"2024-04-25T12:35:45.509602+0000","last_unstale":"2024-04-25T12:35:55.564903+0000","last_undegraded":"2024-04-25T12:35:55.564903+0000","last_fullsized":"2024-04-25T12:35:45.503282+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:48.839277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.758515+0000","last_change":"2024-04-25T12:35:45.517024+0000","last_active":"2024-04-25T12:35:56.758515+0000","last_peered":"2024-04-25T12:35:56.758515+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.517024+0000","last_became_peered":"2024-04-25T12:35:45.517024+0000","last_unstale":"2024-04-25T12:35:56.758515+0000","last_undegraded":"2024-04-25T12:35:56.758515+0000","last_fullsized":"2024-04-25T12:35:45.507573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:30.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":61,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.612475+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:36:01.612475+0000","last_peered":"2024-04-25T12:36:01.612475+0000","last_clean":"2024-04-25T12:36:01.612475+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:36:01.612475+0000","last_undegraded":"2024-04-25T12:36:01.612475+0000","last_fullsized":"2024-04-25T12:36:01.612475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.574394+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:35:55.574394+0000","last_peered":"2024-04-25T12:35:55.574394+0000","last_clean":"2024-04-25T12:35:55.574394+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:35:55.574394+0000","last_undegraded":"2024-04-25T12:35:55.574394+0000","last_fullsized":"2024-04-25T12:35:55.574394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":67,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.617165+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:36:01.617165+0000","last_peered":"2024-04-25T12:36:01.617165+0000","last_clean":"2024-04-25T12:36:01.617165+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:36:01.617165+0000","last_undegraded":"2024-04-25T12:36:01.617165+0000","last_fullsized":"2024-04-25T12:36:01.617165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":72,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.614549+0000","last_change":"2024-04-25T12:36:00.627223+0000","last_active":"2024-04-25T12:36:01.614549+0000","last_peered":"2024-04-25T12:36:01.614549+0000","last_clean":"2024-04-25T12:36:01.614549+0000","last_became_active":"2024-04-25T12:36:00.626675+0000","last_became_peered":"2024-04-25T12:36:00.626675+0000","last_unstale":"2024-04-25T12:36:01.614549+0000","last_undegraded":"2024-04-25T12:36:01.614549+0000","last_fullsized":"2024-04-25T12:36:01.614549+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:11.204775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.574023+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:35:55.574023+0000","last_peered":"2024-04-25T12:35:55.574023+0000","last_clean":"2024-04-25T12:35:55.574023+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:35:55.574023+0000","last_undegraded":"2024-04-25T12:35:55.574023+0000","last_fullsized":"2024-04-25T12:35:55.574023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.759990+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:35:56.759990+0000","last_peered":"2024-04-25T12:35:56.759990+0000","last_clean":"2024-04-25T12:35:56.759990+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:35:56.759990+0000","last_undegraded":"2024-04-25T12:35:56.759990+0000","last_fullsized":"2024-04-25T12:35:56.759990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":72,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.613283+0000","last_change":"2024-04-25T12:36:00.623046+0000","last_active":"2024-04-25T12:36:01.613283+0000","last_peered":"2024-04-25T12:36:01.613283+0000","last_clean":"2024-04-25T12:36:01.613283+0000","last_became_active":"2024-04-25T12:36:00.622702+0000","last_became_peered":"2024-04-25T12:36:00.622702+0000","last_unstale":"2024-04-25T12:36:01.613283+0000","last_undegraded":"2024-04-25T12:36:01.613283+0000","last_fullsized":"2024-04-25T12:36:01.613283+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760195+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:35:56.760195+0000","last_peered":"2024-04-25T12:35:56.760195+0000","last_clean":"2024-04-25T12:35:56.760195+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:35:56.760195+0000","last_undegraded":"2024-04-25T12:35:56.760195+0000","last_fullsized":"2024-04-25T12:35:56.760195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760242+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:35:56.760242+0000","last_peered":"2024-04-25T12:35:56.760242+0000","last_clean":"2024-04-25T12:35:56.760242+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:35:56.760242+0000","last_undegraded":"2024-04-25T12:35:56.760242+0000","last_fullsized":"2024-04-25T12:35:56.760242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":51,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.572431+0000","last_change":"2024-04-25T12:35:45.527560+0000","last_active":"2024-04-25T12:35:55.572431+0000","last_peered":"2024-04-25T12:35:55.572431+0000","last_clean":"2024-04-25T12:35:44.473628+0000","last_became_active":"2024-04-25T12:35:45.527560+0000","last_became_peered":"2024-04-25T12:35:45.527560+0000","last_unstale":"2024-04-25T12:35:55.572431+0000","last_undegraded":"2024-04-25T12:35:55.572431+0000","last_fullsized":"2024-04-25T12:35:45.515701+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:03.477382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,2147483647],"acting":[0,1,7,2147483647],"avail_no_missing":["0(0)","1(1)","7(2)"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.608627+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:36:00.608627+0000","last_peered":"2024-04-25T12:36:00.608627+0000","last_clean":"2024-04-25T12:36:00.608627+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:36:00.608627+0000","last_undegraded":"2024-04-25T12:36:00.608627+0000","last_fullsized":"2024-04-25T12:36:00.608627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.626426+0000","last_change":"2024-04-25T12:36:00.626426+0000","last_active":"2024-04-25T12:36:00.626426+0000","last_peered":"2024-04-25T12:36:00.626426+0000","last_clean":"2024-04-25T12:36:00.626426+0000","last_became_active":"2024-04-25T12:36:00.625818+0000","last_became_peered":"2024-04-25T12:36:00.625818+0000","last_unstale":"2024-04-25T12:36:00.626426+0000","last_undegraded":"2024-04-25T12:36:00.626426+0000","last_fullsized":"2024-04-25T12:36:00.626426+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:06.117031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":53,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.573466+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:35:55.573466+0000","last_peered":"2024-04-25T12:35:55.573466+0000","last_clean":"2024-04-25T12:35:55.573466+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:35:55.573466+0000","last_undegraded":"2024-04-25T12:35:55.573466+0000","last_fullsized":"2024-04-25T12:35:55.573466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.617130+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:36:00.617130+0000","last_peered":"2024-04-25T12:36:00.617130+0000","last_clean":"2024-04-25T12:36:00.617130+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:36:00.617130+0000","last_undegraded":"2024-04-25T12:36:00.617130+0000","last_fullsized":"2024-04-25T12:36:00.617130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.618740+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:36:00.618740+0000","last_peered":"2024-04-25T12:36:00.618740+0000","last_clean":"2024-04-25T12:36:00.618740+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:36:00.618740+0000","last_undegraded":"2024-04-25T12:36:00.618740+0000","last_fullsized":"2024-04-25T12:36:00.618740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":60,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.608317+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:36:00.608317+0000","last_peered":"2024-04-25T12:36:00.608317+0000","last_clean":"2024-04-25T12:36:00.608317+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:36:00.608317+0000","last_undegraded":"2024-04-25T12:36:00.608317+0000","last_fullsized":"2024-04-25T12:36:00.608317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760619+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:35:56.760619+0000","last_peered":"2024-04-25T12:35:56.760619+0000","last_clean":"2024-04-25T12:35:56.760619+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:35:56.760619+0000","last_undegraded":"2024-04-25T12:35:56.760619+0000","last_fullsized":"2024-04-25T12:35:56.760619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.572090+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:35:55.572090+0000","last_peered":"2024-04-25T12:35:55.572090+0000","last_clean":"2024-04-25T12:35:55.572090+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:35:55.572090+0000","last_undegraded":"2024-04-25T12:35:55.572090+0000","last_fullsized":"2024-04-25T12:35:55.572090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":61,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.615935+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:36:01.615935+0000","last_peered":"2024-04-25T12:36:01.615935+0000","last_clean":"2024-04-25T12:36:01.615935+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:36:01.615935+0000","last_undegraded":"2024-04-25T12:36:01.615935+0000","last_fullsized":"2024-04-25T12:36:01.615935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.619708+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:36:00.619708+0000","last_peered":"2024-04-25T12:36:00.619708+0000","last_clean":"2024-04-25T12:36:00.619708+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:36:00.619708+0000","last_undegraded":"2024-04-25T12:36:00.619708+0000","last_fullsized":"2024-04-25T12:36:00.619708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.761555+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:35:56.761555+0000","last_peered":"2024-04-25T12:35:56.761555+0000","last_clean":"2024-04-25T12:35:56.761555+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:35:56.761555+0000","last_undegraded":"2024-04-25T12:35:56.761555+0000","last_fullsized":"2024-04-25T12:35:56.761555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":61,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.612933+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:36:01.612933+0000","last_peered":"2024-04-25T12:36:01.612933+0000","last_clean":"2024-04-25T12:36:01.612933+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:36:01.612933+0000","last_undegraded":"2024-04-25T12:36:01.612933+0000","last_fullsized":"2024-04-25T12:36:01.612933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.618457+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:36:00.618457+0000","last_peered":"2024-04-25T12:36:00.618457+0000","last_clean":"2024-04-25T12:36:00.618457+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:36:00.618457+0000","last_undegraded":"2024-04-25T12:36:00.618457+0000","last_fullsized":"2024-04-25T12:36:00.618457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"43'6","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.619848+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:36:00.619848+0000","last_peered":"2024-04-25T12:36:00.619848+0000","last_clean":"2024-04-25T12:36:00.619848+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:36:00.619848+0000","last_undegraded":"2024-04-25T12:36:00.619848+0000","last_fullsized":"2024-04-25T12:36:00.619848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":59,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.616826+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:36:00.616826+0000","last_peered":"2024-04-25T12:36:00.616826+0000","last_clean":"2024-04-25T12:36:00.616826+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:36:00.616826+0000","last_undegraded":"2024-04-25T12:36:00.616826+0000","last_fullsized":"2024-04-25T12:36:00.616826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":71,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.626931+0000","last_change":"2024-04-25T12:36:00.626931+0000","last_active":"2024-04-25T12:36:00.626931+0000","last_peered":"2024-04-25T12:36:00.626931+0000","last_clean":"2024-04-25T12:36:00.626931+0000","last_became_active":"2024-04-25T12:36:00.626533+0000","last_became_peered":"2024-04-25T12:36:00.626533+0000","last_unstale":"2024-04-25T12:36:00.626931+0000","last_undegraded":"2024-04-25T12:36:00.626931+0000","last_fullsized":"2024-04-25T12:36:00.626931+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.761168+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:35:56.761168+0000","last_peered":"2024-04-25T12:35:56.761168+0000","last_clean":"2024-04-25T12:35:56.761168+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:35:56.761168+0000","last_undegraded":"2024-04-25T12:35:56.761168+0000","last_fullsized":"2024-04-25T12:35:56.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":59,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.613500+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:36:00.613500+0000","last_peered":"2024-04-25T12:36:00.613500+0000","last_clean":"2024-04-25T12:36:00.613500+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:36:00.613500+0000","last_undegraded":"2024-04-25T12:36:00.613500+0000","last_fullsized":"2024-04-25T12:36:00.613500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.615625+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:36:01.615625+0000","last_peered":"2024-04-25T12:36:01.615625+0000","last_clean":"2024-04-25T12:36:01.615625+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:36:01.615625+0000","last_undegraded":"2024-04-25T12:36:01.615625+0000","last_fullsized":"2024-04-25T12:36:01.615625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.756844+0000","last_change":"2024-04-25T12:35:45.506220+0000","last_active":"2024-04-25T12:35:56.756844+0000","last_peered":"2024-04-25T12:35:56.756844+0000","last_clean":"2024-04-25T12:34:11.863765+0000","last_became_active":"2024-04-25T12:35:45.506220+0000","last_became_peered":"2024-04-25T12:35:45.506220+0000","last_unstale":"2024-04-25T12:35:56.756844+0000","last_undegraded":"2024-04-25T12:35:56.756844+0000","last_fullsized":"2024-04-25T12:35:45.501639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:32.158379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.631090+0000","last_change":"2024-04-25T12:36:00.631090+0000","last_active":"2024-04-25T12:36:00.631090+0000","last_peered":"2024-04-25T12:36:00.631090+0000","last_clean":"2024-04-25T12:36:00.631090+0000","last_became_active":"2024-04-25T12:36:00.630484+0000","last_became_peered":"2024-04-25T12:36:00.630484+0000","last_unstale":"2024-04-25T12:36:00.631090+0000","last_undegraded":"2024-04-25T12:36:00.631090+0000","last_fullsized":"2024-04-25T12:36:00.631090+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.738025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.572931+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:35:55.572931+0000","last_peered":"2024-04-25T12:35:55.572931+0000","last_clean":"2024-04-25T12:35:55.572931+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:35:55.572931+0000","last_undegraded":"2024-04-25T12:35:55.572931+0000","last_fullsized":"2024-04-25T12:35:55.572931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.761157+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:35:56.761157+0000","last_peered":"2024-04-25T12:35:56.761157+0000","last_clean":"2024-04-25T12:35:56.761157+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:35:56.761157+0000","last_undegraded":"2024-04-25T12:35:56.761157+0000","last_fullsized":"2024-04-25T12:35:56.761157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":59,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.614985+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:36:00.614985+0000","last_peered":"2024-04-25T12:36:00.614985+0000","last_clean":"2024-04-25T12:36:00.614985+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:36:00.614985+0000","last_undegraded":"2024-04-25T12:36:00.614985+0000","last_fullsized":"2024-04-25T12:36:00.614985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.564787+0000","last_change":"2024-04-25T12:35:45.506886+0000","last_active":"2024-04-25T12:35:55.564787+0000","last_peered":"2024-04-25T12:35:55.564787+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.506886+0000","last_became_peered":"2024-04-25T12:35:45.506886+0000","last_unstale":"2024-04-25T12:35:55.564787+0000","last_undegraded":"2024-04-25T12:35:55.564787+0000","last_fullsized":"2024-04-25T12:35:45.500474+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:30.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.625934+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:36:00.625934+0000","last_peered":"2024-04-25T12:36:00.625934+0000","last_clean":"2024-04-25T12:36:00.625934+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:36:00.625934+0000","last_undegraded":"2024-04-25T12:36:00.625934+0000","last_fullsized":"2024-04-25T12:36:00.625934+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":59,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.613050+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:36:00.613050+0000","last_peered":"2024-04-25T12:36:00.613050+0000","last_clean":"2024-04-25T12:36:00.613050+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:36:00.613050+0000","last_undegraded":"2024-04-25T12:36:00.613050+0000","last_fullsized":"2024-04-25T12:36:00.613050+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.613017+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:36:00.613017+0000","last_peered":"2024-04-25T12:36:00.613017+0000","last_clean":"2024-04-25T12:36:00.613017+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:36:00.613017+0000","last_undegraded":"2024-04-25T12:36:00.613017+0000","last_fullsized":"2024-04-25T12:36:00.613017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":64,"reported_epoch":45,"state":"active+undersized+degraded","last_fresh":"2024-04-25T12:35:56.759528+0000","last_change":"2024-04-25T12:35:45.512537+0000","last_active":"2024-04-25T12:35:56.759528+0000","last_peered":"2024-04-25T12:35:56.759528+0000","last_clean":"2024-04-25T12:35:44.494980+0000","last_became_active":"2024-04-25T12:35:45.512537+0000","last_became_peered":"2024-04-25T12:35:45.512537+0000","last_unstale":"2024-04-25T12:35:56.759528+0000","last_undegraded":"2024-04-25T12:35:45.501431+0000","last_fullsized":"2024-04-25T12:35:45.501343+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:40.609857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.563946+0000","last_change":"2024-04-25T12:35:45.509839+0000","last_active":"2024-04-25T12:35:55.563946+0000","last_peered":"2024-04-25T12:35:55.563946+0000","last_clean":"2024-04-25T12:34:16.951510+0000","last_became_active":"2024-04-25T12:35:45.509839+0000","last_became_peered":"2024-04-25T12:35:45.509839+0000","last_unstale":"2024-04-25T12:35:55.563946+0000","last_undegraded":"2024-04-25T12:35:55.563946+0000","last_fullsized":"2024-04-25T12:35:45.502679+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:59.337585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,7,6,1],"acting":[2147483647,7,6,1],"avail_no_missing":["7(1)","1(3)","6(2)"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.617702+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:36:00.617702+0000","last_peered":"2024-04-25T12:36:00.617702+0000","last_clean":"2024-04-25T12:36:00.617702+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:36:00.617702+0000","last_undegraded":"2024-04-25T12:36:00.617702+0000","last_fullsized":"2024-04-25T12:36:00.617702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760361+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:35:56.760361+0000","last_peered":"2024-04-25T12:35:56.760361+0000","last_clean":"2024-04-25T12:35:56.760361+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:35:56.760361+0000","last_undegraded":"2024-04-25T12:35:56.760361+0000","last_fullsized":"2024-04-25T12:35:56.760361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":176,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.614831+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:36:01.614831+0000","last_peered":"2024-04-25T12:36:01.614831+0000","last_clean":"2024-04-25T12:36:01.614831+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:36:01.614831+0000","last_undegraded":"2024-04-25T12:36:01.614831+0000","last_fullsized":"2024-04-25T12:36:01.614831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":53,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760569+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:35:56.760569+0000","last_peered":"2024-04-25T12:35:56.760569+0000","last_clean":"2024-04-25T12:35:56.760569+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:35:56.760569+0000","last_undegraded":"2024-04-25T12:35:56.760569+0000","last_fullsized":"2024-04-25T12:35:56.760569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760279+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:35:56.760279+0000","last_peered":"2024-04-25T12:35:56.760279+0000","last_clean":"2024-04-25T12:35:56.760279+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:35:56.760279+0000","last_undegraded":"2024-04-25T12:35:56.760279+0000","last_fullsized":"2024-04-25T12:35:56.760279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.615391+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:36:00.615391+0000","last_peered":"2024-04-25T12:36:00.615391+0000","last_clean":"2024-04-25T12:36:00.615391+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:36:00.615391+0000","last_undegraded":"2024-04-25T12:36:00.615391+0000","last_fullsized":"2024-04-25T12:36:00.615391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":61,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.613518+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:36:01.613518+0000","last_peered":"2024-04-25T12:36:01.613518+0000","last_clean":"2024-04-25T12:36:01.613518+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:36:01.613518+0000","last_undegraded":"2024-04-25T12:36:01.613518+0000","last_fullsized":"2024-04-25T12:36:01.613518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.573865+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:35:55.573865+0000","last_peered":"2024-04-25T12:35:55.573865+0000","last_clean":"2024-04-25T12:35:55.573865+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:35:55.573865+0000","last_undegraded":"2024-04-25T12:35:55.573865+0000","last_fullsized":"2024-04-25T12:35:55.573865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":35,"reported_epoch":45,"state":"active+undersized+degraded","last_fresh":"2024-04-25T12:35:56.757989+0000","last_change":"2024-04-25T12:35:45.513545+0000","last_active":"2024-04-25T12:35:56.757989+0000","last_peered":"2024-04-25T12:35:56.757989+0000","last_clean":"2024-04-25T12:34:17.995537+0000","last_became_active":"2024-04-25T12:35:45.513545+0000","last_became_peered":"2024-04-25T12:35:45.513545+0000","last_unstale":"2024-04-25T12:35:56.757989+0000","last_undegraded":"2024-04-25T12:35:45.506566+0000","last_fullsized":"2024-04-25T12:35:45.506472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:01.447468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.563269+0000","last_change":"2024-04-25T12:35:45.517722+0000","last_active":"2024-04-25T12:35:55.563269+0000","last_peered":"2024-04-25T12:35:55.563269+0000","last_clean":"2024-04-25T12:34:16.951510+0000","last_became_active":"2024-04-25T12:35:45.517722+0000","last_became_peered":"2024-04-25T12:35:45.517722+0000","last_unstale":"2024-04-25T12:35:55.563269+0000","last_undegraded":"2024-04-25T12:35:55.563269+0000","last_fullsized":"2024-04-25T12:35:45.500305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:11.204775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,4,1,3],"acting":[2147483647,4,1,3],"avail_no_missing":["4(1)","1(2)","3(3)"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.574710+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:35:55.574710+0000","last_peered":"2024-04-25T12:35:55.574710+0000","last_clean":"2024-04-25T12:35:55.574710+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:35:55.574710+0000","last_undegraded":"2024-04-25T12:35:55.574710+0000","last_fullsized":"2024-04-25T12:35:55.574710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":76,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.627125+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:36:00.627125+0000","last_peered":"2024-04-25T12:36:00.627125+0000","last_clean":"2024-04-25T12:36:00.627125+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:36:00.627125+0000","last_undegraded":"2024-04-25T12:36:00.627125+0000","last_fullsized":"2024-04-25T12:36:00.627125+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":53,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.758838+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:35:56.758838+0000","last_peered":"2024-04-25T12:35:56.758838+0000","last_clean":"2024-04-25T12:35:56.758838+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:35:56.758838+0000","last_undegraded":"2024-04-25T12:35:56.758838+0000","last_fullsized":"2024-04-25T12:35:56.758838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.616222+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:36:01.616222+0000","last_peered":"2024-04-25T12:36:01.616222+0000","last_clean":"2024-04-25T12:36:01.616222+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:36:01.616222+0000","last_undegraded":"2024-04-25T12:36:01.616222+0000","last_fullsized":"2024-04-25T12:36:01.616222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":64,"reported_epoch":45,"state":"active+undersized+degraded","last_fresh":"2024-04-25T12:35:55.574849+0000","last_change":"2024-04-25T12:35:45.526655+0000","last_active":"2024-04-25T12:35:55.574849+0000","last_peered":"2024-04-25T12:35:55.574849+0000","last_clean":"2024-04-25T12:35:44.474971+0000","last_became_active":"2024-04-25T12:35:45.526655+0000","last_became_peered":"2024-04-25T12:35:45.526655+0000","last_unstale":"2024-04-25T12:35:55.574849+0000","last_undegraded":"2024-04-25T12:35:45.523540+0000","last_fullsized":"2024-04-25T12:35:45.523463+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:49.194620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":72,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.614995+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:36:01.614995+0000","last_peered":"2024-04-25T12:36:01.614995+0000","last_clean":"2024-04-25T12:36:01.614995+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:36:01.614995+0000","last_undegraded":"2024-04-25T12:36:01.614995+0000","last_fullsized":"2024-04-25T12:36:01.614995+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760381+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:35:56.760381+0000","last_peered":"2024-04-25T12:35:56.760381+0000","last_clean":"2024-04-25T12:35:56.760381+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:35:56.760381+0000","last_undegraded":"2024-04-25T12:35:56.760381+0000","last_fullsized":"2024-04-25T12:35:56.760381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.617285+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:36:00.617285+0000","last_peered":"2024-04-25T12:36:00.617285+0000","last_clean":"2024-04-25T12:36:00.617285+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:36:00.617285+0000","last_undegraded":"2024-04-25T12:36:00.617285+0000","last_fullsized":"2024-04-25T12:36:00.617285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.564387+0000","last_change":"2024-04-25T12:35:45.510074+0000","last_active":"2024-04-25T12:35:55.564387+0000","last_peered":"2024-04-25T12:35:55.564387+0000","last_clean":"2024-04-25T12:34:16.951510+0000","last_became_active":"2024-04-25T12:35:45.510074+0000","last_became_peered":"2024-04-25T12:35:45.510074+0000","last_unstale":"2024-04-25T12:35:55.564387+0000","last_undegraded":"2024-04-25T12:35:55.564387+0000","last_fullsized":"2024-04-25T12:35:45.503519+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:26.867106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,7,1,6],"acting":[2147483647,7,1,6],"avail_no_missing":["7(1)","1(2)","6(3)"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.613613+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:36:01.613613+0000","last_peered":"2024-04-25T12:36:01.613613+0000","last_clean":"2024-04-25T12:36:01.613613+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:36:01.613613+0000","last_undegraded":"2024-04-25T12:36:01.613613+0000","last_fullsized":"2024-04-25T12:36:01.613613+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.572494+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:35:55.572494+0000","last_peered":"2024-04-25T12:35:55.572494+0000","last_clean":"2024-04-25T12:35:55.572494+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:35:55.572494+0000","last_undegraded":"2024-04-25T12:35:55.572494+0000","last_fullsized":"2024-04-25T12:35:55.572494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.758045+0000","last_change":"2024-04-25T12:35:45.517772+0000","last_active":"2024-04-25T12:35:56.758045+0000","last_peered":"2024-04-25T12:35:56.758045+0000","last_clean":"2024-04-25T12:34:16.951510+0000","last_became_active":"2024-04-25T12:35:45.517772+0000","last_became_peered":"2024-04-25T12:35:45.517772+0000","last_unstale":"2024-04-25T12:35:56.758045+0000","last_undegraded":"2024-04-25T12:35:56.758045+0000","last_fullsized":"2024-04-25T12:35:45.503049+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:39.911733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,1,6,4],"acting":[2147483647,1,6,4],"avail_no_missing":["1(1)","4(3)","6(2)"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.759990+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:35:56.759990+0000","last_peered":"2024-04-25T12:35:56.759990+0000","last_clean":"2024-04-25T12:35:56.759990+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:35:56.759990+0000","last_undegraded":"2024-04-25T12:35:56.759990+0000","last_fullsized":"2024-04-25T12:35:56.759990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760904+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:35:56.760904+0000","last_peered":"2024-04-25T12:35:56.760904+0000","last_clean":"2024-04-25T12:35:56.760904+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:35:56.760904+0000","last_undegraded":"2024-04-25T12:35:56.760904+0000","last_fullsized":"2024-04-25T12:35:56.760904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":59,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.614857+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:36:00.614857+0000","last_peered":"2024-04-25T12:36:00.614857+0000","last_clean":"2024-04-25T12:36:00.614857+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:36:00.614857+0000","last_undegraded":"2024-04-25T12:36:00.614857+0000","last_fullsized":"2024-04-25T12:36:00.614857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.613652+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:36:00.613652+0000","last_peered":"2024-04-25T12:36:00.613652+0000","last_clean":"2024-04-25T12:36:00.613652+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:36:00.613652+0000","last_undegraded":"2024-04-25T12:36:00.613652+0000","last_fullsized":"2024-04-25T12:36:00.613652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.608847+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:36:00.608847+0000","last_peered":"2024-04-25T12:36:00.608847+0000","last_clean":"2024-04-25T12:36:00.608847+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:36:00.608847+0000","last_undegraded":"2024-04-25T12:36:00.608847+0000","last_fullsized":"2024-04-25T12:36:00.608847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.761466+0000","last_change":"2024-04-25T12:35:45.525649+0000","last_active":"2024-04-25T12:35:56.761466+0000","last_peered":"2024-04-25T12:35:56.761466+0000","last_clean":"2024-04-25T12:35:44.495619+0000","last_became_active":"2024-04-25T12:35:45.525649+0000","last_became_peered":"2024-04-25T12:35:45.525649+0000","last_unstale":"2024-04-25T12:35:56.761466+0000","last_undegraded":"2024-04-25T12:35:56.761466+0000","last_fullsized":"2024-04-25T12:35:45.502743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:49.795884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2147483647,6],"acting":[3,0,2147483647,6],"avail_no_missing":["3(0)","0(1)","6(3)"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":52,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.608163+0000","last_change":"2024-04-25T12:35:00.239744+0000","last_active":"2024-04-25T12:36:00.608163+0000","last_peered":"2024-04-25T12:36:00.608163+0000","last_clean":"2024-04-25T12:36:00.608163+0000","last_became_active":"2024-04-25T12:35:00.239414+0000","last_became_peered":"2024-04-25T12:35:00.239414+0000","last_unstale":"2024-04-25T12:36:00.608163+0000","last_undegraded":"2024-04-25T12:36:00.608163+0000","last_fullsized":"2024-04-25T12:36:00.608163+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:17.015606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"42'17","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.613033+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:36:01.613033+0000","last_peered":"2024-04-25T12:36:01.613033+0000","last_clean":"2024-04-25T12:36:01.613033+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:36:01.613033+0000","last_undegraded":"2024-04-25T12:36:01.613033+0000","last_fullsized":"2024-04-25T12:36:01.613033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7907,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.635972+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:36:00.635972+0000","last_peered":"2024-04-25T12:36:00.635972+0000","last_clean":"2024-04-25T12:36:00.635972+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:36:00.635972+0000","last_undegraded":"2024-04-25T12:36:00.635972+0000","last_fullsized":"2024-04-25T12:36:00.635972+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.759066+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:35:56.759066+0000","last_peered":"2024-04-25T12:35:56.759066+0000","last_clean":"2024-04-25T12:35:56.759066+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:35:56.759066+0000","last_undegraded":"2024-04-25T12:35:56.759066+0000","last_fullsized":"2024-04-25T12:35:56.759066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":71,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.608725+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:36:00.608725+0000","last_peered":"2024-04-25T12:36:00.608725+0000","last_clean":"2024-04-25T12:36:00.608725+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:36:00.608725+0000","last_undegraded":"2024-04-25T12:36:00.608725+0000","last_fullsized":"2024-04-25T12:36:00.608725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":61,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.614939+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:36:01.614939+0000","last_peered":"2024-04-25T12:36:01.614939+0000","last_clean":"2024-04-25T12:36:01.614939+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:36:01.614939+0000","last_undegraded":"2024-04-25T12:36:01.614939+0000","last_fullsized":"2024-04-25T12:36:01.614939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.760939+0000","last_change":"2024-04-25T12:35:45.514581+0000","last_active":"2024-04-25T12:35:56.760939+0000","last_peered":"2024-04-25T12:35:56.760939+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.514581+0000","last_became_peered":"2024-04-25T12:35:45.514581+0000","last_unstale":"2024-04-25T12:35:56.760939+0000","last_undegraded":"2024-04-25T12:35:56.760939+0000","last_fullsized":"2024-04-25T12:35:45.510308+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:59.920222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.574622+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:35:55.574622+0000","last_peered":"2024-04-25T12:35:55.574622+0000","last_clean":"2024-04-25T12:35:55.574622+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:35:55.574622+0000","last_undegraded":"2024-04-25T12:35:55.574622+0000","last_fullsized":"2024-04-25T12:35:55.574622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":60,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.607724+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:36:00.607724+0000","last_peered":"2024-04-25T12:36:00.607724+0000","last_clean":"2024-04-25T12:36:00.607724+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:36:00.607724+0000","last_undegraded":"2024-04-25T12:36:00.607724+0000","last_fullsized":"2024-04-25T12:36:00.607724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.616109+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:36:01.616109+0000","last_peered":"2024-04-25T12:36:01.616109+0000","last_clean":"2024-04-25T12:36:01.616109+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:36:01.616109+0000","last_undegraded":"2024-04-25T12:36:01.616109+0000","last_fullsized":"2024-04-25T12:36:01.616109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":72,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.618503+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:36:01.618503+0000","last_peered":"2024-04-25T12:36:01.618503+0000","last_clean":"2024-04-25T12:36:01.618503+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:36:01.618503+0000","last_undegraded":"2024-04-25T12:36:01.618503+0000","last_fullsized":"2024-04-25T12:36:01.618503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":59,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.618513+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:36:00.618513+0000","last_peered":"2024-04-25T12:36:00.618513+0000","last_clean":"2024-04-25T12:36:00.618513+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:36:00.618513+0000","last_undegraded":"2024-04-25T12:36:00.618513+0000","last_fullsized":"2024-04-25T12:36:00.618513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.642290+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:36:00.642290+0000","last_peered":"2024-04-25T12:36:00.642290+0000","last_clean":"2024-04-25T12:36:00.642290+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:36:00.642290+0000","last_undegraded":"2024-04-25T12:36:00.642290+0000","last_fullsized":"2024-04-25T12:36:00.642290+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.618636+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:36:00.618636+0000","last_peered":"2024-04-25T12:36:00.618636+0000","last_clean":"2024-04-25T12:36:00.618636+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:36:00.618636+0000","last_undegraded":"2024-04-25T12:36:00.618636+0000","last_fullsized":"2024-04-25T12:36:00.618636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.759399+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:35:56.759399+0000","last_peered":"2024-04-25T12:35:56.759399+0000","last_clean":"2024-04-25T12:35:56.759399+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:35:56.759399+0000","last_undegraded":"2024-04-25T12:35:56.759399+0000","last_fullsized":"2024-04-25T12:35:56.759399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.760439+0000","last_change":"2024-04-25T12:35:45.521160+0000","last_active":"2024-04-25T12:35:56.760439+0000","last_peered":"2024-04-25T12:35:56.760439+0000","last_clean":"2024-04-25T12:35:44.495302+0000","last_became_active":"2024-04-25T12:35:45.521160+0000","last_became_peered":"2024-04-25T12:35:45.521160+0000","last_unstale":"2024-04-25T12:35:56.760439+0000","last_undegraded":"2024-04-25T12:35:56.760439+0000","last_fullsized":"2024-04-25T12:35:45.504232+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:37.888310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,2147483647,1],"acting":[3,2,2147483647,1],"avail_no_missing":["3(0)","1(3)","2(1)"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.756191+0000","last_change":"2024-04-25T12:35:45.503369+0000","last_active":"2024-04-25T12:35:56.756191+0000","last_peered":"2024-04-25T12:35:56.756191+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.503369+0000","last_became_peered":"2024-04-25T12:35:45.503369+0000","last_unstale":"2024-04-25T12:35:56.756191+0000","last_undegraded":"2024-04-25T12:35:56.756191+0000","last_fullsized":"2024-04-25T12:35:45.500516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:48.667371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.613155+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:36:00.613155+0000","last_peered":"2024-04-25T12:36:00.613155+0000","last_clean":"2024-04-25T12:36:00.613155+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:36:00.613155+0000","last_undegraded":"2024-04-25T12:36:00.613155+0000","last_fullsized":"2024-04-25T12:36:00.613155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760702+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:35:56.760702+0000","last_peered":"2024-04-25T12:35:56.760702+0000","last_clean":"2024-04-25T12:35:56.760702+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:35:56.760702+0000","last_undegraded":"2024-04-25T12:35:56.760702+0000","last_fullsized":"2024-04-25T12:35:56.760702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.618056+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:36:00.618056+0000","last_peered":"2024-04-25T12:36:00.618056+0000","last_clean":"2024-04-25T12:36:00.618056+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:36:00.618056+0000","last_undegraded":"2024-04-25T12:36:00.618056+0000","last_fullsized":"2024-04-25T12:36:00.618056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.624696+0000","last_change":"2024-04-25T12:36:00.624696+0000","last_active":"2024-04-25T12:36:00.624696+0000","last_peered":"2024-04-25T12:36:00.624696+0000","last_clean":"2024-04-25T12:36:00.624696+0000","last_became_active":"2024-04-25T12:36:00.624209+0000","last_became_peered":"2024-04-25T12:36:00.624209+0000","last_unstale":"2024-04-25T12:36:00.624696+0000","last_undegraded":"2024-04-25T12:36:00.624696+0000","last_fullsized":"2024-04-25T12:36:00.624696+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:14.608733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":41,"reported_epoch":45,"state":"active+undersized+degraded","last_fresh":"2024-04-25T12:35:56.756606+0000","last_change":"2024-04-25T12:35:45.506253+0000","last_active":"2024-04-25T12:35:56.756606+0000","last_peered":"2024-04-25T12:35:56.756606+0000","last_clean":"2024-04-25T12:34:50.121039+0000","last_became_active":"2024-04-25T12:35:45.506253+0000","last_became_peered":"2024-04-25T12:35:45.506253+0000","last_unstale":"2024-04-25T12:35:56.756606+0000","last_undegraded":"2024-04-25T12:35:45.501968+0000","last_fullsized":"2024-04-25T12:35:45.501744+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:00.686057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":2}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.616399+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:36:01.616399+0000","last_peered":"2024-04-25T12:36:01.616399+0000","last_clean":"2024-04-25T12:36:01.616399+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:36:01.616399+0000","last_undegraded":"2024-04-25T12:36:01.616399+0000","last_fullsized":"2024-04-25T12:36:01.616399+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":71,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.642554+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:36:00.642554+0000","last_peered":"2024-04-25T12:36:00.642554+0000","last_clean":"2024-04-25T12:36:00.642554+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:36:00.642554+0000","last_undegraded":"2024-04-25T12:36:00.642554+0000","last_fullsized":"2024-04-25T12:36:00.642554+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.612720+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:36:01.612720+0000","last_peered":"2024-04-25T12:36:01.612720+0000","last_clean":"2024-04-25T12:36:01.612720+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:36:01.612720+0000","last_undegraded":"2024-04-25T12:36:01.612720+0000","last_fullsized":"2024-04-25T12:36:01.612720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.607875+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:36:00.607875+0000","last_peered":"2024-04-25T12:36:00.607875+0000","last_clean":"2024-04-25T12:36:00.607875+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:36:00.607875+0000","last_undegraded":"2024-04-25T12:36:00.607875+0000","last_fullsized":"2024-04-25T12:36:00.607875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":59,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.759394+0000","last_change":"2024-04-25T12:35:45.511684+0000","last_active":"2024-04-25T12:35:56.759394+0000","last_peered":"2024-04-25T12:35:56.759394+0000","last_clean":"2024-04-25T12:35:44.503060+0000","last_became_active":"2024-04-25T12:35:45.511684+0000","last_became_peered":"2024-04-25T12:35:45.511684+0000","last_unstale":"2024-04-25T12:35:56.759394+0000","last_undegraded":"2024-04-25T12:35:56.759394+0000","last_fullsized":"2024-04-25T12:35:45.500994+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:05.998848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.564383+0000","last_change":"2024-04-25T12:35:45.506953+0000","last_active":"2024-04-25T12:35:55.564383+0000","last_peered":"2024-04-25T12:35:55.564383+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.506953+0000","last_became_peered":"2024-04-25T12:35:45.506953+0000","last_unstale":"2024-04-25T12:35:55.564383+0000","last_undegraded":"2024-04-25T12:35:55.564383+0000","last_fullsized":"2024-04-25T12:35:45.501010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:35.837323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.615108+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:36:00.615108+0000","last_peered":"2024-04-25T12:36:00.615108+0000","last_clean":"2024-04-25T12:36:00.615108+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:36:00.615108+0000","last_undegraded":"2024-04-25T12:36:00.615108+0000","last_fullsized":"2024-04-25T12:36:00.615108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.614348+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:36:00.614348+0000","last_peered":"2024-04-25T12:36:00.614348+0000","last_clean":"2024-04-25T12:36:00.614348+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:36:00.614348+0000","last_undegraded":"2024-04-25T12:36:00.614348+0000","last_fullsized":"2024-04-25T12:36:00.614348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.616034+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:36:01.616034+0000","last_peered":"2024-04-25T12:36:01.616034+0000","last_clean":"2024-04-25T12:36:01.616034+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:36:01.616034+0000","last_undegraded":"2024-04-25T12:36:01.616034+0000","last_fullsized":"2024-04-25T12:36:01.616034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.758323+0000","last_change":"2024-04-25T12:35:45.513996+0000","last_active":"2024-04-25T12:35:56.758323+0000","last_peered":"2024-04-25T12:35:56.758323+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.513996+0000","last_became_peered":"2024-04-25T12:35:45.513996+0000","last_unstale":"2024-04-25T12:35:56.758323+0000","last_undegraded":"2024-04-25T12:35:56.758323+0000","last_fullsized":"2024-04-25T12:35:45.507419+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:56.832719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":59,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.759192+0000","last_change":"2024-04-25T12:35:45.511462+0000","last_active":"2024-04-25T12:35:56.759192+0000","last_peered":"2024-04-25T12:35:56.759192+0000","last_clean":"2024-04-25T12:35:44.492150+0000","last_became_active":"2024-04-25T12:35:45.511462+0000","last_became_peered":"2024-04-25T12:35:45.511462+0000","last_unstale":"2024-04-25T12:35:56.759192+0000","last_undegraded":"2024-04-25T12:35:56.759192+0000","last_fullsized":"2024-04-25T12:35:45.501352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:00.650033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.685825+0000","last_change":"2024-04-25T12:35:45.506803+0000","last_active":"2024-04-25T12:35:56.685825+0000","last_peered":"2024-04-25T12:35:56.685825+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.506803+0000","last_became_peered":"2024-04-25T12:35:45.506803+0000","last_unstale":"2024-04-25T12:35:56.685825+0000","last_undegraded":"2024-04-25T12:35:56.685825+0000","last_fullsized":"2024-04-25T12:35:45.497205+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:15.572525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":67,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.616247+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:36:01.616247+0000","last_peered":"2024-04-25T12:36:01.616247+0000","last_clean":"2024-04-25T12:36:01.616247+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:36:01.616247+0000","last_undegraded":"2024-04-25T12:36:01.616247+0000","last_fullsized":"2024-04-25T12:36:01.616247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.618424+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:36:00.618424+0000","last_peered":"2024-04-25T12:36:00.618424+0000","last_clean":"2024-04-25T12:36:00.618424+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:36:00.618424+0000","last_undegraded":"2024-04-25T12:36:00.618424+0000","last_fullsized":"2024-04-25T12:36:00.618424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.573778+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:35:55.573778+0000","last_peered":"2024-04-25T12:35:55.573778+0000","last_clean":"2024-04-25T12:35:55.573778+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:35:55.573778+0000","last_undegraded":"2024-04-25T12:35:55.573778+0000","last_fullsized":"2024-04-25T12:35:55.573778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.760890+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:35:56.760890+0000","last_peered":"2024-04-25T12:35:56.760890+0000","last_clean":"2024-04-25T12:35:56.760890+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:35:56.760890+0000","last_undegraded":"2024-04-25T12:35:56.760890+0000","last_fullsized":"2024-04-25T12:35:56.760890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.609277+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:36:00.609277+0000","last_peered":"2024-04-25T12:36:00.609277+0000","last_clean":"2024-04-25T12:36:00.609277+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:36:00.609277+0000","last_undegraded":"2024-04-25T12:36:00.609277+0000","last_fullsized":"2024-04-25T12:36:00.609277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.761318+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:35:56.761318+0000","last_peered":"2024-04-25T12:35:56.761318+0000","last_clean":"2024-04-25T12:35:56.761318+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:35:56.761318+0000","last_undegraded":"2024-04-25T12:35:56.761318+0000","last_fullsized":"2024-04-25T12:35:56.761318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:56.759195+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:35:56.759195+0000","last_peered":"2024-04-25T12:35:56.759195+0000","last_clean":"2024-04-25T12:35:56.759195+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:35:56.759195+0000","last_undegraded":"2024-04-25T12:35:56.759195+0000","last_fullsized":"2024-04-25T12:35:56.759195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.572251+0000","last_change":"2024-04-25T12:35:45.526973+0000","last_active":"2024-04-25T12:35:55.572251+0000","last_peered":"2024-04-25T12:35:55.572251+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.526973+0000","last_became_peered":"2024-04-25T12:35:45.526973+0000","last_unstale":"2024-04-25T12:35:55.572251+0000","last_undegraded":"2024-04-25T12:35:55.572251+0000","last_fullsized":"2024-04-25T12:35:45.519487+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:18.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.613632+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:36:00.613632+0000","last_peered":"2024-04-25T12:36:00.613632+0000","last_clean":"2024-04-25T12:36:00.613632+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:36:00.613632+0000","last_undegraded":"2024-04-25T12:36:00.613632+0000","last_fullsized":"2024-04-25T12:36:00.613632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T12:36:00.613831+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:36:00.613831+0000","last_peered":"2024-04-25T12:36:00.613831+0000","last_clean":"2024-04-25T12:36:00.613831+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:36:00.613831+0000","last_undegraded":"2024-04-25T12:36:00.613831+0000","last_fullsized":"2024-04-25T12:36:00.613831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.573690+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:35:55.573690+0000","last_peered":"2024-04-25T12:35:55.573690+0000","last_clean":"2024-04-25T12:35:55.573690+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:35:55.573690+0000","last_undegraded":"2024-04-25T12:35:55.573690+0000","last_fullsized":"2024-04-25T12:35:55.573690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.574306+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:35:55.574306+0000","last_peered":"2024-04-25T12:35:55.574306+0000","last_clean":"2024-04-25T12:35:55.574306+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:35:55.574306+0000","last_undegraded":"2024-04-25T12:35:55.574306+0000","last_fullsized":"2024-04-25T12:35:55.574306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.574431+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:35:55.574431+0000","last_peered":"2024-04-25T12:35:55.574431+0000","last_clean":"2024-04-25T12:35:55.574431+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:35:55.574431+0000","last_undegraded":"2024-04-25T12:35:55.574431+0000","last_fullsized":"2024-04-25T12:35:55.574431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":67,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.618383+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:36:01.618383+0000","last_peered":"2024-04-25T12:36:01.618383+0000","last_clean":"2024-04-25T12:36:01.618383+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:36:01.618383+0000","last_undegraded":"2024-04-25T12:36:01.618383+0000","last_fullsized":"2024-04-25T12:36:01.618383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.572089+0000","last_change":"2024-04-25T12:35:45.526579+0000","last_active":"2024-04-25T12:35:55.572089+0000","last_peered":"2024-04-25T12:35:55.572089+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.526579+0000","last_became_peered":"2024-04-25T12:35:45.526579+0000","last_unstale":"2024-04-25T12:35:55.572089+0000","last_undegraded":"2024-04-25T12:35:55.572089+0000","last_fullsized":"2024-04-25T12:35:45.516202+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:10.452577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.572473+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:35:55.572473+0000","last_peered":"2024-04-25T12:35:55.572473+0000","last_clean":"2024-04-25T12:35:55.572473+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:35:55.572473+0000","last_undegraded":"2024-04-25T12:35:55.572473+0000","last_fullsized":"2024-04-25T12:35:55.572473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.613895+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:36:01.613895+0000","last_peered":"2024-04-25T12:36:01.613895+0000","last_clean":"2024-04-25T12:36:01.613895+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:36:01.613895+0000","last_undegraded":"2024-04-25T12:36:01.613895+0000","last_fullsized":"2024-04-25T12:36:01.613895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.573034+0000","last_change":"2024-04-25T12:35:45.527597+0000","last_active":"2024-04-25T12:35:55.573034+0000","last_peered":"2024-04-25T12:35:55.573034+0000","last_clean":"2024-04-25T12:35:44.473520+0000","last_became_active":"2024-04-25T12:35:45.527597+0000","last_became_peered":"2024-04-25T12:35:45.527597+0000","last_unstale":"2024-04-25T12:35:55.573034+0000","last_undegraded":"2024-04-25T12:35:55.573034+0000","last_fullsized":"2024-04-25T12:35:45.520281+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:32.438484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T12:36:01.612827+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:36:01.612827+0000","last_peered":"2024-04-25T12:36:01.612827+0000","last_clean":"2024-04-25T12:36:01.612827+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:36:01.612827+0000","last_undegraded":"2024-04-25T12:36:01.612827+0000","last_fullsized":"2024-04-25T12:36:01.612827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:55.563751+0000","last_change":"2024-04-25T12:35:45.506346+0000","last_active":"2024-04-25T12:35:55.563751+0000","last_peered":"2024-04-25T12:35:55.563751+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.506346+0000","last_became_peered":"2024-04-25T12:35:45.506346+0000","last_unstale":"2024-04-25T12:35:55.563751+0000","last_undegraded":"2024-04-25T12:35:55.563751+0000","last_fullsized":"2024-04-25T12:35:45.501936+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:11.844062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":45,"state":"active+undersized","last_fresh":"2024-04-25T12:35:56.756908+0000","last_change":"2024-04-25T12:35:45.503394+0000","last_active":"2024-04-25T12:35:56.756908+0000","last_peered":"2024-04-25T12:35:56.756908+0000","last_clean":"2024-04-25T12:34:12.877100+0000","last_became_active":"2024-04-25T12:35:45.503394+0000","last_became_peered":"2024-04-25T12:35:45.503394+0000","last_unstale":"2024-04-25T12:35:56.756908+0000","last_undegraded":"2024-04-25T12:35:56.756908+0000","last_fullsized":"2024-04-25T12:35:45.500766+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":58,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T12:35:55.574607+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:35:55.574607+0000","last_peered":"2024-04-25T12:35:55.574607+0000","last_clean":"2024-04-25T12:35:55.574607+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:35:55.574607+0000","last_undegraded":"2024-04-25T12:35:55.574607+0000","last_fullsized":"2024-04-25T12:35:55.574607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":113,"acting":113,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11517,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":23034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":58,"acting":58,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":241559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":241649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":264062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":264358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":0,"seq":1,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28672,"kb_used_data":440,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94343168,"statfs":{"total":96636764160,"available":96607404032,"internally_reserved":0,"allocated":450560,"data_stored":199227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":862997,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":271095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":860501,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:36:04.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 2.d 2024-04-25T12:36:04.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.622+0000 7fb213e00640 1 -- 172.21.15.123:0/4294345588 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v50) ==== 97+0+0 (secure 0 0 0) 0x7fb20c07ff10 con 0x7fb21c0ccf10 2024-04-25T12:36:04.621 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:36:04.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.626+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb1f80744e0 msgr2=0x7fb1f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.626+0000 7fb221800640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb1f80744e0 0x7fb1f80769f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb204009560 tx=0x7fb204009190 comp rx=0 tx=0).stop 2024-04-25T12:36:04.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.626+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb21c0ccf10 msgr2=0x7fb21c1e5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.626+0000 7fb221800640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb21c0ccf10 0x7fb21c1e5190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb20c02a2b0 tx=0x7fb20c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 shutdown_connections 2024-04-25T12:36:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb1f80744e0 0x7fb1f80769f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb21c15c060 0x7fb21c157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb21c0d1d80 0x7fb21c1e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 --2- 172.21.15.123:0/4294345588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb21c0ccf10 0x7fb21c1e5190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 >> 172.21.15.123:0/4294345588 conn(0x7fb21c08da20 msgr2=0x7fb21c0cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 shutdown_connections 2024-04-25T12:36:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.629+0000 7fb221800640 1 -- 172.21.15.123:0/4294345588 wait complete. 2024-04-25T12:36:04.643 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:04.644 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:04.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:04.673 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:04.674+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:04.771 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:04.773+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:04.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.871+0000 7fc327000640 1 Processor -- start 2024-04-25T12:36:04.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.871+0000 7fc327000640 1 -- start start 2024-04-25T12:36:04.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc327000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc320152c70 0x7fc320074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc327000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc3200c8170 0x7fc32015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc327000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3200c8580 con 0x7fc3200c2790 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc327000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc320158bd0 con 0x7fc320152c70 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc327000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc320158eb0 con 0x7fc3200c8170 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc31f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc3200c2790 0x7fc3200c3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:37024/0 (socket says 172.21.15.123:37024) 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc31f400640 1 -- 172.21.15.123:0/3911526458 learned_addr learned my addr 172.21.15.123:0/3911526458 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc31fe00640 1 --2- 172.21.15.123:0/3911526458 >> v2:172.21.15.123:3300/0 conn(0x7fc320152c70 0x7fc320074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.873+0000 7fc325000640 1 --2- 172.21.15.123:0/3911526458 >> v2:172.21.15.153:3300/0 conn(0x7fc3200c8170 0x7fc32015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.873 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:04.873+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:04.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.874+0000 7fc31fe00640 1 -- 172.21.15.123:0/3911526458 >> v2:172.21.15.153:3300/0 conn(0x7fc3200c8170 msgr2=0x7fc32015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:04.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.874+0000 7fc31fe00640 1 --2- 172.21.15.123:0/3911526458 >> v2:172.21.15.153:3300/0 conn(0x7fc3200c8170 0x7fc32015c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.874+0000 7fc31fe00640 1 -- 172.21.15.123:0/3911526458 >> v1:172.21.15.123:6789/0 conn(0x7fc3200c2790 legacy=0x7fc3200c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.874+0000 7fc31fe00640 1 -- 172.21.15.123:0/3911526458 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc320159190 con 0x7fc320152c70 2024-04-25T12:36:04.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.875+0000 7fc31fe00640 1 --2- 172.21.15.123:0/3911526458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc320152c70 0x7fc320074820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc3200c4b20 tx=0x7fc31002b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ad2ef401172bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:04.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.875+0000 7fc31ea00640 1 -- 172.21.15.123:0/3911526458 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000b040 con 0x7fc320152c70 2024-04-25T12:36:04.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.875+0000 7fc31ea00640 1 -- 172.21.15.123:0/3911526458 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc310030710 con 0x7fc320152c70 2024-04-25T12:36:04.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.875+0000 7fc31ea00640 1 -- 172.21.15.123:0/3911526458 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310030ae0 con 0x7fc320152c70 2024-04-25T12:36:04.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 -- 172.21.15.123:0/3911526458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc320152c70 msgr2=0x7fc320074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 --2- 172.21.15.123:0/3911526458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc320152c70 0x7fc320074820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc3200c4b20 tx=0x7fc31002b4d0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 -- 172.21.15.123:0/3911526458 shutdown_connections 2024-04-25T12:36:04.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 --2- 172.21.15.123:0/3911526458 >> v2:172.21.15.153:3300/0 conn(0x7fc3200c8170 0x7fc32015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 --2- 172.21.15.123:0/3911526458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc320152c70 0x7fc320074820 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 -- 172.21.15.123:0/3911526458 >> 172.21.15.123:0/3911526458 conn(0x7fc32006f440 msgr2=0x7fc32006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:04.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 -- 172.21.15.123:0/3911526458 shutdown_connections 2024-04-25T12:36:04.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.876+0000 7fc327000640 1 -- 172.21.15.123:0/3911526458 wait complete. 2024-04-25T12:36:04.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.880+0000 7fc327000640 1 Processor -- start 2024-04-25T12:36:04.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.880+0000 7fc327000640 1 -- start start 2024-04-25T12:36:04.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.882+0000 7fc327000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3200c2790 0x7fc320156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.882+0000 7fc327000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3200c8170 0x7fc320153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.882+0000 7fc327000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320152c70 0x7fc3201480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.882+0000 7fc327000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3200c46f0 con 0x7fc3200c8170 2024-04-25T12:36:04.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.882+0000 7fc327000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc320159750 con 0x7fc320152c70 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.882+0000 7fc327000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc3200c4870 con 0x7fc3200c2790 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3200c8170 0x7fc320153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3200c2790 0x7fc320156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3200c2790 0x7fc320156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:51456/0 (socket says 172.21.15.123:51456) 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 -- 172.21.15.123:0/3278444159 learned_addr learned my addr 172.21.15.123:0/3278444159 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc325000640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320152c70 0x7fc3201480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 -- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320152c70 msgr2=0x7fc3201480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320152c70 0x7fc3201480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 -- 172.21.15.123:0/3278444159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3200c8170 msgr2=0x7fc320153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3200c8170 0x7fc320153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:04.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.883+0000 7fc31fe00640 1 -- 172.21.15.123:0/3278444159 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc320157040 con 0x7fc3200c2790 2024-04-25T12:36:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.884+0000 7fc31fe00640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3200c2790 0x7fc320156b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc310000c00 tx=0x7fc310004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.884+0000 7fc317e00640 1 -- 172.21.15.123:0/3278444159 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31003f020 con 0x7fc3200c2790 2024-04-25T12:36:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.884+0000 7fc317e00640 1 -- 172.21.15.123:0/3278444159 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100329c0 con 0x7fc3200c2790 2024-04-25T12:36:04.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.884+0000 7fc317e00640 1 -- 172.21.15.123:0/3278444159 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310032d70 con 0x7fc3200c2790 2024-04-25T12:36:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.884+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3201e3660 con 0x7fc3200c2790 2024-04-25T12:36:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.884+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3201f1870 con 0x7fc3200c2790 2024-04-25T12:36:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.885+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc320076700 con 0x7fc3200c2790 2024-04-25T12:36:04.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.886+0000 7fc317e00640 1 -- 172.21.15.123:0/3278444159 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc310042020 con 0x7fc3200c2790 2024-04-25T12:36:04.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.887+0000 7fc317e00640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc3000744b0 0x7fc3000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.887+0000 7fc317e00640 1 -- 172.21.15.123:0/3278444159 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6619+0+0 (secure 0 0 0) 0x7fc3100b5eb0 con 0x7fc3200c2790 2024-04-25T12:36:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.888+0000 7fc31f400640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc3000744b0 0x7fc3000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.888+0000 7fc31f400640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc3000744b0 0x7fc3000769c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc320158320 tx=0x7fc30c007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:04.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:04.892+0000 7fc317e00640 1 -- 172.21.15.123:0/3278444159 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc310049030 con 0x7fc3200c2790 2024-04-25T12:36:04.975 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:04.976+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:05.003 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:05.003 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:05.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:05.070 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:05.074 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:05.075+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:05.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.131+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg force-recovery", "pgid": ["2.d"], "target": ["mon-mgr", ""]}) -- 0x7fc3201524b0 con 0x7fc3000744b0 2024-04-25T12:36:05.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.131+0000 7fc317e00640 1 -- 172.21.15.123:0/3278444159 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 pg 2.d doesn't require recovery; 2024-04-25T12:36:05.130 INFO:teuthology.orchestra.run.smithi123.stderr:) ==== 42+0+0 (secure 0 0 0) 0x7fc3201524b0 con 0x7fc3000744b0 2024-04-25T12:36:05.131 INFO:teuthology.orchestra.run.smithi123.stderr:pg 2.d doesn't require recovery; 2024-04-25T12:36:05.131 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T12:36:05.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc3000744b0 msgr2=0x7fc3000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc3000744b0 0x7fc3000769c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc320158320 tx=0x7fc30c007e80 comp rx=0 tx=0).stop 2024-04-25T12:36:05.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3200c2790 msgr2=0x7fc320156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3200c2790 0x7fc320156b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc310000c00 tx=0x7fc310004730 comp rx=0 tx=0).stop 2024-04-25T12:36:05.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 shutdown_connections 2024-04-25T12:36:05.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc3000744b0 0x7fc3000769c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc320152c70 0x7fc3201480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3200c8170 0x7fc320153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 --2- 172.21.15.123:0/3278444159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3200c2790 0x7fc320156b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 >> 172.21.15.123:0/3278444159 conn(0x7fc32006f440 msgr2=0x7fc320150330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:05.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 shutdown_connections 2024-04-25T12:36:05.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.134+0000 7fc327000640 1 -- 172.21.15.123:0/3278444159 wait complete. 2024-04-25T12:36:05.174 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:05.175+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:05.275 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:05.276+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:05.342 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:05.343 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:05.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.362+0000 7fa210600640 1 Processor -- start 2024-04-25T12:36:05.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.362+0000 7fa210600640 1 -- start start 2024-04-25T12:36:05.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:05.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.365+0000 7fa210600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2080c67a0 0x7fa20815c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.365+0000 7fa210600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2080cb0d0 con 0x7fa208156a20 2024-04-25T12:36:05.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.365+0000 7fa210600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2080cb3b0 con 0x7fa208160860 2024-04-25T12:36:05.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.365+0000 7fa210600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa2081603d0 con 0x7fa2080c67a0 2024-04-25T12:36:05.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.366+0000 7fa20dc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2080c67a0 0x7fa20815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.366+0000 7fa20d200640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa208160860 0x7fa2080cfba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:52546/0 (socket says 172.21.15.123:52546) 2024-04-25T12:36:05.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.366+0000 7fa20d200640 1 -- 172.21.15.123:0/1398663803 learned_addr learned my addr 172.21.15.123:0/1398663803 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:05.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.366+0000 7fa20dc00640 1 -- 172.21.15.123:0/1398663803 >> v1:172.21.15.153:6789/0 conn(0x7fa208160860 legacy=0x7fa2080cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.366+0000 7fa20dc00640 1 -- 172.21.15.123:0/1398663803 >> v1:172.21.15.123:6789/0 conn(0x7fa208156a20 legacy=0x7fa208167110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.366+0000 7fa20dc00640 1 -- 172.21.15.123:0/1398663803 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa208160060 con 0x7fa2080c67a0 2024-04-25T12:36:05.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.367+0000 7fa20dc00640 1 --2- 172.21.15.123:0/1398663803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2080c67a0 0x7fa20815c790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0097a0 tx=0x7fa1fc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e91dcd8022c9a841 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.368+0000 7fa207e00640 1 -- 172.21.15.123:0/1398663803 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc037070 con 0x7fa2080c67a0 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.368+0000 7fa207e00640 1 -- 172.21.15.123:0/1398663803 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1fc032440 con 0x7fa2080c67a0 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.368+0000 7fa207e00640 1 -- 172.21.15.123:0/1398663803 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc032810 con 0x7fa2080c67a0 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.369+0000 7fa210600640 1 -- 172.21.15.123:0/1398663803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2080c67a0 msgr2=0x7fa20815c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.369+0000 7fa210600640 1 --2- 172.21.15.123:0/1398663803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2080c67a0 0x7fa20815c790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0097a0 tx=0x7fa1fc02a230 comp rx=0 tx=0).stop 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.369+0000 7fa210600640 1 -- 172.21.15.123:0/1398663803 shutdown_connections 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.369+0000 7fa210600640 1 --2- 172.21.15.123:0/1398663803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2080c67a0 0x7fa20815c790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.369+0000 7fa210600640 1 -- 172.21.15.123:0/1398663803 >> 172.21.15.123:0/1398663803 conn(0x7fa208068600 msgr2=0x7fa208068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.369+0000 7fa210600640 1 -- 172.21.15.123:0/1398663803 shutdown_connections 2024-04-25T12:36:05.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.369+0000 7fa210600640 1 -- 172.21.15.123:0/1398663803 wait complete. 2024-04-25T12:36:05.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.373+0000 7fa210600640 1 Processor -- start 2024-04-25T12:36:05.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.373+0000 7fa210600640 1 -- start start 2024-04-25T12:36:05.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa210600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2080c67a0 0x7fa2081f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa210600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa208156a20 0x7fa208155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa210600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa208160860 0x7fa20815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa210600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa20815fd50 con 0x7fa2080c67a0 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa210600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa208160c70 con 0x7fa208160860 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa210600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa208156e30 con 0x7fa208156a20 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa20dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2080c67a0 0x7fa2081f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.375+0000 7fa20e600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa208160860 0x7fa20815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20d200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa208156a20 0x7fa208155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2080c67a0 0x7fa2081f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53780/0 (socket says 172.21.15.123:53780) 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20dc00640 1 -- 172.21.15.123:0/3715831486 learned_addr learned my addr 172.21.15.123:0/3715831486 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:05.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa208156a20 msgr2=0x7fa208155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20e600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa208156a20 0x7fa208155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2080c67a0 msgr2=0x7fa2081f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20e600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2080c67a0 0x7fa2081f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20815d950 con 0x7fa208160860 2024-04-25T12:36:05.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2080c67a0 0x7fa2081f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:05.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.376+0000 7fa20e600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa208160860 0x7fa20815b180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c730 tx=0x7fa20000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.377+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200003b70 con 0x7fa208160860 2024-04-25T12:36:05.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.377+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa200003d10 con 0x7fa208160860 2024-04-25T12:36:05.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.377+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa208155690 con 0x7fa208160860 2024-04-25T12:36:05.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.377+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200005310 con 0x7fa208160860 2024-04-25T12:36:05.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.377+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa208154390 con 0x7fa208160860 2024-04-25T12:36:05.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.378+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa20815db20 con 0x7fa208160860 2024-04-25T12:36:05.379 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:05.379+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:05.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.380+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa2000056c0 con 0x7fa208160860 2024-04-25T12:36:05.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.381+0000 7fa205600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1e80744b0 0x7fa1e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.381+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6619+0+0 (secure 0 0 0) 0x7fa200095410 con 0x7fa208160860 2024-04-25T12:36:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.381+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1e80744b0 0x7fa1e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.381+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v50) ==== 54+0+16 (secure 0 0 0) 0x7fa2000979b0 con 0x7fa208160860 2024-04-25T12:36:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.382+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa208154590 0x7fa2081549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.382+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2081e9520 con 0x7fa208154590 2024-04-25T12:36:05.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.382+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1e80744b0 0x7fa1e80769c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc000c00 tx=0x7fa1fc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.382+0000 7fa20d200640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa208154590 0x7fa2081549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.383+0000 7fa20d200640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa208154590 0x7fa2081549a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.384+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2081e9520 con 0x7fa208154590 2024-04-25T12:36:05.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.400+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa20815adc0 con 0x7fa208154590 2024-04-25T12:36:05.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.403+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa1f4001970 con 0x7fa208154590 2024-04-25T12:36:05.402 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:05.403 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:05.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.404+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa2080b1f80 0x7fa2081f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.404+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa2080b2390 con 0x7fa2080b1f80 2024-04-25T12:36:05.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.404+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa2080b1f80 0x7fa2081f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.405+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa2080b1f80 0x7fa2081f64d0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.406+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2080b2390 con 0x7fa2080b1f80 2024-04-25T12:36:05.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.422+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa2080cf6d0 con 0x7fa2080b1f80 2024-04-25T12:36:05.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.425+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa2080cf6d0 con 0x7fa2080b1f80 2024-04-25T12:36:05.425 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:05.425 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:05.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.426+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa2081f6f20 0x7fa208062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.426+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa2081f7330 con 0x7fa2081f6f20 2024-04-25T12:36:05.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.426+0000 7fa20d200640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa2081f6f20 0x7fa208062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.427+0000 7fa20d200640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa2081f6f20 0x7fa208062fe0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.428+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2081f7330 con 0x7fa2081f6f20 2024-04-25T12:36:05.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.442+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa20806ef90 con 0x7fa2081f6f20 2024-04-25T12:36:05.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.444+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa20806ef90 con 0x7fa2081f6f20 2024-04-25T12:36:05.443 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:05.444 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:05.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.444+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa208063a30 0x7fa2080ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.444+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa2080cbac0 con 0x7fa208063a30 2024-04-25T12:36:05.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.445+0000 7fa20e600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa208063a30 0x7fa2080ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.445+0000 7fa20e600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa208063a30 0x7fa2080ce220 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.446+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2080cbac0 con 0x7fa208063a30 2024-04-25T12:36:05.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.461+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa2080ce760 con 0x7fa208063a30 2024-04-25T12:36:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.463+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa2080ce760 con 0x7fa208063a30 2024-04-25T12:36:05.462 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:05.463 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:05.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.463+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa2080cee80 0x7fa20802c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.463+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa208208fc0 con 0x7fa2080cee80 2024-04-25T12:36:05.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.464+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa2080cee80 0x7fa20802c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.464+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa2080cee80 0x7fa20802c9d0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.466+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa208208fc0 con 0x7fa2080cee80 2024-04-25T12:36:05.479 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:05.480+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:05.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.481+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa20802cf10 con 0x7fa2080cee80 2024-04-25T12:36:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.483+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa20802cf10 con 0x7fa2080cee80 2024-04-25T12:36:05.482 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.483+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fa20802d590 0x7fa20802d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.483+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa20802dee0 con 0x7fa20802d590 2024-04-25T12:36:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.484+0000 7fa20d200640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fa20802d590 0x7fa20802d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.484+0000 7fa20d200640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fa20802d590 0x7fa20802d9a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.485+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa20802dee0 con 0x7fa20802d590 2024-04-25T12:36:05.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.500+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa20802dee0 con 0x7fa20802d590 2024-04-25T12:36:05.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.502+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa20802dee0 con 0x7fa20802d590 2024-04-25T12:36:05.501 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.502+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa208209dd0 0x7fa20820c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.502+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa20820f040 con 0x7fa208209dd0 2024-04-25T12:36:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.503+0000 7fa20e600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa208209dd0 0x7fa20820c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.503+0000 7fa20e600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa208209dd0 0x7fa20820c1f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.504+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa20820f040 con 0x7fa208209dd0 2024-04-25T12:36:05.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.519+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa20820ca00 con 0x7fa208209dd0 2024-04-25T12:36:05.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.521+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa20820ca00 con 0x7fa208209dd0 2024-04-25T12:36:05.520 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:05.521 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:05.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.521+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa20820d120 0x7fa20820d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:05.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.522+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa208212410 con 0x7fa20820d120 2024-04-25T12:36:05.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.522+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa20820d120 0x7fa20820d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:05.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.523+0000 7fa20dc00640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa20820d120 0x7fa20820d530 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:05.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.524+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa208212410 con 0x7fa20820d120 2024-04-25T12:36:05.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.547+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa208211290 con 0x7fa20820d120 2024-04-25T12:36:05.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa205600640 1 -- 172.21.15.123:0/3715831486 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa208211290 con 0x7fa20820d120 2024-04-25T12:36:05.548 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa208154590 msgr2=0x7fa2081549a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fa208154590 0x7fa2081549a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa2080b1f80 msgr2=0x7fa2081f64d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fa2080b1f80 0x7fa2081f64d0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa2081f6f20 msgr2=0x7fa208062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa2081f6f20 0x7fa208062fe0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa208063a30 msgr2=0x7fa2080ce220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa208063a30 0x7fa2080ce220 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa2080cee80 msgr2=0x7fa20802c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa2080cee80 0x7fa20802c9d0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fa20802d590 msgr2=0x7fa20802d9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fa20802d590 0x7fa20802d9a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa208209dd0 msgr2=0x7fa20820c1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fa208209dd0 0x7fa20820c1f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa20820d120 msgr2=0x7fa20820d530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.549+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa20820d120 0x7fa20820d530 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 reap_dead start 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 reap_dead start 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 reap_dead start 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 reap_dead start 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa20e600640 1 -- 172.21.15.123:0/3715831486 reap_dead start 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1e80744b0 msgr2=0x7fa1e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1e80744b0 0x7fa1e80769c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc000c00 tx=0x7fa1fc02e000 comp rx=0 tx=0).stop 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa208160860 msgr2=0x7fa20815b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa208160860 0x7fa20815b180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c730 tx=0x7fa20000cbf0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 shutdown_connections 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa1e80744b0 0x7fa1e80769c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 --2- 172.21.15.123:0/3715831486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa208160860 0x7fa20815b180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.550+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 >> 172.21.15.123:0/3715831486 conn(0x7fa208068600 msgr2=0x7fa20806dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.551+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 shutdown_connections 2024-04-25T12:36:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:05.551+0000 7fa210600640 1 -- 172.21.15.123:0/3715831486 wait complete. 2024-04-25T12:36:05.579 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:05.580+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:05.619 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:05.619 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:05.619 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:05.619 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:05.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:05.688 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:05.689+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:05.780 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:05.781+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:05.881 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:05.882+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:05.960 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:05.960 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:05.960 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:05.960 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:05.960 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:05.984 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:05.985+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:05.985 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:06.083 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:06.084+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:06.184 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:06.185+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:06.285 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:06.286+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:06.367 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:06.367 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:06.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 118.340863873, 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53759151699999996, 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1 2024-04-25T12:36:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024334+0000", 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7179999999999999e-06 2024-04-25T12:36:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024345+0000", 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0608000000000001e-05 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024373+0000", 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7966e-05 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024392+0000", 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024490+0000", 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7823000000000002e-05 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for peered", 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024520+0000", 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0056000000000002e-05 2024-04-25T12:36:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.559770+0000", 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53525036000000004 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560001+0000", 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023139100000000001 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560330+0000", 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032824500000000001 2024-04-25T12:36:06.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561700+0000", 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013704730000000001 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561742+0000", 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1826000000000002e-05 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561855+0000", 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011287599999999999 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561924+0000", 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9484000000000001e-05 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.77794065400001, 2024-04-25T12:36:06.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095393000000000006, 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587257+0000", 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9100000000000003e-07 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587490+0000", 2024-04-25T12:36:06.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000233186 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587502+0000", 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2499e-05 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587512+0000", 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7699999999999996e-06 2024-04-25T12:36:06.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.591891+0000", 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043783770000000001 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592036+0000", 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014543999999999999 2024-04-25T12:36:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592306+0000", 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026964399999999998 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595185+0000", 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028791730000000001 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596722+0000", 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001537131 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596762+0000", 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0491000000000002e-05 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596795+0000", 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2798e-05 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.77766773499999, 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012300247, 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.382 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.383 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 24 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587530+0000", 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3799999999999996e-07 2024-04-25T12:36:06.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587688+0000", 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015835999999999999 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587699+0000", 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0727000000000001e-05 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587710+0000", 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0987e-05 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593399+0000", 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005688935 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593685+0000", 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028662199999999999 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593893+0000", 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020758500000000001 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595599+0000", 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017058209999999999 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599668+0000", 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040696439999999999 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599769+0000", 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100752 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599829+0000", 2024-04-25T12:36:06.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0075999999999999e-05 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.777445677, 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012351434, 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.392 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 25 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587752+0000", 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587758+0000", 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2199999999999997e-06 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587770+0000", 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2129e-05 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587778+0000", 2024-04-25T12:36:06.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594340+0000", 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065618270000000001 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594415+0000", 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4535000000000004e-05 2024-04-25T12:36:06.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594510+0000", 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5100999999999996e-05 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595607+0000", 2024-04-25T12:36:06.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010968239999999999 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599926+0000", 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043194469999999997 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600027+0000", 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100961 2024-04-25T12:36:06.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600102+0000", 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5339000000000005e-05 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.572496487, 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091591999999999993, 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.399 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792701+0000", 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.17e-06 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792911+0000", 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000210163 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792932+0000", 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0472e-05 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792952+0000", 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0553000000000001e-05 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799157+0000", 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062046030000000004 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799246+0000", 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8505999999999996e-05 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799389+0000", 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014319100000000001 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801232+0000", 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018427599999999999 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801753+0000", 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052153699999999998 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801814+0000", 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1014000000000002e-05 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801859+0000", 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5231000000000001e-05 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.404 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.57205813900001, 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092910930000000003, 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 52 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.405 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793140+0000", 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-06 2024-04-25T12:36:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793151+0000", 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0966e-05 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793168+0000", 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793184+0000", 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6200000000000001e-05 2024-04-25T12:36:06.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801271+0000", 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080863189999999998 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801360+0000", 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9248999999999999e-05 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801515+0000", 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000154883 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802197+0000", 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068243299999999998 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802339+0000", 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014159 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802393+0000", 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4215000000000002e-05 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802430+0000", 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6434999999999999e-05 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.114054869, 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012798129, 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.410 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 70 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251143+0000", 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2729999999999999e-06 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251153+0000", 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8439999999999993e-06 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251172+0000", 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8906000000000002e-05 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251183+0000", 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1653e-05 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257332+0000", 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061486459999999998 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257490+0000", 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000157947 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257724+0000", 2024-04-25T12:36:06.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023408200000000001 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259695+0000", 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001971169 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263731+0000", 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035683 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263842+0000", 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110985 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263940+0000", 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7941000000000003e-05 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.11399796000001, 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013164844, 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.417 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 71 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:06.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251200+0000", 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2999999999999999e-07 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251206+0000", 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9399999999999999e-06 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251215+0000", 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251223+0000", 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2260000000000005e-06 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259955+0000", 2024-04-25T12:36:06.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087315629999999995 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260077+0000", 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122263 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260310+0000", 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232702 2024-04-25T12:36:06.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263699+0000", 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033888080000000001 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264218+0000", 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051973199999999996 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264304+0000", 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5433999999999996e-05 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264363+0000", 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9743e-05 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.423 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.113956351, 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017184050999999999, 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.424 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 72 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251241+0000", 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.23e-07 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251477+0000", 2024-04-25T12:36:06.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023584499999999999 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251504+0000", 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7633000000000001e-05 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251516+0000", 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.164e-05 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260993+0000", 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094771119999999993 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261122+0000", 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012831400000000001 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261319+0000", 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019727199999999999 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264078+0000", 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027589120000000001 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268258+0000", 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041806569999999999 2024-04-25T12:36:06.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268357+0000", 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8721000000000006e-05 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268424+0000", 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7222000000000001e-05 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.430 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.113661408, 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017105597, 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.431 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 73 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251536+0000", 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4300000000000001e-07 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251543+0000", 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3840000000000002e-06 2024-04-25T12:36:06.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251552+0000", 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6860000000000001e-06 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251560+0000", 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0499999999999992e-06 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262615+0000", 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011054494 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262757+0000", 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142521 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262999+0000", 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024147400000000001 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264190+0000", 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011911840000000001 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268472+0000", 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042823940000000001 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268571+0000", 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8218000000000001e-05 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268641+0000", 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0248999999999998e-05 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.113624422, 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017309201999999999, 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.437 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 74 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251573+0000", 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9999999999999997e-07 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251579+0000", 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6330000000000001e-06 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251592+0000", 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3869999999999999e-05 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251621+0000", 2024-04-25T12:36:06.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8966e-05 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264505+0000", 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012883307 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264627+0000", 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012204 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264832+0000", 2024-04-25T12:36:06.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020571899999999999 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268227+0000", 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003394308 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268663+0000", 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043630699999999998 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268761+0000", 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7774000000000003e-05 2024-04-25T12:36:06.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268881+0000", 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120578 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.11355431299999, 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01751983, 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.442 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 75 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:06.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251643+0000", 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5300000000000003e-07 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251849+0000", 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000206345 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251863+0000", 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3077e-05 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251873+0000", 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.08e-05 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265525+0000", 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013651876 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265667+0000", 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014156699999999999 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265939+0000", 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027219700000000001 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268931+0000", 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029921990000000001 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269008+0000", 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6619000000000006e-05 2024-04-25T12:36:06.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269082+0000", 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4401999999999997e-05 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269162+0000", 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9895000000000007e-05 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.11330664499999, 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018357248, 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.448 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 76 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251891+0000", 2024-04-25T12:36:06.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7300000000000005e-07 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251905+0000", 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4279e-05 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251915+0000", 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251935+0000", 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.961e-05 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267152+0000", 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015217341000000001 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267276+0000", 2024-04-25T12:36:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012396 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267509+0000", 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023329500000000001 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268988+0000", 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014781709999999999 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270041+0000", 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001053806 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270161+0000", 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119494 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270247+0000", 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6382000000000001e-05 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.094082136, 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099439890000000003, 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.454 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 80 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:06.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271115+0000", 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271221+0000", 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105713 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271236+0000", 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5415999999999999e-05 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271240+0000", 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3449999999999999e-06 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275441+0000", 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042009090000000001 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275579+0000", 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013740700000000001 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275795+0000", 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021634900000000001 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.277926+0000", 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021311220000000001 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.280958+0000", 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030320490000000002 2024-04-25T12:36:06.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281016+0000", 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8232000000000001e-05 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281058+0000", 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1993999999999998e-05 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.460 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 116.739065771, 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094693499999999996, 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.461 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 92 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626132+0000", 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4399999999999999e-07 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626137+0000", 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4929999999999998e-06 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626147+0000", 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5300000000000002e-06 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626155+0000", 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4130000000000006e-06 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629532+0000", 2024-04-25T12:36:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033774830000000001 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629626+0000", 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3394999999999996e-05 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629792+0000", 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016611100000000001 2024-04-25T12:36:06.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633343+0000", 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00355055 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635469+0000", 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021262009999999999 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635548+0000", 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9167999999999998e-05 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635600+0000", 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2262000000000002e-05 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 116.73902943500001, 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096152330000000008, 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 93 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626168+0000", 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9699999999999995e-07 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626290+0000", 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121941 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626304+0000", 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3729e-05 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626321+0000", 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7062999999999999e-05 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630147+0000", 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003826 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630229+0000", 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1953000000000006e-05 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630372+0000", 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014345199999999999 2024-04-25T12:36:06.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633348+0000", 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029755789999999999 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635672+0000", 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023242829999999999 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635733+0000", 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0899000000000001e-05 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635782+0000", 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9636999999999999e-05 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 116.738855515, 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097157200000000006, 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 94 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:06.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626342+0000", 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4300000000000002e-07 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626348+0000", 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0059999999999998e-06 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626358+0000", 2024-04-25T12:36:06.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626367+0000", 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8939999999999999e-06 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631146+0000", 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047795429999999998 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631307+0000", 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016041699999999999 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631539+0000", 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232258 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635089+0000", 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035503980000000002 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635880+0000", 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079070200000000001 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635984+0000", 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104221 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.636057+0000", 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2597999999999998e-05 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.247145962999994, 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010749583, 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118054+0000", 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118075+0000", 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118119+0000", 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4372000000000001e-05 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118163+0000", 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3766000000000001e-05 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119484+0000", 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001320494 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119803+0000", 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031975599999999998 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120121+0000", 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031758999999999997 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128491+0000", 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083702940000000003 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128605+0000", 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113158 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128733+0000", 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012861099999999999 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128800+0000", 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7086000000000001e-05 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.246946976999993, 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011301215999999999, 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118251+0000", 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6160000000000001e-06 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118263+0000", 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2243e-05 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118305+0000", 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1838999999999997e-05 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118327+0000", 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2082999999999998e-05 2024-04-25T12:36:06.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120981+0000", 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002653201 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121221+0000", 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000240058 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121655+0000", 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043405400000000002 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128501+0000", 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068465519999999997 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128984+0000", 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048240400000000003 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129487+0000", 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050289499999999995 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129551+0000", 2024-04-25T12:36:06.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4270999999999995e-05 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.246839546000004, 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012520414000000001, 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:06.491 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:06.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118358+0000", 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3230000000000001e-06 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118368+0000", 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3899999999999999e-06 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2481999999999999e-05 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118410+0000", 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9304e-05 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126108+0000", 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076979409999999998 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126379+0000", 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027136700000000001 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126653+0000", 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274238 2024-04-25T12:36:06.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129707+0000", 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030541800000000001 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130686+0000", 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097908900000000004 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130791+0000", 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104824 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130877+0000", 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6275999999999999e-05 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:06.497 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:06.498 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:06.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:06.500 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:06.500+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:06.600 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:06.601+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:06.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:36:06.701 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:06.702+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:06.802 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:06.803+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:06.843 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:06.844 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:06.873 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:06.914 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:06.915+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:06.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.943+0000 7fd053e00640 1 Processor -- start 2024-04-25T12:36:06.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.944+0000 7fd053e00640 1 -- start start 2024-04-25T12:36:06.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd053e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd05406c570 0x7fd05406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:06.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd053e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd054164390 con 0x7fd05415ac10 2024-04-25T12:36:06.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd053e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd054164630 con 0x7fd054163ec0 2024-04-25T12:36:06.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd053e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd054164e70 con 0x7fd05406c570 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd052000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd05415ac10 0x7fd054161000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:37036/0 (socket says 172.21.15.123:37036) 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd052000640 1 -- 172.21.15.123:0/2310185774 learned_addr learned my addr 172.21.15.123:0/2310185774 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd052a00640 1 --1- 172.21.15.123:0/2310185774 >> v1:172.21.15.153:6789/0 conn(0x7fd054163ec0 0x7fd0540c8520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:52548/0 (socket says 172.21.15.123:52548) 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.946+0000 7fd053400640 1 --2- 172.21.15.123:0/2310185774 >> v2:172.21.15.153:3300/0 conn(0x7fd05406c570 0x7fd05406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.947+0000 7fd051600640 1 -- 172.21.15.123:0/2310185774 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2446091913 0 0) 0x7fd054164630 con 0x7fd054163ec0 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.947+0000 7fd051600640 1 -- 172.21.15.123:0/2310185774 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd038003520 con 0x7fd054163ec0 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.947+0000 7fd053400640 1 -- 172.21.15.123:0/2310185774 >> v1:172.21.15.153:6789/0 conn(0x7fd054163ec0 legacy=0x7fd0540c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.947+0000 7fd053400640 1 -- 172.21.15.123:0/2310185774 >> v1:172.21.15.123:6789/0 conn(0x7fd05415ac10 legacy=0x7fd054161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:06.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.947+0000 7fd051600640 1 -- 172.21.15.123:0/2310185774 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614083488 0 0) 0x7fd054164390 con 0x7fd05415ac10 2024-04-25T12:36:06.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.947+0000 7fd053400640 1 -- 172.21.15.123:0/2310185774 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0540c8c30 con 0x7fd05406c570 2024-04-25T12:36:06.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.949+0000 7fd053400640 1 --2- 172.21.15.123:0/2310185774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05406c980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009d90 tx=0x7fd04c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cfbc416561fea0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:06.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.949+0000 7fd051600640 1 -- 172.21.15.123:0/2310185774 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c007740 con 0x7fd05406c570 2024-04-25T12:36:06.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.949+0000 7fd051600640 1 -- 172.21.15.123:0/2310185774 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04c0078e0 con 0x7fd05406c570 2024-04-25T12:36:06.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.949+0000 7fd051600640 1 -- 172.21.15.123:0/2310185774 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c007cb0 con 0x7fd05406c570 2024-04-25T12:36:06.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.950+0000 7fd053e00640 1 -- 172.21.15.123:0/2310185774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 msgr2=0x7fd05406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:06.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.950+0000 7fd053e00640 1 --2- 172.21.15.123:0/2310185774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05406c980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009d90 tx=0x7fd04c02a6f0 comp rx=0 tx=0).stop 2024-04-25T12:36:06.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.951+0000 7fd053e00640 1 -- 172.21.15.123:0/2310185774 shutdown_connections 2024-04-25T12:36:06.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.951+0000 7fd053e00640 1 --2- 172.21.15.123:0/2310185774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05406c980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:06.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.951+0000 7fd053e00640 1 -- 172.21.15.123:0/2310185774 >> 172.21.15.123:0/2310185774 conn(0x7fd0540cd600 msgr2=0x7fd0540cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:06.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.951+0000 7fd053e00640 1 -- 172.21.15.123:0/2310185774 shutdown_connections 2024-04-25T12:36:06.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.951+0000 7fd053e00640 1 -- 172.21.15.123:0/2310185774 wait complete. 2024-04-25T12:36:06.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.954+0000 7fd053e00640 1 Processor -- start 2024-04-25T12:36:06.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.955+0000 7fd053e00640 1 -- start start 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.956+0000 7fd053e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.956+0000 7fd053e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd05415ac10 0x7fd05415f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.956+0000 7fd053e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054163ec0 0x7fd05415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.956+0000 7fd053e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd054160b70 con 0x7fd054163ec0 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.956+0000 7fd053e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd05415b020 con 0x7fd05406c570 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.956+0000 7fd053e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd054164b90 con 0x7fd05415ac10 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.956+0000 7fd052a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41546/0 (socket says 172.21.15.123:41546) 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052a00640 1 -- 172.21.15.123:0/1154485049 learned_addr learned my addr 172.21.15.123:0/1154485049 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:06.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052000640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd05415ac10 0x7fd05415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:06.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd053400640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054163ec0 0x7fd05415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052a00640 1 -- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd05415ac10 msgr2=0x7fd05415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052a00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd05415ac10 0x7fd05415f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052a00640 1 -- 172.21.15.123:0/1154485049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054163ec0 msgr2=0x7fd05415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052a00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054163ec0 0x7fd05415cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.957+0000 7fd052a00640 1 -- 172.21.15.123:0/1154485049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05415b7e0 con 0x7fd05406c570 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.958+0000 7fd052a00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05415b1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c440 tx=0x7fd04400c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.958+0000 7fd04aa00640 1 -- 172.21.15.123:0/1154485049 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044018070 con 0x7fd05406c570 2024-04-25T12:36:06.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.958+0000 7fd04aa00640 1 -- 172.21.15.123:0/1154485049 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044007480 con 0x7fd05406c570 2024-04-25T12:36:06.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.958+0000 7fd04aa00640 1 -- 172.21.15.123:0/1154485049 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440047e0 con 0x7fd05406c570 2024-04-25T12:36:06.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.959+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541ec7a0 con 0x7fd05406c570 2024-04-25T12:36:06.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.959+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05415f130 con 0x7fd05406c570 2024-04-25T12:36:06.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.960+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd054076da0 con 0x7fd05406c570 2024-04-25T12:36:06.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.961+0000 7fd04aa00640 1 -- 172.21.15.123:0/1154485049 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd0440049a0 con 0x7fd05406c570 2024-04-25T12:36:06.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.963+0000 7fd04aa00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd0340744e0 0x7fd0340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:06.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.963+0000 7fd04aa00640 1 -- 172.21.15.123:0/1154485049 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6619+0+0 (secure 0 0 0) 0x7fd044094f30 con 0x7fd05406c570 2024-04-25T12:36:06.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.963+0000 7fd052000640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd0340744e0 0x7fd0340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:06.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.964+0000 7fd052000640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd0340744e0 0x7fd0340769f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd040005880 tx=0x7fd040007f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:06.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:06.966+0000 7fd04aa00640 1 -- 172.21.15.123:0/1154485049 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0440c3b80 con 0x7fd05406c570 2024-04-25T12:36:07.004 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:07.005+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:07.104 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:07.105+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:07.186 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:07.186 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:07.186 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:07.186 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:07.186 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:07.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:07.206 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:07.206+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:07.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.261+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fd0540c8e60 con 0x7fd05406c570 2024-04-25T12:36:07.305 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:07.306+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:07.407 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:07.408+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:07.440 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:36:07.446 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:36:07.487 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.53491677, 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:36:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.534804827, 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:36:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:36:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:36:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:36:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.534756027, 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:36:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:36:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:36:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.534684867, 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:36:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:36:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.534636455, 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:36:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.52644052700001, 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:36:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.420196528999995, 2024-04-25T12:36:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:36:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:36:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:36:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:36:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:36:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:36:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.420014527999996, 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:36:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:36:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:36:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.367980540999994, 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:36:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.367904585000005, 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:36:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:36:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.363475844999996, 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:36:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:36:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.363293782, 2024-04-25T12:36:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:36:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:36:07.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.555 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.363189418000005, 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.556 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.361913861000005, 2024-04-25T12:36:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:36:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.360859637000004, 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:36:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:36:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:36:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 72.331993996999998, 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.610482501999996, 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:36:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 54.567921949999999, 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:36:07.580 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:36:07.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:48 2.14 2:2e2fa760:::200.00000001:head [write 5505~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.695530019, 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010227916, 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.587 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790256+0000", 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7570000000000002e-06 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790280+0000", 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4456000000000001e-05 2024-04-25T12:36:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790319+0000", 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9103000000000002e-05 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790344+0000", 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4998999999999999e-05 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790557+0000", 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021251199999999999 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790572+0000", 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5003e-05 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.796879+0000", 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063072359999999999 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797129+0000", 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025002800000000001 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797609+0000", 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047965200000000001 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800188+0000", 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002578922 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800281+0000", 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3561999999999996e-05 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800387+0000", 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010545800000000001 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800480+0000", 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3227999999999996e-05 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:49 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.695260437999998, 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011186097000000001, 2024-04-25T12:36:07.595 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:07.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790524+0000", 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999998e-06 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790541+0000", 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7567e-05 2024-04-25T12:36:07.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790566+0000", 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4338999999999999e-05 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790581+0000", 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5123000000000001e-05 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790708+0000", 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012676699999999999 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790718+0000", 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:07.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.798785+0000", 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080674500000000003 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799024+0000", 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000239044 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799376+0000", 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035158299999999998 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800980+0000", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016040939999999999 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801537+0000", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055677400000000001 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801639+0000", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102388 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801708+0000", 2024-04-25T12:36:07.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8504999999999995e-05 2024-04-25T12:36:07.602 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.602 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.602 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.602 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:07.602 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:07.602 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:07.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:07.605 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:07.605 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:07.604+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:07.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.637+0000 7fd04aa00640 1 -- 172.21.15.123:0/1154485049 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v51) ==== 91+0+0 (secure 0 0 0) 0x7fd044021070 con 0x7fd05406c570 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd0340744e0 msgr2=0x7fd0340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd0340744e0 0x7fd0340769f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd040005880 tx=0x7fd040007f00 comp rx=0 tx=0).stop 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 msgr2=0x7fd05415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05415b1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c440 tx=0x7fd04400c900 comp rx=0 tx=0).stop 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 shutdown_connections 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd0340744e0 0x7fd0340769f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054163ec0 0x7fd05415cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd05415ac10 0x7fd05415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 --2- 172.21.15.123:0/1154485049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd05406c570 0x7fd05415b1c0 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c440 tx=0x7fd04400c900 comp rx=0 tx=0).stop 2024-04-25T12:36:07.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.641+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 >> 172.21.15.123:0/1154485049 conn(0x7fd0540cd600 msgr2=0x7fd0540ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:07.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.642+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 shutdown_connections 2024-04-25T12:36:07.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.642+0000 7fd053e00640 1 -- 172.21.15.123:0/1154485049 wait complete. 2024-04-25T12:36:07.704 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:07.705+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:07.804 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:07.805+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:07.905 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:07.906+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:07.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.977+0000 7efe9ae00640 1 Processor -- start 2024-04-25T12:36:07.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.977+0000 7efe9ae00640 1 -- start start 2024-04-25T12:36:07.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9ae00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efe9406f070 0x7efe9415c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9ae00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe9415c950 0x7efe940ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9ae00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe940d0520 con 0x7efe940d0110 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9ae00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efe940cfd10 con 0x7efe9415c950 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9ae00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7efe940cc950 con 0x7efe9406f070 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9a400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe9415c950 0x7efe940ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe99000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efe9406f070 0x7efe9415c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9a400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe9415c950 0x7efe940ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53790/0 (socket says 172.21.15.123:53790) 2024-04-25T12:36:07.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.979+0000 7efe9a400640 1 -- 172.21.15.123:0/3334902010 learned_addr learned my addr 172.21.15.123:0/3334902010 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:07.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.980+0000 7efe99000640 1 -- 172.21.15.123:0/3334902010 >> v2:172.21.15.123:3300/0 conn(0x7efe9415c950 msgr2=0x7efe940ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:07.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.980+0000 7efe99000640 1 --2- 172.21.15.123:0/3334902010 >> v2:172.21.15.123:3300/0 conn(0x7efe9415c950 0x7efe940ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.980+0000 7efe99000640 1 -- 172.21.15.123:0/3334902010 >> v1:172.21.15.123:6789/0 conn(0x7efe940d0110 legacy=0x7efe94156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:07.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.980+0000 7efe99000640 1 -- 172.21.15.123:0/3334902010 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe940cc340 con 0x7efe9406f070 2024-04-25T12:36:07.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.980+0000 7efe9a400640 1 --2- 172.21.15.123:0/3334902010 >> v2:172.21.15.123:3300/0 conn(0x7efe9415c950 0x7efe940ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:07.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.981+0000 7efe99000640 1 --2- 172.21.15.123:0/3334902010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe9406f070 0x7efe9415c410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efe940b11d0 tx=0x7efe8002a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc4f3a5639e8bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:07.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.982+0000 7efe93e00640 1 -- 172.21.15.123:0/3334902010 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80037070 con 0x7efe9406f070 2024-04-25T12:36:07.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.982+0000 7efe93e00640 1 -- 172.21.15.123:0/3334902010 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8002ade0 con 0x7efe9406f070 2024-04-25T12:36:07.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.982+0000 7efe93e00640 1 -- 172.21.15.123:0/3334902010 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80032660 con 0x7efe9406f070 2024-04-25T12:36:07.982 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:07.982 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:07.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.983+0000 7efe9ae00640 1 -- 172.21.15.123:0/3334902010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe9406f070 msgr2=0x7efe9415c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:07.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.983+0000 7efe9ae00640 1 --2- 172.21.15.123:0/3334902010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe9406f070 0x7efe9415c410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efe940b11d0 tx=0x7efe8002a090 comp rx=0 tx=0).stop 2024-04-25T12:36:07.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.983+0000 7efe9ae00640 1 -- 172.21.15.123:0/3334902010 shutdown_connections 2024-04-25T12:36:07.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.983+0000 7efe9ae00640 1 --2- 172.21.15.123:0/3334902010 >> v2:172.21.15.123:3300/0 conn(0x7efe9415c950 0x7efe940ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.983+0000 7efe9ae00640 1 --2- 172.21.15.123:0/3334902010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe9406f070 0x7efe9415c410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.983+0000 7efe9ae00640 1 -- 172.21.15.123:0/3334902010 >> 172.21.15.123:0/3334902010 conn(0x7efe940b33b0 msgr2=0x7efe940b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:07.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.983+0000 7efe9ae00640 1 -- 172.21.15.123:0/3334902010 shutdown_connections 2024-04-25T12:36:07.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.984+0000 7efe9ae00640 1 -- 172.21.15.123:0/3334902010 wait complete. 2024-04-25T12:36:07.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.987+0000 7efe9ae00640 1 Processor -- start 2024-04-25T12:36:07.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.987+0000 7efe9ae00640 1 -- start start 2024-04-25T12:36:07.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.989+0000 7efe9ae00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efe9406f070 0x7efe940c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:07.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.989+0000 7efe9ae00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe940d0110 0x7efe940ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:07.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.989+0000 7efe9ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9415c950 0x7efe94154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:07.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.989+0000 7efe9ae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe940cfa30 con 0x7efe9415c950 2024-04-25T12:36:07.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.989+0000 7efe9ae00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe940cf750 con 0x7efe940d0110 2024-04-25T12:36:07.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.989+0000 7efe9ae00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7efe940cc5e0 con 0x7efe9406f070 2024-04-25T12:36:07.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe940d0110 0x7efe940ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:07.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efe9406f070 0x7efe940c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:07.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe940d0110 0x7efe940ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41558/0 (socket says 172.21.15.123:41558) 2024-04-25T12:36:07.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 -- 172.21.15.123:0/1048182381 learned_addr learned my addr 172.21.15.123:0/1048182381 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:07.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe9a400640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9415c950 0x7efe94154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:07.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efe9406f070 msgr2=0x7efe940c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efe9406f070 0x7efe940c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9415c950 msgr2=0x7efe94154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9415c950 0x7efe94154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe941574d0 con 0x7efe940d0110 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.990+0000 7efe99000640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe940d0110 0x7efe940ca380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efe80000c00 tx=0x7efe80004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.991+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80041020 con 0x7efe940d0110 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.991+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8003d640 con 0x7efe940d0110 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.991+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80037040 con 0x7efe940d0110 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.991+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe941f2280 con 0x7efe940d0110 2024-04-25T12:36:07.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.991+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe941f2840 con 0x7efe940d0110 2024-04-25T12:36:07.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.992+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7efe94076da0 con 0x7efe940d0110 2024-04-25T12:36:07.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.994+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efe80044020 con 0x7efe940d0110 2024-04-25T12:36:07.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.994+0000 7efe91600640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7efe74074500 0x7efe74076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:07.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.995+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6619+0+0 (secure 0 0 0) 0x7efe8007bdd0 con 0x7efe940d0110 2024-04-25T12:36:07.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.995+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7efe74074500 0x7efe74076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:07.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.995+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v51) ==== 54+0+16 (secure 0 0 0) 0x7efe8007fee0 con 0x7efe940d0110 2024-04-25T12:36:07.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.995+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7efe940cafd0 0x7efe940cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:07.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.996+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe941577e0 con 0x7efe940cafd0 2024-04-25T12:36:07.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.996+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7efe74074500 0x7efe74076a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efe88014f70 tx=0x7efe88016040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:07.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.996+0000 7efe9a400640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7efe940cafd0 0x7efe940cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:07.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.997+0000 7efe9a400640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7efe940cafd0 0x7efe940cb3e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:07.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:07.998+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe941577e0 con 0x7efe940cafd0 2024-04-25T12:36:08.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:08.006 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:08.006+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:08.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.018+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe940b4b80 con 0x7efe940cafd0 2024-04-25T12:36:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.020+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe940b4b80 con 0x7efe940cafd0 2024-04-25T12:36:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.021+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7efe941500c0 0x7efe941f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.021+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7efe940cb920 con 0x7efe941500c0 2024-04-25T12:36:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.022+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7efe941500c0 0x7efe941f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.023+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7efe941500c0 0x7efe941f5690 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.024+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe940cb920 con 0x7efe941500c0 2024-04-25T12:36:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.054+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe941e5c60 con 0x7efe941500c0 2024-04-25T12:36:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.057+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe941e5c60 con 0x7efe941500c0 2024-04-25T12:36:08.056 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.057+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7efe941f60e0 0x7efe94062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.057+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7efe941f64f0 con 0x7efe941f60e0 2024-04-25T12:36:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.058+0000 7efe99000640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7efe941f60e0 0x7efe94062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.058+0000 7efe99000640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7efe941f60e0 0x7efe94062fe0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:08.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.060+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe941f64f0 con 0x7efe941f60e0 2024-04-25T12:36:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.087+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe940738b0 con 0x7efe941f60e0 2024-04-25T12:36:08.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.090+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe940738b0 con 0x7efe941f60e0 2024-04-25T12:36:08.090 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.091+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7efe9406d680 0x7efe9406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.091+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7efe94063df0 con 0x7efe9406d680 2024-04-25T12:36:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.091+0000 7efe9a400640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7efe9406d680 0x7efe9406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:08.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.092+0000 7efe9a400640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7efe9406d680 0x7efe9406da90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:08.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.093+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe94063df0 con 0x7efe9406d680 2024-04-25T12:36:08.106 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:08.107+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:08.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.121+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe9406e2a0 con 0x7efe9406d680 2024-04-25T12:36:08.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.124+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe9406e2a0 con 0x7efe9406d680 2024-04-25T12:36:08.123 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:08.123 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:08.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.124+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7efe9402cee0 0x7efe9402d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:08.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.124+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7efe9406e530 con 0x7efe9402cee0 2024-04-25T12:36:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.124+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7efe9402cee0 0x7efe9402d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.125+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7efe9402cee0 0x7efe9402d2f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.126+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe9406e530 con 0x7efe9402cee0 2024-04-25T12:36:08.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.140+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe9406e4b0 con 0x7efe9402cee0 2024-04-25T12:36:08.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.143+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe9406e4b0 con 0x7efe9402cee0 2024-04-25T12:36:08.142 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:08.143 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:08.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.143+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7efe94202a40 0x7efe94204e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:08.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.143+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7efe9402df00 con 0x7efe94202a40 2024-04-25T12:36:08.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.144+0000 7efe99000640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7efe94202a40 0x7efe94204e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:08.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.144+0000 7efe99000640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7efe94202a40 0x7efe94204e60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:08.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.145+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe9402df00 con 0x7efe94202a40 2024-04-25T12:36:08.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.173+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe942053a0 con 0x7efe94202a40 2024-04-25T12:36:08.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.175+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe942053a0 con 0x7efe94202a40 2024-04-25T12:36:08.174 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:08.175 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:08.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.176+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7efe94205ac0 0x7efe94205ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:08.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.176+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7efe9420aea0 con 0x7efe94205ac0 2024-04-25T12:36:08.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.176+0000 7efe9a400640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7efe94205ac0 0x7efe94205ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.177+0000 7efe9a400640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7efe94205ac0 0x7efe94205ed0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.178+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe9420aea0 con 0x7efe94205ac0 2024-04-25T12:36:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.192+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe94209d20 con 0x7efe94205ac0 2024-04-25T12:36:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.194+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe94209d20 con 0x7efe94205ac0 2024-04-25T12:36:08.193 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:08.194 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:08.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.194+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7efe9420a3a0 0x7efe9420a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:08.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.194+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7efe9420e290 con 0x7efe9420a3a0 2024-04-25T12:36:08.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.195+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7efe9420a3a0 0x7efe9420a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:08.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.196+0000 7efe99a00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7efe9420a3a0 0x7efe9420a7d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:08.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.196+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe9420e290 con 0x7efe9420a3a0 2024-04-25T12:36:08.208 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:08.209+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.218+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7efe94206410 con 0x7efe9420a3a0 2024-04-25T12:36:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.220+0000 7efe91600640 1 -- 172.21.15.123:0/1048182381 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe94206410 con 0x7efe9420a3a0 2024-04-25T12:36:08.219 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.220+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7efe940cafd0 msgr2=0x7efe940cb3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7efe940cafd0 0x7efe940cb3e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7efe941500c0 msgr2=0x7efe941f5690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7efe941500c0 0x7efe941f5690 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7efe941f60e0 msgr2=0x7efe94062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7efe941f60e0 0x7efe94062fe0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7efe9406d680 msgr2=0x7efe9406da90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7efe9406d680 0x7efe9406da90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7efe9402cee0 msgr2=0x7efe9402d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7efe9402cee0 0x7efe9402d2f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7efe94202a40 msgr2=0x7efe94204e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7efe94202a40 0x7efe94204e60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7efe94205ac0 msgr2=0x7efe94205ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7efe94205ac0 0x7efe94205ed0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7efe9420a3a0 msgr2=0x7efe9420a7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7efe9420a3a0 0x7efe9420a7d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9a400640 1 -- 172.21.15.123:0/1048182381 reap_dead start 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9a400640 1 -- 172.21.15.123:0/1048182381 reap_dead start 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9a400640 1 -- 172.21.15.123:0/1048182381 reap_dead start 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9a400640 1 -- 172.21.15.123:0/1048182381 reap_dead start 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9a400640 1 -- 172.21.15.123:0/1048182381 reap_dead start 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7efe74074500 msgr2=0x7efe74076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7efe74074500 0x7efe74076a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efe88014f70 tx=0x7efe88016040 comp rx=0 tx=0).stop 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe940d0110 msgr2=0x7efe940ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe940d0110 0x7efe940ca380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efe80000c00 tx=0x7efe80004730 comp rx=0 tx=0).stop 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 shutdown_connections 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7efe74074500 0x7efe74076a10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 --2- 172.21.15.123:0/1048182381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe940d0110 0x7efe940ca380 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.221+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 >> 172.21.15.123:0/1048182381 conn(0x7efe940b33b0 msgr2=0x7efe9415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:08.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.222+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 shutdown_connections 2024-04-25T12:36:08.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:08.222+0000 7efe9ae00640 1 -- 172.21.15.123:0/1048182381 wait complete. 2024-04-25T12:36:08.309 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:08.310+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:08.325 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:08.325 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:08.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:08.410 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:08.411+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:08.511 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:08.512+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:08.611 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:08.613+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:08.617 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:08.617 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:08.639 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:08.713 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:08.714+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:08.761 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:08.762 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:08.771 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:08.813 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:08.814+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:08.884 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:08.884 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:08.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:08.914 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:08.915+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:09.005 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:09.005 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:09.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:09.015 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:09.015+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:09.114 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:09.115+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:09.169 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:09.169 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:09.178 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:09.216 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:09.217+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:09.317 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:09.317+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:09.329 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:09.330 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:09.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:09.417 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:09.417+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:09.452 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:09.452 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:09.461 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:09.545 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:09.546+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:09.614 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:09.614 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:09.618 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:09.619+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:09.623 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:09.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:36:09.722 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:09.723+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:09.775 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:09.775 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:09.784 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:09.820 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:09.821+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:09.896 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:09.896 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:09.905 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:09.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.909+0000 7fd8c5a00640 1 Processor -- start 2024-04-25T12:36:09.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.909+0000 7fd8c5a00640 1 -- start start 2024-04-25T12:36:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.913+0000 7fd8c5a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd8c00c94f0 0x7fd8c00c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.913+0000 7fd8c5a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c01593c0 con 0x7fd8c0072d70 2024-04-25T12:36:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.913+0000 7fd8c5a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c01596a0 con 0x7fd8c00c2720 2024-04-25T12:36:09.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.913+0000 7fd8c5a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c0159fd0 con 0x7fd8c00c94f0 2024-04-25T12:36:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.913+0000 7fd8bf400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd8c00c94f0 0x7fd8c00c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.913+0000 7fd8c5000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd8c00c2720 0x7fd8c00c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:52550/0 (socket says 172.21.15.123:52550) 2024-04-25T12:36:09.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.913+0000 7fd8c5000640 1 -- 172.21.15.123:0/3942927175 learned_addr learned my addr 172.21.15.123:0/3942927175 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.914+0000 7fd8bf400640 1 -- 172.21.15.123:0/3942927175 >> v1:172.21.15.153:6789/0 conn(0x7fd8c00c2720 legacy=0x7fd8c00c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.914+0000 7fd8bea00640 1 -- 172.21.15.123:0/3942927175 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684644438 0 0) 0x7fd8c01596a0 con 0x7fd8c00c2720 2024-04-25T12:36:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.914+0000 7fd8bf400640 1 -- 172.21.15.123:0/3942927175 >> v1:172.21.15.123:6789/0 conn(0x7fd8c0072d70 legacy=0x7fd8c01420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.914+0000 7fd8bf400640 1 -- 172.21.15.123:0/3942927175 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c0159980 con 0x7fd8c00c94f0 2024-04-25T12:36:09.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.915+0000 7fd8bf400640 1 --2- 172.21.15.123:0/3942927175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c00c94f0 0x7fd8c00c9900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0097a0 tx=0x7fd8ac02c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724f9075c1ede19c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:09.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.915+0000 7fd8bea00640 1 -- 172.21.15.123:0/3942927175 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac038070 con 0x7fd8c00c94f0 2024-04-25T12:36:09.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.915+0000 7fd8bea00640 1 -- 172.21.15.123:0/3942927175 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ac031710 con 0x7fd8c00c94f0 2024-04-25T12:36:09.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.915+0000 7fd8bea00640 1 -- 172.21.15.123:0/3942927175 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac031ae0 con 0x7fd8c00c94f0 2024-04-25T12:36:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.916+0000 7fd8c5a00640 1 -- 172.21.15.123:0/3942927175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c00c94f0 msgr2=0x7fd8c00c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.916+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/3942927175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c00c94f0 0x7fd8c00c9900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0097a0 tx=0x7fd8ac02c4d0 comp rx=0 tx=0).stop 2024-04-25T12:36:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.916+0000 7fd8c5a00640 1 -- 172.21.15.123:0/3942927175 shutdown_connections 2024-04-25T12:36:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.916+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/3942927175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c00c94f0 0x7fd8c00c9900 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.916+0000 7fd8c5a00640 1 -- 172.21.15.123:0/3942927175 >> 172.21.15.123:0/3942927175 conn(0x7fd8c00cd600 msgr2=0x7fd8c00cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.916+0000 7fd8c5a00640 1 -- 172.21.15.123:0/3942927175 shutdown_connections 2024-04-25T12:36:09.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.916+0000 7fd8c5a00640 1 -- 172.21.15.123:0/3942927175 wait complete. 2024-04-25T12:36:09.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.920+0000 7fd8c5a00640 1 Processor -- start 2024-04-25T12:36:09.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.920+0000 7fd8c5a00640 1 -- start start 2024-04-25T12:36:09.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8c5a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c0072d70 0x7fd8c01eee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:09.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8c5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c00c2720 0x7fd8c00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:09.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8c5a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8c00c94f0 0x7fd8c0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8c5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00c8990 con 0x7fd8c00c2720 2024-04-25T12:36:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8c5a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00c8810 con 0x7fd8c0072d70 2024-04-25T12:36:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8c5a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c015b9a0 con 0x7fd8c00c94f0 2024-04-25T12:36:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8c5000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8c00c94f0 0x7fd8c0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c0072d70 0x7fd8c01eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c0072d70 0x7fd8c01eee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41582/0 (socket says 172.21.15.123:41582) 2024-04-25T12:36:09.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8bfe00640 1 -- 172.21.15.123:0/715511589 learned_addr learned my addr 172.21.15.123:0/715511589 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:09.923 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:09.922+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.922+0000 7fd8bf400640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c00c2720 0x7fd8c00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8c5000640 1 -- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c0072d70 msgr2=0x7fd8c01eee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8c5000640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c0072d70 0x7fd8c01eee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8c5000640 1 -- 172.21.15.123:0/715511589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c00c2720 msgr2=0x7fd8c00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8c5000640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c00c2720 0x7fd8c00cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8c5000640 1 -- 172.21.15.123:0/715511589 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c0156510 con 0x7fd8c00c94f0 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8bf400640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c00c2720 0x7fd8c00cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:09.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8bfe00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c0072d70 0x7fd8c01eee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:09.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8c5000640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8c00c94f0 0x7fd8c0154af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800c750 tx=0x7fd8b800cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:09.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8b7e00640 1 -- 172.21.15.123:0/715511589 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8018070 con 0x7fd8c00c94f0 2024-04-25T12:36:09.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8b7e00640 1 -- 172.21.15.123:0/715511589 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b8004d60 con 0x7fd8c00c94f0 2024-04-25T12:36:09.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.923+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c01567f0 con 0x7fd8c00c94f0 2024-04-25T12:36:09.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.924+0000 7fd8b7e00640 1 -- 172.21.15.123:0/715511589 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800a840 con 0x7fd8c00c94f0 2024-04-25T12:36:09.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.924+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01ee970 con 0x7fd8c00c94f0 2024-04-25T12:36:09.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.925+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c0076da0 con 0x7fd8c00c94f0 2024-04-25T12:36:09.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.927+0000 7fd8b7e00640 1 -- 172.21.15.123:0/715511589 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd8b800a9e0 con 0x7fd8c00c94f0 2024-04-25T12:36:09.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.929+0000 7fd8b7e00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd8a00744b0 0x7fd8a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:09.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.929+0000 7fd8b7e00640 1 -- 172.21.15.123:0/715511589 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 6619+0+0 (secure 0 0 0) 0x7fd8b8094b80 con 0x7fd8c00c94f0 2024-04-25T12:36:09.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.929+0000 7fd8bfe00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd8a00744b0 0x7fd8a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:09.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.930+0000 7fd8bfe00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd8a00744b0 0x7fd8a00769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00016e0 tx=0x7fd8b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:09.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:09.933+0000 7fd8b7e00640 1 -- 172.21.15.123:0/715511589 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8b805eee0 con 0x7fd8c00c94f0 2024-04-25T12:36:10.018 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:10.018 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:10.021 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:10.022+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:10.028 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:10.122 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:10.123+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:10.159 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:10.159 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:10.159 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-25T12:36:10.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-25T12:36:10.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.168+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fd8c0073810 con 0x7fd8c00c94f0 2024-04-25T12:36:10.180 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:10.180 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:10.189 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:10.223 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:10.224+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:10.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:10.324 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:10.325+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:10.341 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:10.341 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:10.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:10.425 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:10.426+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.454+0000 7f5f8fc00640 1 Processor -- start 2024-04-25T12:36:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.454+0000 7f5f8fc00640 1 -- start start 2024-04-25T12:36:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.457+0000 7f5f8fc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5f880c90d0 0x7f5f880c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.457+0000 7f5f8fc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f880c4240 0x7f5f880cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.457+0000 7f5f8fc00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f880c4810 con 0x7f5f8815a730 2024-04-25T12:36:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.457+0000 7f5f8fc00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f5f880c83f0 con 0x7f5f880c90d0 2024-04-25T12:36:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.457+0000 7f5f8fc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5f880c86d0 con 0x7f5f880c4240 2024-04-25T12:36:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f8dc00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5f8815a730 0x7f5f88073db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:52552/0 (socket says 172.21.15.123:52552) 2024-04-25T12:36:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f8dc00640 1 -- 172.21.15.123:0/8540938 learned_addr learned my addr 172.21.15.123:0/8540938 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f87e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f880c4240 0x7f5f880cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f8d200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5f880c90d0 0x7f5f880c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f87e00640 1 -- 172.21.15.123:0/8540938 >> v2:172.21.15.153:3300/0 conn(0x7f5f880c90d0 msgr2=0x7f5f880c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f87e00640 1 --2- 172.21.15.123:0/8540938 >> v2:172.21.15.153:3300/0 conn(0x7f5f880c90d0 0x7f5f880c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f87e00640 1 -- 172.21.15.123:0/8540938 >> v1:172.21.15.153:6789/0 conn(0x7f5f8815a730 legacy=0x7f5f88073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f87e00640 1 -- 172.21.15.123:0/8540938 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f880c89b0 con 0x7f5f880c4240 2024-04-25T12:36:10.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.458+0000 7f5f8d200640 1 --2- 172.21.15.123:0/8540938 >> v2:172.21.15.153:3300/0 conn(0x7f5f880c90d0 0x7f5f880c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:10.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.460+0000 7f5f87e00640 1 --2- 172.21.15.123:0/8540938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f880c4240 0x7f5f880cdc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5f740097a0 tx=0x7f5f7402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c33110ca87eae77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.460+0000 7f5f87400640 1 -- 172.21.15.123:0/8540938 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74037070 con 0x7f5f880c4240 2024-04-25T12:36:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.460+0000 7f5f87400640 1 -- 172.21.15.123:0/8540938 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f74032440 con 0x7f5f880c4240 2024-04-25T12:36:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.460+0000 7f5f87400640 1 -- 172.21.15.123:0/8540938 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74032810 con 0x7f5f880c4240 2024-04-25T12:36:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.461+0000 7f5f8fc00640 1 -- 172.21.15.123:0/8540938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f880c4240 msgr2=0x7f5f880cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.461+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/8540938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f880c4240 0x7f5f880cdc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5f740097a0 tx=0x7f5f7402a230 comp rx=0 tx=0).stop 2024-04-25T12:36:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.462+0000 7f5f8fc00640 1 -- 172.21.15.123:0/8540938 shutdown_connections 2024-04-25T12:36:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.462+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/8540938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f880c4240 0x7f5f880cdc70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.462+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/8540938 >> v2:172.21.15.153:3300/0 conn(0x7f5f880c90d0 0x7f5f880c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.462+0000 7f5f8fc00640 1 -- 172.21.15.123:0/8540938 >> 172.21.15.123:0/8540938 conn(0x7f5f8806c750 msgr2=0x7f5f8806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.462+0000 7f5f8fc00640 1 -- 172.21.15.123:0/8540938 shutdown_connections 2024-04-25T12:36:10.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.462+0000 7f5f8fc00640 1 -- 172.21.15.123:0/8540938 wait complete. 2024-04-25T12:36:10.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.465+0000 7f5f8fc00640 1 Processor -- start 2024-04-25T12:36:10.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.466+0000 7f5f8fc00640 1 -- start start 2024-04-25T12:36:10.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8fc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f880c4240 0x7f5f8814ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8fc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f880c90d0 0x7f5f88149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8fc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f8815a730 0x7f5f88149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8fc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880ce330 con 0x7f5f880c90d0 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8fc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880ce1b0 con 0x7f5f880c4240 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8fc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5f880744c0 con 0x7f5f8815a730 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f8815a730 0x7f5f88149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8d200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f880c4240 0x7f5f8814ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f8815a730 0x7f5f88149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:51488/0 (socket says 172.21.15.123:51488) 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.468+0000 7f5f8dc00640 1 -- 172.21.15.123:0/829935043 learned_addr learned my addr 172.21.15.123:0/829935043 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:10.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.469+0000 7f5f87e00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f880c90d0 0x7f5f88149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.469+0000 7f5f8d200640 1 -- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f8815a730 msgr2=0x7f5f88149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.469+0000 7f5f8d200640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f8815a730 0x7f5f88149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.469+0000 7f5f8d200640 1 -- 172.21.15.123:0/829935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f880c90d0 msgr2=0x7f5f88149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.469+0000 7f5f8d200640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f880c90d0 0x7f5f88149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.469+0000 7f5f8d200640 1 -- 172.21.15.123:0/829935043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f881568f0 con 0x7f5f880c4240 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.469+0000 7f5f87e00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f880c90d0 0x7f5f88149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.470+0000 7f5f8d200640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f880c4240 0x7f5f8814ab90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00e870 tx=0x7f5f7c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.470+0000 7f5f84c00640 1 -- 172.21.15.123:0/829935043 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c017070 con 0x7f5f880c4240 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.470+0000 7f5f84c00640 1 -- 172.21.15.123:0/829935043 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7c00c940 con 0x7f5f880c4240 2024-04-25T12:36:10.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.470+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f88156b90 con 0x7f5f880c4240 2024-04-25T12:36:10.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.470+0000 7f5f84c00640 1 -- 172.21.15.123:0/829935043 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c00cd30 con 0x7f5f880c4240 2024-04-25T12:36:10.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.470+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f880c77a0 con 0x7f5f880c4240 2024-04-25T12:36:10.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.472+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8806d500 con 0x7f5f880c4240 2024-04-25T12:36:10.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.474+0000 7f5f84c00640 1 -- 172.21.15.123:0/829935043 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f7c01a070 con 0x7f5f880c4240 2024-04-25T12:36:10.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.475+0000 7f5f84c00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f680744e0 0x7f5f680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.475+0000 7f5f84c00640 1 -- 172.21.15.123:0/829935043 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6619+0+0 (secure 0 0 0) 0x7f5f7c094fa0 con 0x7f5f880c4240 2024-04-25T12:36:10.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.476+0000 7f5f87e00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f680744e0 0x7f5f680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.477+0000 7f5f87e00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f680744e0 0x7f5f680769f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f74034000 tx=0x7f5f7402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.480+0000 7f5f84c00640 1 -- 172.21.15.123:0/829935043 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f7c05ed40 con 0x7f5f880c4240 2024-04-25T12:36:10.526 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:10.528+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:10.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.529+0000 7fb3b8e00640 1 Processor -- start 2024-04-25T12:36:10.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.529+0000 7fb3b8e00640 1 -- start start 2024-04-25T12:36:10.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.531+0000 7fb3b8e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3b41543f0 0x7fb3b40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.531+0000 7fb3b8e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb3b40ccbb0 0x7fb3b40d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.531+0000 7fb3b8e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b415b0b0 con 0x7fb3b40c4500 2024-04-25T12:36:10.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.531+0000 7fb3b8e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b40cb370 con 0x7fb3b41543f0 2024-04-25T12:36:10.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.531+0000 7fb3b8e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b40cc010 con 0x7fb3b40ccbb0 2024-04-25T12:36:10.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb3b40ccbb0 0x7fb3b40d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb3b40ccbb0 0x7fb3b40d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41604/0 (socket says 172.21.15.123:41604) 2024-04-25T12:36:10.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3e00640 1 -- 172.21.15.123:0/1346235162 learned_addr learned my addr 172.21.15.123:0/1346235162 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:10.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3400640 1 --2- 172.21.15.123:0/1346235162 >> v2:172.21.15.123:3300/0 conn(0x7fb3b41543f0 0x7fb3b40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3e00640 1 -- 172.21.15.123:0/1346235162 >> v2:172.21.15.123:3300/0 conn(0x7fb3b41543f0 msgr2=0x7fb3b40cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:10.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/1346235162 >> v2:172.21.15.123:3300/0 conn(0x7fb3b41543f0 0x7fb3b40cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3e00640 1 -- 172.21.15.123:0/1346235162 >> v1:172.21.15.153:6789/0 conn(0x7fb3b40c4500 legacy=0x7fb3b40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:10.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.532+0000 7fb3b3e00640 1 -- 172.21.15.123:0/1346235162 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40cadb0 con 0x7fb3b40ccbb0 2024-04-25T12:36:10.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.534+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/1346235162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b40ccbb0 0x7fb3b40d0090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40726f0 tx=0x7fb3ac02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9713cce54d95115a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.534+0000 7fb3b2000640 1 -- 172.21.15.123:0/1346235162 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac036070 con 0x7fb3b40ccbb0 2024-04-25T12:36:10.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.534+0000 7fb3b2000640 1 -- 172.21.15.123:0/1346235162 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac031600 con 0x7fb3b40ccbb0 2024-04-25T12:36:10.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.534+0000 7fb3b2000640 1 -- 172.21.15.123:0/1346235162 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0319d0 con 0x7fb3b40ccbb0 2024-04-25T12:36:10.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.535+0000 7fb3b8e00640 1 -- 172.21.15.123:0/1346235162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b40ccbb0 msgr2=0x7fb3b40d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.535+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/1346235162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b40ccbb0 0x7fb3b40d0090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40726f0 tx=0x7fb3ac02a780 comp rx=0 tx=0).stop 2024-04-25T12:36:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.535+0000 7fb3b8e00640 1 -- 172.21.15.123:0/1346235162 shutdown_connections 2024-04-25T12:36:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.535+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/1346235162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b40ccbb0 0x7fb3b40d0090 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.535+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/1346235162 >> v2:172.21.15.123:3300/0 conn(0x7fb3b41543f0 0x7fb3b40cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.535+0000 7fb3b8e00640 1 -- 172.21.15.123:0/1346235162 >> 172.21.15.123:0/1346235162 conn(0x7fb3b4068810 msgr2=0x7fb3b4068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.535+0000 7fb3b8e00640 1 -- 172.21.15.123:0/1346235162 shutdown_connections 2024-04-25T12:36:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.536+0000 7fb3b8e00640 1 -- 172.21.15.123:0/1346235162 wait complete. 2024-04-25T12:36:10.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.538+0000 7fb3b8e00640 1 Processor -- start 2024-04-25T12:36:10.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.538+0000 7fb3b8e00640 1 -- start start 2024-04-25T12:36:10.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b8e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3b40c4500 0x7fb3b4154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b40ccbb0 0x7fb3b40cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41543f0 0x7fb3b41500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40cb590 con 0x7fb3b40ccbb0 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b8e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40cbca0 con 0x7fb3b41543f0 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b8e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b40cb710 con 0x7fb3b40c4500 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b3400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3b40c4500 0x7fb3b4154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41543f0 0x7fb3b41500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41543f0 0x7fb3b41500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41620/0 (socket says 172.21.15.123:41620) 2024-04-25T12:36:10.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.540+0000 7fb3b3e00640 1 -- 172.21.15.123:0/3816592836 learned_addr learned my addr 172.21.15.123:0/3816592836 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:10.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.541+0000 7fb3b2a00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b40ccbb0 0x7fb3b40cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.541+0000 7fb3b3400640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41543f0 msgr2=0x7fb3b41500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.541+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41543f0 0x7fb3b41500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.541+0000 7fb3b3400640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b40ccbb0 msgr2=0x7fb3b40cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.541+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b40ccbb0 0x7fb3b40cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.541+0000 7fb3b3400640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b4155140 con 0x7fb3b40c4500 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.541+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3b41543f0 0x7fb3b41500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.542+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3b40c4500 0x7fb3b4154c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e8a0 tx=0x7fb3a400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.542+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017070 con 0x7fb3b40c4500 2024-04-25T12:36:10.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.542+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a400c940 con 0x7fb3b40c4500 2024-04-25T12:36:10.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.542+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4009e60 con 0x7fb3b40c4500 2024-04-25T12:36:10.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.542+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b41e8970 con 0x7fb3b40c4500 2024-04-25T12:36:10.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.542+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b41ee6c0 con 0x7fb3b40c4500 2024-04-25T12:36:10.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.544+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb3b40b5880 con 0x7fb3b40c4500 2024-04-25T12:36:10.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.545+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb3a401a070 con 0x7fb3b40c4500 2024-04-25T12:36:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.546+0000 7fb3ab400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb394074500 0x7fb394076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.547+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 6619+0+0 (secure 0 0 0) 0x7fb3a40956f0 con 0x7fb3b40c4500 2024-04-25T12:36:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.547+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v51) ==== 54+0+16 (secure 0 0 0) 0x7fb3b40b5880 con 0x7fb3b40c4500 2024-04-25T12:36:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.547+0000 7fb3b2a00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb394074500 0x7fb394076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.547+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3b415e340 0x7fb3b41ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.547+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3b41eee00 con 0x7fb3b415e340 2024-04-25T12:36:10.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.548+0000 7fb3b2a00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb394074500 0x7fb394076a10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0006fd0 tx=0x7fb3a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.548+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3b415e340 0x7fb3b41ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.549+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3b415e340 0x7fb3b41ee8c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.550+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b41eee00 con 0x7fb3b415e340 2024-04-25T12:36:10.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.577+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b4076da0 con 0x7fb3b415e340 2024-04-25T12:36:10.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.581+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b4076da0 con 0x7fb3b415e340 2024-04-25T12:36:10.580 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:10.581 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:10.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.581+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3b41f18c0 0x7fb3b41f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.581+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb3b40cb8e0 con 0x7fb3b41f18c0 2024-04-25T12:36:10.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.582+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3b41f18c0 0x7fb3b41f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.583+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3b41f18c0 0x7fb3b41f1cd0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.583+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b40cb8e0 con 0x7fb3b41f18c0 2024-04-25T12:36:10.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.613+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b4159db0 con 0x7fb3b41f18c0 2024-04-25T12:36:10.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.616+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b4159db0 con 0x7fb3b41f18c0 2024-04-25T12:36:10.615 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:10.615 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:10.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.616+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3b4062fe0 0x7fb3b40633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.616+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb3b41f2720 con 0x7fb3b4062fe0 2024-04-25T12:36:10.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.616+0000 7fb3b2a00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3b4062fe0 0x7fb3b40633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.618+0000 7fb3b2a00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3b4062fe0 0x7fb3b40633f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.619+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b41f2720 con 0x7fb3b4062fe0 2024-04-25T12:36:10.626 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:10.627+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:10.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.641+0000 7fd8b7e00640 1 -- 172.21.15.123:0/715511589 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v52) ==== 101+0+0 (secure 0 0 0) 0x7fd8b805e7e0 con 0x7fd8c00c94f0 2024-04-25T12:36:10.640 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:36:10.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.647+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b40c96a0 con 0x7fb3b4062fe0 2024-04-25T12:36:10.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.647+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd8a00744b0 msgr2=0x7fd8a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.647+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd8a00744b0 0x7fd8a00769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00016e0 tx=0x7fd8b0009040 comp rx=0 tx=0).stop 2024-04-25T12:36:10.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.647+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8c00c94f0 msgr2=0x7fd8c0154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.647+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8c00c94f0 0x7fd8c0154af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800c750 tx=0x7fd8b800cc10 comp rx=0 tx=0).stop 2024-04-25T12:36:10.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 shutdown_connections 2024-04-25T12:36:10.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd8a00744b0 0x7fd8a00769c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd8c00c94f0 0x7fd8c0154af0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c00c2720 0x7fd8c00cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 --2- 172.21.15.123:0/715511589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8c0072d70 0x7fd8c01eee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 >> 172.21.15.123:0/715511589 conn(0x7fd8c00cd600 msgr2=0x7fd8c013f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:10.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 shutdown_connections 2024-04-25T12:36:10.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.649+0000 7fd8c5a00640 1 -- 172.21.15.123:0/715511589 wait complete. 2024-04-25T12:36:10.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.650+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b40c96a0 con 0x7fb3b4062fe0 2024-04-25T12:36:10.649 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.650+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3b406d680 0x7fb3b406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.650+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb3b4063e40 con 0x7fb3b406d680 2024-04-25T12:36:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.651+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3b406d680 0x7fb3b406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.652+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3b406d680 0x7fb3b406da90 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.653+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b4063e40 con 0x7fb3b406d680 2024-04-25T12:36:10.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.683+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b406dfd0 con 0x7fb3b406d680 2024-04-25T12:36:10.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.685+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b406dfd0 con 0x7fb3b406d680 2024-04-25T12:36:10.685 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:10.685 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:10.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.688+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3b402c9d0 0x7fb3b402cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.688+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb3b402d320 con 0x7fb3b402c9d0 2024-04-25T12:36:10.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.688+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3b402c9d0 0x7fb3b402cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.689+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3b402c9d0 0x7fb3b402cde0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.690+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b402d320 con 0x7fb3b402c9d0 2024-04-25T12:36:10.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.718+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b406e650 con 0x7fb3b402c9d0 2024-04-25T12:36:10.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.720+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b406e650 con 0x7fb3b402c9d0 2024-04-25T12:36:10.719 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.720+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fb3b402d8e0 0x7fb3b4204cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.720+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb3b4207a80 con 0x7fb3b402d8e0 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.721+0000 7fb3b2a00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fb3b402d8e0 0x7fb3b4204cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:10.720 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:10.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.721+0000 7fb3b2a00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fb3b402d8e0 0x7fb3b4204cb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.722+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b4207a80 con 0x7fb3b402d8e0 2024-04-25T12:36:10.727 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:10.728+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.748+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b42051f0 con 0x7fb3b402d8e0 2024-04-25T12:36:10.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.750+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b42051f0 con 0x7fb3b402d8e0 2024-04-25T12:36:10.749 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.750+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3b4205910 0x7fb3b4207d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.751+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb3b420ac90 con 0x7fb3b4205910 2024-04-25T12:36:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.751+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3b4205910 0x7fb3b4207d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.751+0000 7fb3b3e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3b4205910 0x7fb3b4207d30 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.752+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b420ac90 con 0x7fb3b4205910 2024-04-25T12:36:10.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.781+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b4000950 con 0x7fb3b4205910 2024-04-25T12:36:10.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.784+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b4000950 con 0x7fb3b4205910 2024-04-25T12:36:10.783 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:10.783 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:10.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.784+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3b4209f10 0x7fb3b420a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:10.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.784+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb3b420e060 con 0x7fb3b4209f10 2024-04-25T12:36:10.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.784+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3b4209f10 0x7fb3b420a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:10.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.785+0000 7fb3b3400640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3b4209f10 0x7fb3b420a340 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:10.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.786+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b420e060 con 0x7fb3b4209f10 2024-04-25T12:36:10.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.796+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f5f88076700 con 0x7f5f880c4240 2024-04-25T12:36:10.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.798+0000 7f5f84c00640 1 -- 172.21.15.123:0/829935043 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) ==== 81+0+37930 (secure 0 0 0) 0x7f5f7c063040 con 0x7f5f880c4240 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout: "epoch": 52, 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout: "fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout: "created": "2024-04-25T12:33:54.601604+0000", 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout: "modified": "2024-04-25T12:36:10.632684+0000", 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout: "last_up_change": "2024-04-25T12:35:59.575545+0000", 2024-04-25T12:36:10.798 INFO:teuthology.orchestra.run.smithi123.stdout: "last_in_change": "2024-04-25T12:33:58.290536+0000", 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_num": 5799936, 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_set": [ 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "pglog_hardlimit", 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "purged_snapdirs", 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_deletes", 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "sortbitwise" 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_version": 7, 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_snapshot": "", 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_max": 4, 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd": 8, 2024-04-25T12:36:10.799 INFO:teuthology.orchestra.run.smithi123.stdout: "require_min_compat_client": "luminous", 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "min_compat_client": "luminous", 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "require_osd_release": "squid", 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_crimson": false, 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "pools": [ 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 1, 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": ".mgr", 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:06.827674+0000", 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:36:10.800 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 1, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 1, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 1, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 1, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 1, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:10.801 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "17", 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:10.802 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:10.803 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_max": 32, 2024-04-25T12:36:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 1 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": {} 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 8, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 8, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 0.25, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:36:10.805 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 2, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_metadata", 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:11.030660+0000", 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:10.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "20", 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:10.808 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:10.809 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_bias": 4, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 16, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_priority": 5 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.810 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": "cephfs" 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 2.25, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 2.25, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2.25, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2.25, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:36:10.811 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 3, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data", 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:12.395253+0000", 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:10.812 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 64, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 64, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 64, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 64, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 64, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:10.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "21", 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:10.814 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.815 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.05000000074505806 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 1.5, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 1.5, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 1.5, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 1.5, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 4, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data_ec", 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:15.975923+0000", 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 5, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 3, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 4, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 3, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 1, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:10.818 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.819 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "25", 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:10.820 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 8192, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:10.822 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "osds": [ 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "2d9ee216-abfa-45db-a0ac-ef834981be6f", 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:10.823 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6816", 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6817", 2024-04-25T12:36:10.824 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6818", 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6819", 2024-04-25T12:36:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6822", 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6823", 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6820", 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6821", 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6817/1405705428", 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6819/1405705428", 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6823/1405705428", 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6821/1405705428", 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "a60a3a6f-376a-4fb2-baf1-e13fc615f6e5", 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.828 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6800", 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6801", 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6802", 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6803", 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6806", 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.831 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6807", 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6804", 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6805", 2024-04-25T12:36:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6801/1234958824", 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6803/1234958824", 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6807/1234958824", 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6805/1234958824", 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:36:10.833 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "cfb7cbbd-82cd-46fe-b89b-15775df1fe93", 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6808", 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6809", 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6810", 2024-04-25T12:36:10.835 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6811", 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6814", 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6815", 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6812", 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6813", 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6809/2222173849", 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6811/2222173849", 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6815/2222173849", 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6813/2222173849", 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9e8ccd97-ab68-4f03-bbb3-5820eccf4568", 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.839 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6824", 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6825", 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6826", 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6827", 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6830", 2024-04-25T12:36:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6831", 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6828", 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6829", 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6825/1491071423", 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6827/1491071423", 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6831/1491071423", 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6829/1491071423", 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "7d5bfd6e-173a-413d-abdb-df6d82c941d3", 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6802", 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6803", 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6804", 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6805", 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6808", 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6809", 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6806", 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6807", 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6803/3135982807", 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6805/3135982807", 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6809/3135982807", 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6807/3135982807", 2024-04-25T12:36:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9141fa91-c547-4711-910a-9ef123017fb5", 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 14, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 39, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 47, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 40, 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6826", 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6827", 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6828", 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6829", 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6832", 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6833", 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6830", 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6831", 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6827/4270711740", 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6829/4270711740", 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6833/4270711740", 2024-04-25T12:36:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6831/4270711740", 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "66fbc935-086e-4b03-aa03-96e0bc8b1d27", 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:10.854 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6810", 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6811", 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.855 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6812", 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6813", 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.856 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6816", 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6817", 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6814", 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6815", 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.858 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6811/3626282289", 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6813/3626282289", 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6817/3626282289", 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6815/3626282289", 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "de59a619-b84a-4623-ac89-447178815d65", 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:10.859 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6818", 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.860 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6819", 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6820", 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6821", 2024-04-25T12:36:10.861 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6824", 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6825", 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.862 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6822", 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6823", 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:10.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6819/3086604284", 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6821/3086604284", 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6825/3086604284", 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6823/3086604284", 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_xinfo": [ 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:36:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.805055+0000", 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.865 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.746266+0000", 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.706883+0000", 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.866 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.826277+0000", 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.867 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.402228+0000", 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.536018+0000", 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 40 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.868 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.375016+0000", 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:10.869 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.454835+0000", 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap": [], 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_items": [ 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "1.0", 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 5, 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 4 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "3.15", 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 0, 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 7 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "4.d", 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 3, 2024-04-25T12:36:10.871 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 0 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_primaries": [], 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_temp": [], 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_temp": [], 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklist": {}, 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "range_blocklist": {}, 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profiles": { 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_data_ec": { 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-device-class": "", 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-num-failure-domains": "0", 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-osds-per-failure-domain": "0", 2024-04-25T12:36:10.872 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-root": "default", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "jerasure-per-chunk-alignment": "false", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "2", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "w": "8" 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "default": { 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "1", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van" 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps_queue": [], 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "new_removed_snaps": [], 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "new_purged_snaps": [], 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_node_flags": {}, 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "device_class_flags": {}, 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode": { 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_enabled": false, 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_bucket_count": 0, 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "degraded_stretch_mode": 0, 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "recovering_stretch_mode": 0, 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_bucket": 0 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:10.874 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f680744e0 msgr2=0x7f5f680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f680744e0 0x7f5f680769f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f74034000 tx=0x7f5f7402e000 comp rx=0 tx=0).stop 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f880c4240 msgr2=0x7f5f8814ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f880c4240 0x7f5f8814ab90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00e870 tx=0x7f5f7c00ed30 comp rx=0 tx=0).stop 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 shutdown_connections 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5f680744e0 0x7f5f680769f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5f8815a730 0x7f5f88149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f880c90d0 0x7f5f88149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 --2- 172.21.15.123:0/829935043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f880c4240 0x7f5f8814ab90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 >> 172.21.15.123:0/829935043 conn(0x7f5f8806c750 msgr2=0x7f5f8806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 shutdown_connections 2024-04-25T12:36:10.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.802+0000 7f5f8fc00640 1 -- 172.21.15.123:0/829935043 wait complete. 2024-04-25T12:36:10.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.815+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb3b42082c0 con 0x7fb3b4209f10 2024-04-25T12:36:10.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.817+0000 7fb3ab400640 1 -- 172.21.15.123:0/3816592836 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb3b42082c0 con 0x7fb3b4209f10 2024-04-25T12:36:10.877 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3b415e340 msgr2=0x7fb3b41ee8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fb3b415e340 0x7fb3b41ee8c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3b41f18c0 msgr2=0x7fb3b41f1cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fb3b41f18c0 0x7fb3b41f1cd0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3b4062fe0 msgr2=0x7fb3b40633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fb3b4062fe0 0x7fb3b40633f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3b406d680 msgr2=0x7fb3b406da90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fb3b406d680 0x7fb3b406da90 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3b402c9d0 msgr2=0x7fb3b402cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fb3b402c9d0 0x7fb3b402cde0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fb3b402d8e0 msgr2=0x7fb3b4204cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fb3b402d8e0 0x7fb3b4204cb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3b4205910 msgr2=0x7fb3b4207d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fb3b4205910 0x7fb3b4207d30 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3b4209f10 msgr2=0x7fb3b420a340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.818+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb3b4209f10 0x7fb3b420a340 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b3e00640 1 -- 172.21.15.123:0/3816592836 reap_dead start 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b3e00640 1 -- 172.21.15.123:0/3816592836 reap_dead start 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b3e00640 1 -- 172.21.15.123:0/3816592836 reap_dead start 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b3e00640 1 -- 172.21.15.123:0/3816592836 reap_dead start 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b3e00640 1 -- 172.21.15.123:0/3816592836 reap_dead start 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb394074500 msgr2=0x7fb394076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb394074500 0x7fb394076a10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0006fd0 tx=0x7fb3a0008040 comp rx=0 tx=0).stop 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3b40c4500 msgr2=0x7fb3b4154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.819+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3b40c4500 0x7fb3b4154c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e8a0 tx=0x7fb3a400ed60 comp rx=0 tx=0).stop 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.820+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 shutdown_connections 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.820+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb394074500 0x7fb394076a10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.820+0000 7fb3b8e00640 1 --2- 172.21.15.123:0/3816592836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb3b40c4500 0x7fb3b4154c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:10.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.820+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 >> 172.21.15.123:0/3816592836 conn(0x7fb3b4068810 msgr2=0x7fb3b4150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:10.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.820+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 shutdown_connections 2024-04-25T12:36:10.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:10.820+0000 7fb3b8e00640 1 -- 172.21.15.123:0/3816592836 wait complete. 2024-04-25T12:36:10.881 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:10.877+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:10.882 INFO:tasks.thrashosds.thrasher:j is {'epoch': 52, 'fsid': '2239057a-dc25-4ee3-8341-8c80e403f296', 'created': '2024-04-25T12:33:54.601604+0000', 'modified': '2024-04-25T12:36:10.632684+0000', 'last_up_change': '2024-04-25T12:35:59.575545+0000', 'last_in_change': '2024-04-25T12:33:58.290536+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': '2d9ee216-abfa-45db-a0ac-ef834981be6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 1405705428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 1405705428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 1405705428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 1405705428}]}, 'public_addr': '172.21.15.123:6817/1405705428', 'cluster_addr': '172.21.15.123:6819/1405705428', 'heartbeat_back_addr': '172.21.15.123:6823/1405705428', 'heartbeat_front_addr': '172.21.15.123:6821/1405705428', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a60a3a6f-376a-4fb2-baf1-e13fc615f6e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 1234958824}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 1234958824}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 1234958824}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 1234958824}]}, 'public_addr': '172.21.15.123:6801/1234958824', 'cluster_addr': '172.21.15.123:6803/1234958824', 'heartbeat_back_addr': '172.21.15.123:6807/1234958824', 'heartbeat_front_addr': '172.21.15.123:6805/1234958824', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'cfb7cbbd-82cd-46fe-b89b-15775df1fe93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 2222173849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 2222173849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 2222173849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 2222173849}]}, 'public_addr': '172.21.15.123:6809/2222173849', 'cluster_addr': '172.21.15.123:6811/2222173849', 'heartbeat_back_addr': '172.21.15.123:6815/2222173849', 'heartbeat_front_addr': '172.21.15.123:6813/2222173849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9e8ccd97-ab68-4f03-bbb3-5820eccf4568', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 1491071423}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 1491071423}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 1491071423}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 1491071423}]}, 'public_addr': '172.21.15.123:6825/1491071423', 'cluster_addr': '172.21.15.123:6827/1491071423', 'heartbeat_back_addr': '172.21.15.123:6831/1491071423', 'heartbeat_front_addr': '172.21.15.123:6829/1491071423', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d5bfd6e-173a-413d-abdb-df6d82c941d3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6802', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6803', 'nonce': 3135982807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6804', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6805', 'nonce': 3135982807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6808', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6809', 'nonce': 3135982807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6806', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6807', 'nonce': 3135982807}]}, 'public_addr': '172.21.15.153:6803/3135982807', 'cluster_addr': '172.21.15.153:6805/3135982807', 'heartbeat_back_addr': '172.21.15.153:6809/3135982807', 'heartbeat_front_addr': '172.21.15.153:6807/3135982807', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9141fa91-c547-4711-910a-9ef123017fb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 39, 'up_from': 47, 'up_thru': 47, 'down_at': 40, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6826', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6827', 'nonce': 4270711740}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6828', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6829', 'nonce': 4270711740}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 4270711740}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6830', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6831', 'nonce': 4270711740}]}, 'public_addr': '172.21.15.153:6827/4270711740', 'cluster_addr': '172.21.15.153:6829/4270711740', 'heartbeat_back_addr': '172.21.15.153:6833/4270711740', 'heartbeat_front_addr': '172.21.15.153:6831/4270711740', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '66fbc935-086e-4b03-aa03-96e0bc8b1d27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6810', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6811', 'nonce': 3626282289}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6812', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6813', 'nonce': 3626282289}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6816', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6817', 'nonce': 3626282289}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6814', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6815', 'nonce': 3626282289}]}, 'public_addr': '172.21.15.153:6811/3626282289', 'cluster_addr': '172.21.15.153:6813/3626282289', 'heartbeat_back_addr': '172.21.15.153:6817/3626282289', 'heartbeat_front_addr': '172.21.15.153:6815/3626282289', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'de59a619-b84a-4623-ac89-447178815d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6818', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6819', 'nonce': 3086604284}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6820', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6821', 'nonce': 3086604284}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6824', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6825', 'nonce': 3086604284}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6822', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6823', 'nonce': 3086604284}]}, 'public_addr': '172.21.15.153:6819/3086604284', 'cluster_addr': '172.21.15.153:6821/3086604284', 'heartbeat_back_addr': '172.21.15.153:6825/3086604284', 'heartbeat_front_addr': '172.21.15.153:6823/3086604284', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.805055+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.746266+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.706883+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.826277+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.402228+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.536018+0000', 'dead_epoch': 40}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.375016+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.454835+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '1.0', 'mappings': [{'from': 5, 'to': 4}]}, {'pgid': '3.15', 'mappings': [{'from': 0, 'to': 7}]}, {'pgid': '4.d', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-25T12:36:10.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:36:10.977 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:10.978+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:11.077 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:11.078+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:11.114 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:11.114 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:11.114 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:11.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:11.178 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:11.179+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:11.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.191+0000 7fc8bcc00640 1 Processor -- start 2024-04-25T12:36:11.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.191+0000 7fc8bcc00640 1 -- start start 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.193+0000 7fc8bcc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc8b80cdc60 0x7fc8b80b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.193+0000 7fc8bcc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b80d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8bcc00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b815fdd0 con 0x7fc8b80d1ad0 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8bcc00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b81600b0 con 0x7fc8b80cdc60 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8bcc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b8160390 con 0x7fc8b815cf60 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8b6a00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc8b80d1ad0 0x7fc8b815c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:52562/0 (socket says 172.21.15.123:52562) 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8b6a00640 1 -- 172.21.15.123:0/1529050965 learned_addr learned my addr 172.21.15.123:0/1529050965 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:11.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8b7400640 1 --2- 172.21.15.123:0/1529050965 >> v2:172.21.15.153:3300/0 conn(0x7fc8b80cdc60 0x7fc8b80b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8b7e00640 1 --2- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b80d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8b6000640 1 -- 172.21.15.123:0/1529050965 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744926918 0 0) 0x7fc8b815fdd0 con 0x7fc8b80d1ad0 2024-04-25T12:36:11.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.194+0000 7fc8b6000640 1 -- 172.21.15.123:0/1529050965 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc89c003520 con 0x7fc8b80d1ad0 2024-04-25T12:36:11.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.195+0000 7fc8b7400640 1 -- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 msgr2=0x7fc8b80d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:11.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.195+0000 7fc8b7400640 1 --2- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b80d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.195+0000 7fc8b7400640 1 -- 172.21.15.123:0/1529050965 >> v1:172.21.15.153:6789/0 conn(0x7fc8b80d1ad0 legacy=0x7fc8b815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.195+0000 7fc8b7400640 1 -- 172.21.15.123:0/1529050965 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b8160670 con 0x7fc8b80cdc60 2024-04-25T12:36:11.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.196+0000 7fc8b7400640 1 --2- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80cdc60 0x7fc8b80b42c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80097a0 tx=0x7fc8a802a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f052d11458f1b613 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:11.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.197+0000 7fc8b6000640 1 -- 172.21.15.123:0/1529050965 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8037070 con 0x7fc8b80cdc60 2024-04-25T12:36:11.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.197+0000 7fc8b6000640 1 -- 172.21.15.123:0/1529050965 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a802ab80 con 0x7fc8b80cdc60 2024-04-25T12:36:11.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.197+0000 7fc8b6000640 1 -- 172.21.15.123:0/1529050965 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4001670 con 0x7fc8b80cdc60 2024-04-25T12:36:11.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 -- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80cdc60 msgr2=0x7fc8b80b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80cdc60 0x7fc8b80b42c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80097a0 tx=0x7fc8a802a0f0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 -- 172.21.15.123:0/1529050965 shutdown_connections 2024-04-25T12:36:11.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b80d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/1529050965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80cdc60 0x7fc8b80b42c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 -- 172.21.15.123:0/1529050965 >> 172.21.15.123:0/1529050965 conn(0x7fc8b808d840 msgr2=0x7fc8b808dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:11.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 -- 172.21.15.123:0/1529050965 shutdown_connections 2024-04-25T12:36:11.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.198+0000 7fc8bcc00640 1 -- 172.21.15.123:0/1529050965 wait complete. 2024-04-25T12:36:11.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.202+0000 7fc8bcc00640 1 Processor -- start 2024-04-25T12:36:11.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.202+0000 7fc8bcc00640 1 -- start start 2024-04-25T12:36:11.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.204+0000 7fc8bcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8b80cdc60 0x7fc8b81f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.204+0000 7fc8bcc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80d1ad0 0x7fc8b80d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.204+0000 7fc8bcc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b815eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.204+0000 7fc8bcc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b815bd30 con 0x7fc8b80cdc60 2024-04-25T12:36:11.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.204+0000 7fc8bcc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b815bbb0 con 0x7fc8b80d1ad0 2024-04-25T12:36:11.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.204+0000 7fc8bcc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b815beb0 con 0x7fc8b815cf60 2024-04-25T12:36:11.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80d1ad0 0x7fc8b80d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b815eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b815eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:51522/0 (socket says 172.21.15.123:51522) 2024-04-25T12:36:11.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 -- 172.21.15.123:0/3280967495 learned_addr learned my addr 172.21.15.123:0/3280967495 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:11.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80d1ad0 0x7fc8b80d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41646/0 (socket says 172.21.15.123:41646) 2024-04-25T12:36:11.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7400640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8b80cdc60 0x7fc8b81f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 -- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80d1ad0 msgr2=0x7fc8b80d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80d1ad0 0x7fc8b80d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 -- 172.21.15.123:0/3280967495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8b80cdc60 msgr2=0x7fc8b81f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8b80cdc60 0x7fc8b81f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.205+0000 7fc8b7e00640 1 -- 172.21.15.123:0/3280967495 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b81f2790 con 0x7fc8b815cf60 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.206+0000 7fc8b7e00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b815eaa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009cb0 tx=0x7fc8b000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.206+0000 7fc8af400640 1 -- 172.21.15.123:0/3280967495 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018020 con 0x7fc8b815cf60 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.206+0000 7fc8af400640 1 -- 172.21.15.123:0/3280967495 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b0014480 con 0x7fc8b815cf60 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.206+0000 7fc8af400640 1 -- 172.21.15.123:0/3280967495 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0014870 con 0x7fc8b815cf60 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.206+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b81e9890 con 0x7fc8b815cf60 2024-04-25T12:36:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.206+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b81f60c0 con 0x7fc8b815cf60 2024-04-25T12:36:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.208+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b8076da0 con 0x7fc8b815cf60 2024-04-25T12:36:11.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.210+0000 7fc8af400640 1 -- 172.21.15.123:0/3280967495 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc8b0005d10 con 0x7fc8b815cf60 2024-04-25T12:36:11.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.211+0000 7fc8af400640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8980744b0 0x7fc8980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.211+0000 7fc8af400640 1 -- 172.21.15.123:0/3280967495 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 6619+0+0 (secure 0 0 0) 0x7fc8b00961c0 con 0x7fc8b815cf60 2024-04-25T12:36:11.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.212+0000 7fc8b7400640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8980744b0 0x7fc8980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.212+0000 7fc8b7400640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8980744b0 0x7fc8980769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.215+0000 7fc8af400640 1 -- 172.21.15.123:0/3280967495 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8b0022030 con 0x7fc8b815cf60 2024-04-25T12:36:11.279 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:11.279+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:11.379 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:11.379+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 112.50560536899999, 2024-04-25T12:36:11.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012628781, 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951638+0000", 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951659+0000", 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1027999999999999e-05 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951691+0000", 2024-04-25T12:36:11.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2267000000000001e-05 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951710+0000", 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9063999999999999e-05 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952026+0000", 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000315719 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952367+0000", 2024-04-25T12:36:11.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034072600000000002 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955799+0000", 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034325969999999999 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962852+0000", 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070528889999999997 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964076+0000", 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012234119999999999 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964178+0000", 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102166 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964264+0000", 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6175999999999996e-05 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.463 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 112.505467538, 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012746265999999999, 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951774+0000", 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-06 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951782+0000", 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2090000000000005e-06 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951796+0000", 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3559e-05 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951808+0000", 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2183e-05 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956591+0000", 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047823550000000003 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956696+0000", 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105696 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956907+0000", 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000211165 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962907+0000", 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059995650000000001 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964283+0000", 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013758189999999999 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964381+0000", 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7977000000000001e-05 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964519+0000", 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138675 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:31 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 112.50429881399999, 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076541370000000001, 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.469 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952943+0000", 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:11.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952999+0000", 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5968999999999999e-05 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953032+0000", 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2598999999999997e-05 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953058+0000", 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6585999999999999e-05 2024-04-25T12:36:11.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953207+0000", 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014848300000000001 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953263+0000", 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6005999999999999e-05 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960434+0000", 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071716219999999999 2024-04-25T12:36:11.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960596+0000", 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016173899999999999 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 112.504116895, 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010940785, 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.473 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953125+0000", 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.22e-06 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953138+0000", 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2996000000000001e-05 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953158+0000", 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0173000000000002e-05 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953176+0000", 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7927e-05 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954316+0000", 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011402599999999999 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954375+0000", 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963968+0000", 2024-04-25T12:36:11.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095929269999999994 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964065+0000", 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6628999999999999e-05 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 112.504018071, 2024-04-25T12:36:11.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080627230000000008, 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953225+0000", 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3640000000000002e-06 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953259+0000", 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3955999999999998e-05 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953292+0000", 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3549e-05 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953320+0000", 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7892e-05 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953495+0000", 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017444800000000001 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953705+0000", 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021011000000000001 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:11.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953862+0000", 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015660399999999999 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959861+0000", 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059993959999999997 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961071+0000", 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012101900000000001 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961192+0000", 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121279 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961285+0000", 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2935000000000004e-05 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:11.482 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 112.503841334, 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010652488999999999, 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953401+0000", 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.513e-06 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953412+0000", 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209000000000001e-05 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953473+0000", 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0668999999999998e-05 2024-04-25T12:36:11.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953496+0000", 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3246000000000001e-05 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956356+0000", 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00286002 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956620+0000", 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026411099999999998 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.957005+0000", 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038473499999999998 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962757+0000", 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057517080000000003 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963927+0000", 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00117023 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964002+0000", 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5483999999999998e-05 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964052+0000", 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9564000000000002e-05 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:11.487 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 112.503712884, 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011827087999999999, 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953529+0000", 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953594+0000", 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4807999999999993e-05 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953617+0000", 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3220000000000001e-05 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953683+0000", 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5328999999999997e-05 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954191+0000", 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000508428 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040280099999999999 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954922+0000", 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032838300000000002 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960432+0000", 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055099989999999998 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965226+0000", 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047938700000000004 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965297+0000", 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0992000000000001e-05 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965355+0000", 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7695000000000002e-05 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.492 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:19 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.340809084, 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010949354999999999, 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116434+0000", 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116475+0000", 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1486000000000002e-05 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116505+0000", 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9709e-05 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116529+0000", 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3839999999999999e-05 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116651+0000", 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122013 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116685+0000", 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4513e-05 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127273+0000", 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010587653000000001 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127381+0000", 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108204 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:20 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.340653957000001, 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014061374, 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116588+0000", 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0499999999999999e-06 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116602+0000", 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3793e-05 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116621+0000", 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9545999999999999e-05 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116637+0000", 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5696e-05 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117243+0000", 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060668400000000002 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117280+0000", 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6619000000000003e-05 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130559+0000", 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013278570999999999 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130648+0000", 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9414999999999997e-05 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:27 2.9 e25/18 2:91bd965c:::501.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:11.501 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.340227307000006, 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013688128000000001, 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117015+0000", 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3200000000000001e-06 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117121+0000", 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106451 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117145+0000", 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.404e-05 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117210+0000", 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4314999999999997e-05 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120564+0000", 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033548050000000002 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120624+0000", 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9234999999999998e-05 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130654+0000", 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010030437999999999 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130702+0000", 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7524000000000001e-05 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:18 2.13 e25/18 2:ca4dd7f5:::611.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.339874007999995, 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010076254, 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117369+0000", 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.88e-06 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117385+0000", 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6670000000000001e-05 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117413+0000", 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7691999999999998e-05 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117435+0000", 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2416e-05 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117549+0000", 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113436 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1509e-05 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127347+0000", 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097767759999999992 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127443+0000", 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5874999999999999e-05 2024-04-25T12:36:11.509 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.338662334999995, 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012295627, 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118582+0000", 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000001e-06 2024-04-25T12:36:11.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118599+0000", 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7176e-05 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2505999999999997e-05 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118659+0000", 2024-04-25T12:36:11.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7559000000000002e-05 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120040+0000", 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001380956 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120305+0000", 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026497700000000001 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:11.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120707+0000", 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040189499999999999 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128035+0000", 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073284049999999996 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130711+0000", 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026752910000000002 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130787+0000", 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6384999999999995e-05 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130874+0000", 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7020000000000004e-05 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:11.515 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.338506335000005, 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012319366, 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118747+0000", 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1286e-05 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118772+0000", 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5150000000000001e-05 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118793+0000", 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0778999999999999e-05 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121911+0000", 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031179409999999999 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122331+0000", 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041955999999999997 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124441+0000", 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021107629999999999 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130205+0000", 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057636140000000002 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130888+0000", 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068289100000000003 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130961+0000", 2024-04-25T12:36:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3026000000000002e-05 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131054+0000", 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2855999999999993e-05 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:21 2.8 2:1470b2f8:::613.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.338417606999997, 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012409474, 2024-04-25T12:36:11.520 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118824+0000", 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118834+0000", 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118855+0000", 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999999e-05 2024-04-25T12:36:11.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118875+0000", 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9809999999999998e-05 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119612+0000", 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073729299999999998 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119912+0000", 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029931000000000001 2024-04-25T12:36:11.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120202+0000", 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000289907 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130342+0000", 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010140188 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131091+0000", 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074893900000000001 2024-04-25T12:36:11.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131166+0000", 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5272999999999996e-05 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131233+0000", 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6625000000000007e-05 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.525 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.338308859999998, 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012992702, 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118933+0000", 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3069999999999999e-06 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118943+0000", 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7200000000000001e-06 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118967+0000", 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3828999999999998e-05 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118986+0000", 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9057000000000001e-05 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124788+0000", 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00580271 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125092+0000", 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030394999999999999 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125479+0000", 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038660000000000002 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131299+0000", 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058200559999999997 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131823+0000", 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052382700000000004 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131888+0000", 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4740999999999996e-05 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131925+0000", 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6905e-05 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.530 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:5 2.7 e25/18 2:e455cf45:::60b.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 81.338185412000001, 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011712752, 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:11.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119057+0000", 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.296e-06 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119068+0000", 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.189e-05 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119091+0000", 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2802999999999998e-05 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119110+0000", 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8473000000000001e-05 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121085+0000", 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019750689999999999 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121149+0000", 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3979000000000005e-05 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130709+0000", 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009559817 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130768+0000", 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9425000000000001e-05 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:21 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.302396123999998, 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066623919999999996, 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154846+0000", 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.06e-06 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154856+0000", 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0713000000000001e-05 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154878+0000", 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1180000000000001e-05 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154896+0000", 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8343e-05 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155332+0000", 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043663000000000003 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155389+0000", 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6818999999999999e-05 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161381+0000", 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059914440000000003 2024-04-25T12:36:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161507+0000", 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126203 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:22 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.302122353000001, 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064678330000000001, 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155120+0000", 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155131+0000", 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1793e-05 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155155+0000", 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3306e-05 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155174+0000", 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9383000000000001e-05 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157101+0000", 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019271570000000001 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157161+0000", 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9945000000000003e-05 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161512+0000", 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043506389999999999 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161586+0000", 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4250000000000002e-05 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:26 2.11 2:893cd530:::mds2_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.301185380000007, 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064127289999999998, 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156057+0000", 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8330000000000001e-06 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156090+0000", 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2719000000000003e-05 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156116+0000", 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6531999999999999e-05 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156140+0000", 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3453999999999999e-05 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156350+0000", 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021060999999999999 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156467+0000", 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116823 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156594+0000", 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012698900000000001 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161255+0000", 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046609570000000003 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162360+0000", 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001105167 2024-04-25T12:36:11.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162416+0000", 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6076999999999999e-05 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162468+0000", 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1567999999999998e-05 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.301034731000001, 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066007829999999998, 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:11.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156208+0000", 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5570000000000001e-06 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156216+0000", 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0900000000000005e-06 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156237+0000", 2024-04-25T12:36:11.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1452999999999999e-05 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156255+0000", 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8091999999999999e-05 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156841+0000", 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00058573200000000005 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156983+0000", 2024-04-25T12:36:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000141676 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162807+0000", 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058241830000000001 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:11.552 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:11.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.458+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8b815c8f0 con 0x7fc8980744b0 2024-04-25T12:36:11.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.467+0000 7fc8af400640 1 -- 172.21.15.123:0/3280967495 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333010 (secure 0 0 0) 0x7fc8b815c8f0 con 0x7fc8980744b0 2024-04-25T12:36:11.556 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:36:11.558 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8980744b0 msgr2=0x7fc8980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8980744b0 0x7fc8980769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802e000 comp rx=0 tx=0).stop 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 msgr2=0x7fc8b815eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b815eaa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009cb0 tx=0x7fc8b000c490 comp rx=0 tx=0).stop 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 shutdown_connections 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc8980744b0 0x7fc8980769c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8b815cf60 0x7fc8b815eaa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b80d1ad0 0x7fc8b80d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 --2- 172.21.15.123:0/3280967495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8b80cdc60 0x7fc8b81f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 >> 172.21.15.123:0/3280967495 conn(0x7fc8b808d840 msgr2=0x7fc8b8157690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 shutdown_connections 2024-04-25T12:36:11.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.473+0000 7fc8bcc00640 1 -- 172.21.15.123:0/3280967495 wait complete. 2024-04-25T12:36:11.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:11.562 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-25T12:36:10.822794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601885,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":242,"num_write_kb":1682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222072,"kb_used_data":4696,"kb_used_omap":20,"kb_used_meta":217259,"kb_avail":754752648,"statfs":{"total":773094113280,"available":772866711552,"internally_reserved":0,"allocated":4808704,"data_stored":3418106,"data_compressed":66734,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":20525,"internal_metadata":222474195},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.202388"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":80,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973922+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:36:04.973922+0000","last_peered":"2024-04-25T12:36:04.973922+0000","last_clean":"2024-04-25T12:36:04.973922+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:36:04.973922+0000","last_undegraded":"2024-04-25T12:36:04.973922+0000","last_fullsized":"2024-04-25T12:36:04.973922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973374+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:36:04.973374+0000","last_peered":"2024-04-25T12:36:04.973374+0000","last_clean":"2024-04-25T12:36:04.973374+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:36:04.973374+0000","last_undegraded":"2024-04-25T12:36:04.973374+0000","last_fullsized":"2024-04-25T12:36:04.973374+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":64,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974460+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:36:04.974460+0000","last_peered":"2024-04-25T12:36:04.974460+0000","last_clean":"2024-04-25T12:36:04.974460+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:36:04.974460+0000","last_undegraded":"2024-04-25T12:36:04.974460+0000","last_fullsized":"2024-04-25T12:36:04.974460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973185+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:36:04.973185+0000","last_peered":"2024-04-25T12:36:04.973185+0000","last_clean":"2024-04-25T12:36:04.973185+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:36:04.973185+0000","last_undegraded":"2024-04-25T12:36:04.973185+0000","last_fullsized":"2024-04-25T12:36:04.973185+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973755+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:36:04.973755+0000","last_peered":"2024-04-25T12:36:04.973755+0000","last_clean":"2024-04-25T12:36:04.973755+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:36:04.973755+0000","last_undegraded":"2024-04-25T12:36:04.973755+0000","last_fullsized":"2024-04-25T12:36:04.973755+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.626984+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:36:04.626984+0000","last_peered":"2024-04-25T12:36:04.626984+0000","last_clean":"2024-04-25T12:36:04.626984+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:36:04.626984+0000","last_undegraded":"2024-04-25T12:36:04.626984+0000","last_fullsized":"2024-04-25T12:36:04.626984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.624755+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:36:04.624755+0000","last_peered":"2024-04-25T12:36:04.624755+0000","last_clean":"2024-04-25T12:36:04.624755+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:36:04.624755+0000","last_undegraded":"2024-04-25T12:36:04.624755+0000","last_fullsized":"2024-04-25T12:36:04.624755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.977162+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:36:04.977162+0000","last_peered":"2024-04-25T12:36:04.977162+0000","last_clean":"2024-04-25T12:36:04.977162+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:36:04.977162+0000","last_undegraded":"2024-04-25T12:36:04.977162+0000","last_fullsized":"2024-04-25T12:36:04.977162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627254+0000","last_change":"2024-04-25T12:36:00.627223+0000","last_active":"2024-04-25T12:36:04.627254+0000","last_peered":"2024-04-25T12:36:04.627254+0000","last_clean":"2024-04-25T12:36:04.627254+0000","last_became_active":"2024-04-25T12:36:00.626675+0000","last_became_peered":"2024-04-25T12:36:00.626675+0000","last_unstale":"2024-04-25T12:36:04.627254+0000","last_undegraded":"2024-04-25T12:36:04.627254+0000","last_fullsized":"2024-04-25T12:36:04.627254+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:11.204775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":72,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.629879+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:36:04.629879+0000","last_peered":"2024-04-25T12:36:04.629879+0000","last_clean":"2024-04-25T12:36:04.629879+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:36:04.629879+0000","last_undegraded":"2024-04-25T12:36:04.629879+0000","last_fullsized":"2024-04-25T12:36:04.629879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.640855+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:36:07.640855+0000","last_peered":"2024-04-25T12:36:07.640855+0000","last_clean":"2024-04-25T12:36:07.640855+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:36:07.640855+0000","last_undegraded":"2024-04-25T12:36:07.640855+0000","last_fullsized":"2024-04-25T12:36:07.640855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.626833+0000","last_change":"2024-04-25T12:36:00.623046+0000","last_active":"2024-04-25T12:36:04.626833+0000","last_peered":"2024-04-25T12:36:04.626833+0000","last_clean":"2024-04-25T12:36:04.626833+0000","last_became_active":"2024-04-25T12:36:00.622702+0000","last_became_peered":"2024-04-25T12:36:00.622702+0000","last_unstale":"2024-04-25T12:36:04.626833+0000","last_undegraded":"2024-04-25T12:36:04.626833+0000","last_fullsized":"2024-04-25T12:36:04.626833+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.639508+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:36:07.639508+0000","last_peered":"2024-04-25T12:36:07.639508+0000","last_clean":"2024-04-25T12:36:07.639508+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:36:07.639508+0000","last_undegraded":"2024-04-25T12:36:07.639508+0000","last_fullsized":"2024-04-25T12:36:07.639508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.640287+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:36:07.640287+0000","last_peered":"2024-04-25T12:36:07.640287+0000","last_clean":"2024-04-25T12:36:07.640287+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:36:07.640287+0000","last_undegraded":"2024-04-25T12:36:07.640287+0000","last_fullsized":"2024-04-25T12:36:07.640287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.624328+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:36:04.624328+0000","last_peered":"2024-04-25T12:36:04.624328+0000","last_clean":"2024-04-25T12:36:04.624328+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:36:04.624328+0000","last_undegraded":"2024-04-25T12:36:04.624328+0000","last_fullsized":"2024-04-25T12:36:04.624328+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975163+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:36:04.975163+0000","last_peered":"2024-04-25T12:36:04.975163+0000","last_clean":"2024-04-25T12:36:04.975163+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:36:04.975163+0000","last_undegraded":"2024-04-25T12:36:04.975163+0000","last_fullsized":"2024-04-25T12:36:04.975163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974982+0000","last_change":"2024-04-25T12:36:00.626426+0000","last_active":"2024-04-25T12:36:04.974982+0000","last_peered":"2024-04-25T12:36:04.974982+0000","last_clean":"2024-04-25T12:36:04.974982+0000","last_became_active":"2024-04-25T12:36:00.625818+0000","last_became_peered":"2024-04-25T12:36:00.625818+0000","last_unstale":"2024-04-25T12:36:04.974982+0000","last_undegraded":"2024-04-25T12:36:04.974982+0000","last_fullsized":"2024-04-25T12:36:04.974982+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:06.117031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.628243+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:36:04.628243+0000","last_peered":"2024-04-25T12:36:04.628243+0000","last_clean":"2024-04-25T12:36:04.628243+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:36:04.628243+0000","last_undegraded":"2024-04-25T12:36:04.628243+0000","last_fullsized":"2024-04-25T12:36:04.628243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972554+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:36:04.972554+0000","last_peered":"2024-04-25T12:36:04.972554+0000","last_clean":"2024-04-25T12:36:04.972554+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:36:04.972554+0000","last_undegraded":"2024-04-25T12:36:04.972554+0000","last_fullsized":"2024-04-25T12:36:04.972554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972216+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:36:04.972216+0000","last_peered":"2024-04-25T12:36:04.972216+0000","last_clean":"2024-04-25T12:36:04.972216+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:36:04.972216+0000","last_undegraded":"2024-04-25T12:36:04.972216+0000","last_fullsized":"2024-04-25T12:36:04.972216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":64,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975493+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:36:04.975493+0000","last_peered":"2024-04-25T12:36:04.975493+0000","last_clean":"2024-04-25T12:36:04.975493+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:36:04.975493+0000","last_undegraded":"2024-04-25T12:36:04.975493+0000","last_fullsized":"2024-04-25T12:36:04.975493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.639110+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:36:07.639110+0000","last_peered":"2024-04-25T12:36:07.639110+0000","last_clean":"2024-04-25T12:36:07.639110+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:36:07.639110+0000","last_undegraded":"2024-04-25T12:36:07.639110+0000","last_fullsized":"2024-04-25T12:36:07.639110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":72,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.623913+0000","last_change":"2024-04-25T12:34:12.921643+0000","last_active":"2024-04-25T12:36:04.623913+0000","last_peered":"2024-04-25T12:36:04.623913+0000","last_clean":"2024-04-25T12:36:04.623913+0000","last_became_active":"2024-04-25T12:34:12.921108+0000","last_became_peered":"2024-04-25T12:34:12.921108+0000","last_unstale":"2024-04-25T12:36:04.623913+0000","last_undegraded":"2024-04-25T12:36:04.623913+0000","last_fullsized":"2024-04-25T12:36:04.623913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:08.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.628214+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:36:04.628214+0000","last_peered":"2024-04-25T12:36:04.628214+0000","last_clean":"2024-04-25T12:36:04.628214+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:36:04.628214+0000","last_undegraded":"2024-04-25T12:36:04.628214+0000","last_fullsized":"2024-04-25T12:36:04.628214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973686+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:36:04.973686+0000","last_peered":"2024-04-25T12:36:04.973686+0000","last_clean":"2024-04-25T12:36:04.973686+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:36:04.973686+0000","last_undegraded":"2024-04-25T12:36:04.973686+0000","last_fullsized":"2024-04-25T12:36:04.973686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.641981+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:36:07.641981+0000","last_peered":"2024-04-25T12:36:07.641981+0000","last_clean":"2024-04-25T12:36:07.641981+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:36:07.641981+0000","last_undegraded":"2024-04-25T12:36:07.641981+0000","last_fullsized":"2024-04-25T12:36:07.641981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627565+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:36:04.627565+0000","last_peered":"2024-04-25T12:36:04.627565+0000","last_clean":"2024-04-25T12:36:04.627565+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:36:04.627565+0000","last_undegraded":"2024-04-25T12:36:04.627565+0000","last_fullsized":"2024-04-25T12:36:04.627565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973886+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:36:04.973886+0000","last_peered":"2024-04-25T12:36:04.973886+0000","last_clean":"2024-04-25T12:36:04.973886+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:36:04.973886+0000","last_undegraded":"2024-04-25T12:36:04.973886+0000","last_fullsized":"2024-04-25T12:36:04.973886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"43'6","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972223+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:36:04.972223+0000","last_peered":"2024-04-25T12:36:04.972223+0000","last_clean":"2024-04-25T12:36:04.972223+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:36:04.972223+0000","last_undegraded":"2024-04-25T12:36:04.972223+0000","last_fullsized":"2024-04-25T12:36:04.972223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972539+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:36:04.972539+0000","last_peered":"2024-04-25T12:36:04.972539+0000","last_clean":"2024-04-25T12:36:04.972539+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:36:04.972539+0000","last_undegraded":"2024-04-25T12:36:04.972539+0000","last_fullsized":"2024-04-25T12:36:04.972539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":75,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975308+0000","last_change":"2024-04-25T12:36:00.626931+0000","last_active":"2024-04-25T12:36:04.975308+0000","last_peered":"2024-04-25T12:36:04.975308+0000","last_clean":"2024-04-25T12:36:04.975308+0000","last_became_active":"2024-04-25T12:36:00.626533+0000","last_became_peered":"2024-04-25T12:36:00.626533+0000","last_unstale":"2024-04-25T12:36:04.975308+0000","last_undegraded":"2024-04-25T12:36:04.975308+0000","last_fullsized":"2024-04-25T12:36:04.975308+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":72,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.641762+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:36:07.641762+0000","last_peered":"2024-04-25T12:36:07.641762+0000","last_clean":"2024-04-25T12:36:07.641762+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:36:07.641762+0000","last_undegraded":"2024-04-25T12:36:07.641762+0000","last_fullsized":"2024-04-25T12:36:07.641762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974787+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:36:04.974787+0000","last_peered":"2024-04-25T12:36:04.974787+0000","last_clean":"2024-04-25T12:36:04.974787+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:36:04.974787+0000","last_undegraded":"2024-04-25T12:36:04.974787+0000","last_fullsized":"2024-04-25T12:36:04.974787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627969+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:36:04.627969+0000","last_peered":"2024-04-25T12:36:04.627969+0000","last_clean":"2024-04-25T12:36:04.627969+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:36:04.627969+0000","last_undegraded":"2024-04-25T12:36:04.627969+0000","last_fullsized":"2024-04-25T12:36:04.627969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973187+0000","last_change":"2024-04-25T12:36:00.622303+0000","last_active":"2024-04-25T12:36:04.973187+0000","last_peered":"2024-04-25T12:36:04.973187+0000","last_clean":"2024-04-25T12:36:04.973187+0000","last_became_active":"2024-04-25T12:36:00.621920+0000","last_became_peered":"2024-04-25T12:36:00.621920+0000","last_unstale":"2024-04-25T12:36:04.973187+0000","last_undegraded":"2024-04-25T12:36:04.973187+0000","last_fullsized":"2024-04-25T12:36:04.973187+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:25.840527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974752+0000","last_change":"2024-04-25T12:36:00.631090+0000","last_active":"2024-04-25T12:36:04.974752+0000","last_peered":"2024-04-25T12:36:04.974752+0000","last_clean":"2024-04-25T12:36:04.974752+0000","last_became_active":"2024-04-25T12:36:00.630484+0000","last_became_peered":"2024-04-25T12:36:00.630484+0000","last_unstale":"2024-04-25T12:36:04.974752+0000","last_undegraded":"2024-04-25T12:36:04.974752+0000","last_fullsized":"2024-04-25T12:36:04.974752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.738025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.625644+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:36:04.625644+0000","last_peered":"2024-04-25T12:36:04.625644+0000","last_clean":"2024-04-25T12:36:04.625644+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:36:04.625644+0000","last_undegraded":"2024-04-25T12:36:04.625644+0000","last_fullsized":"2024-04-25T12:36:04.625644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":72,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.639884+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:36:07.639884+0000","last_peered":"2024-04-25T12:36:07.639884+0000","last_clean":"2024-04-25T12:36:07.639884+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:36:07.639884+0000","last_undegraded":"2024-04-25T12:36:07.639884+0000","last_fullsized":"2024-04-25T12:36:07.639884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975111+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:36:04.975111+0000","last_peered":"2024-04-25T12:36:04.975111+0000","last_clean":"2024-04-25T12:36:04.975111+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:36:04.975111+0000","last_undegraded":"2024-04-25T12:36:04.975111+0000","last_fullsized":"2024-04-25T12:36:04.975111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972904+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:36:04.972904+0000","last_peered":"2024-04-25T12:36:04.972904+0000","last_clean":"2024-04-25T12:36:04.972904+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:36:04.972904+0000","last_undegraded":"2024-04-25T12:36:04.972904+0000","last_fullsized":"2024-04-25T12:36:04.972904+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":82,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974683+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:36:04.974683+0000","last_peered":"2024-04-25T12:36:04.974683+0000","last_clean":"2024-04-25T12:36:04.974683+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:36:04.974683+0000","last_undegraded":"2024-04-25T12:36:04.974683+0000","last_fullsized":"2024-04-25T12:36:04.974683+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975423+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:36:04.975423+0000","last_peered":"2024-04-25T12:36:04.975423+0000","last_clean":"2024-04-25T12:36:04.975423+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:36:04.975423+0000","last_undegraded":"2024-04-25T12:36:04.975423+0000","last_fullsized":"2024-04-25T12:36:04.975423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974714+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:36:04.974714+0000","last_peered":"2024-04-25T12:36:04.974714+0000","last_clean":"2024-04-25T12:36:04.974714+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:36:04.974714+0000","last_undegraded":"2024-04-25T12:36:04.974714+0000","last_fullsized":"2024-04-25T12:36:04.974714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.638562+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:36:07.638562+0000","last_peered":"2024-04-25T12:36:07.638562+0000","last_clean":"2024-04-25T12:36:07.638562+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:36:07.638562+0000","last_undegraded":"2024-04-25T12:36:07.638562+0000","last_fullsized":"2024-04-25T12:36:07.638562+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973638+0000","last_change":"2024-04-25T12:36:00.633695+0000","last_active":"2024-04-25T12:36:04.973638+0000","last_peered":"2024-04-25T12:36:04.973638+0000","last_clean":"2024-04-25T12:36:04.973638+0000","last_became_active":"2024-04-25T12:36:00.633193+0000","last_became_peered":"2024-04-25T12:36:00.633193+0000","last_unstale":"2024-04-25T12:36:04.973638+0000","last_undegraded":"2024-04-25T12:36:04.973638+0000","last_fullsized":"2024-04-25T12:36:04.973638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:10.743500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973135+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:36:04.973135+0000","last_peered":"2024-04-25T12:36:04.973135+0000","last_clean":"2024-04-25T12:36:04.973135+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:36:04.973135+0000","last_undegraded":"2024-04-25T12:36:04.973135+0000","last_fullsized":"2024-04-25T12:36:04.973135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":74,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.640389+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:36:07.640389+0000","last_peered":"2024-04-25T12:36:07.640389+0000","last_clean":"2024-04-25T12:36:07.640389+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:36:07.640389+0000","last_undegraded":"2024-04-25T12:36:07.640389+0000","last_fullsized":"2024-04-25T12:36:07.640389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":178,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975019+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:36:04.975019+0000","last_peered":"2024-04-25T12:36:04.975019+0000","last_clean":"2024-04-25T12:36:04.975019+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:36:04.975019+0000","last_undegraded":"2024-04-25T12:36:04.975019+0000","last_fullsized":"2024-04-25T12:36:04.975019+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":65,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.641548+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:36:07.641548+0000","last_peered":"2024-04-25T12:36:07.641548+0000","last_clean":"2024-04-25T12:36:07.641548+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:36:07.641548+0000","last_undegraded":"2024-04-25T12:36:07.641548+0000","last_fullsized":"2024-04-25T12:36:07.641548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.640948+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:36:07.640948+0000","last_peered":"2024-04-25T12:36:07.640948+0000","last_clean":"2024-04-25T12:36:07.640948+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:36:07.640948+0000","last_undegraded":"2024-04-25T12:36:07.640948+0000","last_fullsized":"2024-04-25T12:36:07.640948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":73,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974169+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:36:04.974169+0000","last_peered":"2024-04-25T12:36:04.974169+0000","last_clean":"2024-04-25T12:36:04.974169+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:36:04.974169+0000","last_undegraded":"2024-04-25T12:36:04.974169+0000","last_fullsized":"2024-04-25T12:36:04.974169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627014+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:36:04.627014+0000","last_peered":"2024-04-25T12:36:04.627014+0000","last_clean":"2024-04-25T12:36:04.627014+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:36:04.627014+0000","last_undegraded":"2024-04-25T12:36:04.627014+0000","last_fullsized":"2024-04-25T12:36:04.627014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.629382+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:36:04.629382+0000","last_peered":"2024-04-25T12:36:04.629382+0000","last_clean":"2024-04-25T12:36:04.629382+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:36:04.629382+0000","last_undegraded":"2024-04-25T12:36:04.629382+0000","last_fullsized":"2024-04-25T12:36:04.629382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973973+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:36:04.973973+0000","last_peered":"2024-04-25T12:36:04.973973+0000","last_clean":"2024-04-25T12:36:04.973973+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:36:04.973973+0000","last_undegraded":"2024-04-25T12:36:04.973973+0000","last_fullsized":"2024-04-25T12:36:04.973973+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973074+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:36:04.973074+0000","last_peered":"2024-04-25T12:36:04.973074+0000","last_clean":"2024-04-25T12:36:04.973074+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:36:04.973074+0000","last_undegraded":"2024-04-25T12:36:04.973074+0000","last_fullsized":"2024-04-25T12:36:04.973074+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.630335+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:36:04.630335+0000","last_peered":"2024-04-25T12:36:04.630335+0000","last_clean":"2024-04-25T12:36:04.630335+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:36:04.630335+0000","last_undegraded":"2024-04-25T12:36:04.630335+0000","last_fullsized":"2024-04-25T12:36:04.630335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":80,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973785+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:36:04.973785+0000","last_peered":"2024-04-25T12:36:04.973785+0000","last_clean":"2024-04-25T12:36:04.973785+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:36:04.973785+0000","last_undegraded":"2024-04-25T12:36:04.973785+0000","last_fullsized":"2024-04-25T12:36:04.973785+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":65,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.638074+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:36:07.638074+0000","last_peered":"2024-04-25T12:36:07.638074+0000","last_clean":"2024-04-25T12:36:07.638074+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:36:07.638074+0000","last_undegraded":"2024-04-25T12:36:07.638074+0000","last_fullsized":"2024-04-25T12:36:07.638074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.628519+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:36:04.628519+0000","last_peered":"2024-04-25T12:36:04.628519+0000","last_clean":"2024-04-25T12:36:04.628519+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:36:04.628519+0000","last_undegraded":"2024-04-25T12:36:04.628519+0000","last_fullsized":"2024-04-25T12:36:04.628519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.630640+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:36:04.630640+0000","last_peered":"2024-04-25T12:36:04.630640+0000","last_clean":"2024-04-25T12:36:04.630640+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:36:04.630640+0000","last_undegraded":"2024-04-25T12:36:04.630640+0000","last_fullsized":"2024-04-25T12:36:04.630640+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975495+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:36:04.975495+0000","last_peered":"2024-04-25T12:36:04.975495+0000","last_clean":"2024-04-25T12:36:04.975495+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:36:04.975495+0000","last_undegraded":"2024-04-25T12:36:04.975495+0000","last_fullsized":"2024-04-25T12:36:04.975495+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.639017+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:36:07.639017+0000","last_peered":"2024-04-25T12:36:07.639017+0000","last_clean":"2024-04-25T12:36:07.639017+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:36:07.639017+0000","last_undegraded":"2024-04-25T12:36:07.639017+0000","last_fullsized":"2024-04-25T12:36:07.639017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":86,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972637+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:36:04.972637+0000","last_peered":"2024-04-25T12:36:04.972637+0000","last_clean":"2024-04-25T12:36:04.972637+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:36:04.972637+0000","last_undegraded":"2024-04-25T12:36:04.972637+0000","last_fullsized":"2024-04-25T12:36:04.972637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973477+0000","last_change":"2024-04-25T12:36:00.633538+0000","last_active":"2024-04-25T12:36:04.973477+0000","last_peered":"2024-04-25T12:36:04.973477+0000","last_clean":"2024-04-25T12:36:04.973477+0000","last_became_active":"2024-04-25T12:36:00.632957+0000","last_became_peered":"2024-04-25T12:36:00.632957+0000","last_unstale":"2024-04-25T12:36:04.973477+0000","last_undegraded":"2024-04-25T12:36:04.973477+0000","last_fullsized":"2024-04-25T12:36:04.973477+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:41.919677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":76,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974757+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:36:04.974757+0000","last_peered":"2024-04-25T12:36:04.974757+0000","last_clean":"2024-04-25T12:36:04.974757+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:36:04.974757+0000","last_undegraded":"2024-04-25T12:36:04.974757+0000","last_fullsized":"2024-04-25T12:36:04.974757+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":73,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627507+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:36:04.627507+0000","last_peered":"2024-04-25T12:36:04.627507+0000","last_clean":"2024-04-25T12:36:04.627507+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:36:04.627507+0000","last_undegraded":"2024-04-25T12:36:04.627507+0000","last_fullsized":"2024-04-25T12:36:04.627507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973781+0000","last_change":"2024-04-25T12:36:00.631519+0000","last_active":"2024-04-25T12:36:04.973781+0000","last_peered":"2024-04-25T12:36:04.973781+0000","last_clean":"2024-04-25T12:36:04.973781+0000","last_became_active":"2024-04-25T12:36:00.631077+0000","last_became_peered":"2024-04-25T12:36:00.631077+0000","last_unstale":"2024-04-25T12:36:04.973781+0000","last_undegraded":"2024-04-25T12:36:04.973781+0000","last_fullsized":"2024-04-25T12:36:04.973781+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:45.009426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.640127+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:36:07.640127+0000","last_peered":"2024-04-25T12:36:07.640127+0000","last_clean":"2024-04-25T12:36:07.640127+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:36:07.640127+0000","last_undegraded":"2024-04-25T12:36:07.640127+0000","last_fullsized":"2024-04-25T12:36:07.640127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":74,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.639781+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:36:07.639781+0000","last_peered":"2024-04-25T12:36:07.639781+0000","last_clean":"2024-04-25T12:36:07.639781+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:36:07.639781+0000","last_undegraded":"2024-04-25T12:36:07.639781+0000","last_fullsized":"2024-04-25T12:36:07.639781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975101+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:36:04.975101+0000","last_peered":"2024-04-25T12:36:04.975101+0000","last_clean":"2024-04-25T12:36:04.975101+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:36:04.975101+0000","last_undegraded":"2024-04-25T12:36:04.975101+0000","last_fullsized":"2024-04-25T12:36:04.975101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974884+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:36:04.974884+0000","last_peered":"2024-04-25T12:36:04.974884+0000","last_clean":"2024-04-25T12:36:04.974884+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:36:04.974884+0000","last_undegraded":"2024-04-25T12:36:04.974884+0000","last_fullsized":"2024-04-25T12:36:04.974884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974346+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:36:04.974346+0000","last_peered":"2024-04-25T12:36:04.974346+0000","last_clean":"2024-04-25T12:36:04.974346+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:36:04.974346+0000","last_undegraded":"2024-04-25T12:36:04.974346+0000","last_fullsized":"2024-04-25T12:36:04.974346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.640754+0000","last_change":"2024-04-25T12:36:00.636248+0000","last_active":"2024-04-25T12:36:07.640754+0000","last_peered":"2024-04-25T12:36:07.640754+0000","last_clean":"2024-04-25T12:36:07.640754+0000","last_became_active":"2024-04-25T12:36:00.635874+0000","last_became_peered":"2024-04-25T12:36:00.635874+0000","last_unstale":"2024-04-25T12:36:07.640754+0000","last_undegraded":"2024-04-25T12:36:07.640754+0000","last_fullsized":"2024-04-25T12:36:07.640754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.244233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":56,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975446+0000","last_change":"2024-04-25T12:35:00.239744+0000","last_active":"2024-04-25T12:36:04.975446+0000","last_peered":"2024-04-25T12:36:04.975446+0000","last_clean":"2024-04-25T12:36:04.975446+0000","last_became_active":"2024-04-25T12:35:00.239414+0000","last_became_peered":"2024-04-25T12:35:00.239414+0000","last_unstale":"2024-04-25T12:36:04.975446+0000","last_undegraded":"2024-04-25T12:36:04.975446+0000","last_fullsized":"2024-04-25T12:36:04.975446+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:17.015606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"42'17","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627803+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:36:04.627803+0000","last_peered":"2024-04-25T12:36:04.627803+0000","last_clean":"2024-04-25T12:36:04.627803+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:36:04.627803+0000","last_undegraded":"2024-04-25T12:36:04.627803+0000","last_fullsized":"2024-04-25T12:36:04.627803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7907,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972820+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:36:04.972820+0000","last_peered":"2024-04-25T12:36:04.972820+0000","last_clean":"2024-04-25T12:36:04.972820+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:36:04.972820+0000","last_undegraded":"2024-04-25T12:36:04.972820+0000","last_fullsized":"2024-04-25T12:36:04.972820+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.640576+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:36:07.640576+0000","last_peered":"2024-04-25T12:36:07.640576+0000","last_clean":"2024-04-25T12:36:07.640576+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:36:07.640576+0000","last_undegraded":"2024-04-25T12:36:07.640576+0000","last_fullsized":"2024-04-25T12:36:07.640576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":75,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975558+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:36:04.975558+0000","last_peered":"2024-04-25T12:36:04.975558+0000","last_clean":"2024-04-25T12:36:04.975558+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:36:04.975558+0000","last_undegraded":"2024-04-25T12:36:04.975558+0000","last_fullsized":"2024-04-25T12:36:04.975558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627369+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:36:04.627369+0000","last_peered":"2024-04-25T12:36:04.627369+0000","last_clean":"2024-04-25T12:36:04.627369+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:36:04.627369+0000","last_undegraded":"2024-04-25T12:36:04.627369+0000","last_fullsized":"2024-04-25T12:36:04.627369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973530+0000","last_change":"2024-04-25T12:36:00.636634+0000","last_active":"2024-04-25T12:36:04.973530+0000","last_peered":"2024-04-25T12:36:04.973530+0000","last_clean":"2024-04-25T12:36:04.973530+0000","last_became_active":"2024-04-25T12:36:00.636371+0000","last_became_peered":"2024-04-25T12:36:00.636371+0000","last_unstale":"2024-04-25T12:36:04.973530+0000","last_undegraded":"2024-04-25T12:36:04.973530+0000","last_fullsized":"2024-04-25T12:36:04.973530+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:36.185819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":72,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.629979+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:36:04.629979+0000","last_peered":"2024-04-25T12:36:04.629979+0000","last_clean":"2024-04-25T12:36:04.629979+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:36:04.629979+0000","last_undegraded":"2024-04-25T12:36:04.629979+0000","last_fullsized":"2024-04-25T12:36:04.629979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":64,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974386+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:36:04.974386+0000","last_peered":"2024-04-25T12:36:04.974386+0000","last_clean":"2024-04-25T12:36:04.974386+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:36:04.974386+0000","last_undegraded":"2024-04-25T12:36:04.974386+0000","last_fullsized":"2024-04-25T12:36:04.974386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.628413+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:36:04.628413+0000","last_peered":"2024-04-25T12:36:04.628413+0000","last_clean":"2024-04-25T12:36:04.628413+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:36:04.628413+0000","last_undegraded":"2024-04-25T12:36:04.628413+0000","last_fullsized":"2024-04-25T12:36:04.628413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.979234+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:36:04.979234+0000","last_peered":"2024-04-25T12:36:04.979234+0000","last_clean":"2024-04-25T12:36:04.979234+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:36:04.979234+0000","last_undegraded":"2024-04-25T12:36:04.979234+0000","last_fullsized":"2024-04-25T12:36:04.979234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":63,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972502+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:36:04.972502+0000","last_peered":"2024-04-25T12:36:04.972502+0000","last_clean":"2024-04-25T12:36:04.972502+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:36:04.972502+0000","last_undegraded":"2024-04-25T12:36:04.972502+0000","last_fullsized":"2024-04-25T12:36:04.972502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973975+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:36:04.973975+0000","last_peered":"2024-04-25T12:36:04.973975+0000","last_clean":"2024-04-25T12:36:04.973975+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:36:04.973975+0000","last_undegraded":"2024-04-25T12:36:04.973975+0000","last_fullsized":"2024-04-25T12:36:04.973975+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972539+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:36:04.972539+0000","last_peered":"2024-04-25T12:36:04.972539+0000","last_clean":"2024-04-25T12:36:04.972539+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:36:04.972539+0000","last_undegraded":"2024-04-25T12:36:04.972539+0000","last_fullsized":"2024-04-25T12:36:04.972539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":72,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.638443+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:36:07.638443+0000","last_peered":"2024-04-25T12:36:07.638443+0000","last_clean":"2024-04-25T12:36:07.638443+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:36:07.638443+0000","last_undegraded":"2024-04-25T12:36:07.638443+0000","last_fullsized":"2024-04-25T12:36:07.638443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.641834+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:36:07.641834+0000","last_peered":"2024-04-25T12:36:07.641834+0000","last_clean":"2024-04-25T12:36:07.641834+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:36:07.641834+0000","last_undegraded":"2024-04-25T12:36:07.641834+0000","last_fullsized":"2024-04-25T12:36:07.641834+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973031+0000","last_change":"2024-04-25T12:36:00.623718+0000","last_active":"2024-04-25T12:36:04.973031+0000","last_peered":"2024-04-25T12:36:04.973031+0000","last_clean":"2024-04-25T12:36:04.973031+0000","last_became_active":"2024-04-25T12:36:00.622664+0000","last_became_peered":"2024-04-25T12:36:00.622664+0000","last_unstale":"2024-04-25T12:36:04.973031+0000","last_undegraded":"2024-04-25T12:36:04.973031+0000","last_fullsized":"2024-04-25T12:36:04.973031+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.061061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":72,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974138+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:36:04.974138+0000","last_peered":"2024-04-25T12:36:04.974138+0000","last_clean":"2024-04-25T12:36:04.974138+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:36:04.974138+0000","last_undegraded":"2024-04-25T12:36:04.974138+0000","last_fullsized":"2024-04-25T12:36:04.974138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":74,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.639675+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:36:07.639675+0000","last_peered":"2024-04-25T12:36:07.639675+0000","last_clean":"2024-04-25T12:36:07.639675+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:36:07.639675+0000","last_undegraded":"2024-04-25T12:36:07.639675+0000","last_fullsized":"2024-04-25T12:36:07.639675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973179+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:36:04.973179+0000","last_peered":"2024-04-25T12:36:04.973179+0000","last_clean":"2024-04-25T12:36:04.973179+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:36:04.973179+0000","last_undegraded":"2024-04-25T12:36:04.973179+0000","last_fullsized":"2024-04-25T12:36:04.973179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973534+0000","last_change":"2024-04-25T12:36:00.624696+0000","last_active":"2024-04-25T12:36:04.973534+0000","last_peered":"2024-04-25T12:36:04.973534+0000","last_clean":"2024-04-25T12:36:04.973534+0000","last_became_active":"2024-04-25T12:36:00.624209+0000","last_became_peered":"2024-04-25T12:36:00.624209+0000","last_unstale":"2024-04-25T12:36:04.973534+0000","last_undegraded":"2024-04-25T12:36:04.973534+0000","last_fullsized":"2024-04-25T12:36:04.973534+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:14.608733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":43,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973025+0000","last_change":"2024-04-25T12:36:00.623752+0000","last_active":"2024-04-25T12:36:04.973025+0000","last_peered":"2024-04-25T12:36:04.973025+0000","last_clean":"2024-04-25T12:36:04.973025+0000","last_became_active":"2024-04-25T12:36:00.623008+0000","last_became_peered":"2024-04-25T12:36:00.623008+0000","last_unstale":"2024-04-25T12:36:04.973025+0000","last_undegraded":"2024-04-25T12:36:04.973025+0000","last_fullsized":"2024-04-25T12:36:04.973025+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:12.341618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":76,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.976235+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:36:04.976235+0000","last_peered":"2024-04-25T12:36:04.976235+0000","last_clean":"2024-04-25T12:36:04.976235+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:36:04.976235+0000","last_undegraded":"2024-04-25T12:36:04.976235+0000","last_fullsized":"2024-04-25T12:36:04.976235+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":75,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975321+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:36:04.975321+0000","last_peered":"2024-04-25T12:36:04.975321+0000","last_clean":"2024-04-25T12:36:04.975321+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:36:04.975321+0000","last_undegraded":"2024-04-25T12:36:04.975321+0000","last_fullsized":"2024-04-25T12:36:04.975321+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627244+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:36:04.627244+0000","last_peered":"2024-04-25T12:36:04.627244+0000","last_clean":"2024-04-25T12:36:04.627244+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:36:04.627244+0000","last_undegraded":"2024-04-25T12:36:04.627244+0000","last_fullsized":"2024-04-25T12:36:04.627244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974610+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:36:04.974610+0000","last_peered":"2024-04-25T12:36:04.974610+0000","last_clean":"2024-04-25T12:36:04.974610+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:36:04.974610+0000","last_undegraded":"2024-04-25T12:36:04.974610+0000","last_fullsized":"2024-04-25T12:36:04.974610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.639408+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:36:07.639408+0000","last_peered":"2024-04-25T12:36:07.639408+0000","last_clean":"2024-04-25T12:36:07.639408+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:36:07.639408+0000","last_undegraded":"2024-04-25T12:36:07.639408+0000","last_fullsized":"2024-04-25T12:36:07.639408+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973163+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:36:04.973163+0000","last_peered":"2024-04-25T12:36:04.973163+0000","last_clean":"2024-04-25T12:36:04.973163+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:36:04.973163+0000","last_undegraded":"2024-04-25T12:36:04.973163+0000","last_fullsized":"2024-04-25T12:36:04.973163+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975267+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:36:04.975267+0000","last_peered":"2024-04-25T12:36:04.975267+0000","last_clean":"2024-04-25T12:36:04.975267+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:36:04.975267+0000","last_undegraded":"2024-04-25T12:36:04.975267+0000","last_fullsized":"2024-04-25T12:36:04.975267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975305+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:36:04.975305+0000","last_peered":"2024-04-25T12:36:04.975305+0000","last_clean":"2024-04-25T12:36:04.975305+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:36:04.975305+0000","last_undegraded":"2024-04-25T12:36:04.975305+0000","last_fullsized":"2024-04-25T12:36:04.975305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.628317+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:36:04.628317+0000","last_peered":"2024-04-25T12:36:04.628317+0000","last_clean":"2024-04-25T12:36:04.628317+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:36:04.628317+0000","last_undegraded":"2024-04-25T12:36:04.628317+0000","last_fullsized":"2024-04-25T12:36:04.628317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973674+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:36:04.973674+0000","last_peered":"2024-04-25T12:36:04.973674+0000","last_clean":"2024-04-25T12:36:04.973674+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:36:04.973674+0000","last_undegraded":"2024-04-25T12:36:04.973674+0000","last_fullsized":"2024-04-25T12:36:04.973674+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.638346+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:36:07.638346+0000","last_peered":"2024-04-25T12:36:07.638346+0000","last_clean":"2024-04-25T12:36:07.638346+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:36:07.638346+0000","last_undegraded":"2024-04-25T12:36:07.638346+0000","last_fullsized":"2024-04-25T12:36:07.638346+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.972960+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:36:04.972960+0000","last_peered":"2024-04-25T12:36:04.972960+0000","last_clean":"2024-04-25T12:36:04.972960+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:36:04.972960+0000","last_undegraded":"2024-04-25T12:36:04.972960+0000","last_fullsized":"2024-04-25T12:36:04.972960+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.976120+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:36:04.976120+0000","last_peered":"2024-04-25T12:36:04.976120+0000","last_clean":"2024-04-25T12:36:04.976120+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:36:04.976120+0000","last_undegraded":"2024-04-25T12:36:04.976120+0000","last_fullsized":"2024-04-25T12:36:04.976120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973087+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:36:04.973087+0000","last_peered":"2024-04-25T12:36:04.973087+0000","last_clean":"2024-04-25T12:36:04.973087+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:36:04.973087+0000","last_undegraded":"2024-04-25T12:36:04.973087+0000","last_fullsized":"2024-04-25T12:36:04.973087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.629125+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:36:04.629125+0000","last_peered":"2024-04-25T12:36:04.629125+0000","last_clean":"2024-04-25T12:36:04.629125+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:36:04.629125+0000","last_undegraded":"2024-04-25T12:36:04.629125+0000","last_fullsized":"2024-04-25T12:36:04.629125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.641665+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:36:07.641665+0000","last_peered":"2024-04-25T12:36:07.641665+0000","last_clean":"2024-04-25T12:36:07.641665+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:36:07.641665+0000","last_undegraded":"2024-04-25T12:36:07.641665+0000","last_fullsized":"2024-04-25T12:36:07.641665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.975624+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:36:04.975624+0000","last_peered":"2024-04-25T12:36:04.975624+0000","last_clean":"2024-04-25T12:36:04.975624+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:36:04.975624+0000","last_undegraded":"2024-04-25T12:36:04.975624+0000","last_fullsized":"2024-04-25T12:36:04.975624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.641970+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:36:07.641970+0000","last_peered":"2024-04-25T12:36:07.641970+0000","last_clean":"2024-04-25T12:36:07.641970+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:36:07.641970+0000","last_undegraded":"2024-04-25T12:36:07.641970+0000","last_fullsized":"2024-04-25T12:36:07.641970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T12:36:07.638251+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:36:07.638251+0000","last_peered":"2024-04-25T12:36:07.638251+0000","last_clean":"2024-04-25T12:36:07.638251+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:36:07.638251+0000","last_undegraded":"2024-04-25T12:36:07.638251+0000","last_fullsized":"2024-04-25T12:36:07.638251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973016+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:36:04.973016+0000","last_peered":"2024-04-25T12:36:04.973016+0000","last_clean":"2024-04-25T12:36:04.973016+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:36:04.973016+0000","last_undegraded":"2024-04-25T12:36:04.973016+0000","last_fullsized":"2024-04-25T12:36:04.973016+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974799+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:36:04.974799+0000","last_peered":"2024-04-25T12:36:04.974799+0000","last_clean":"2024-04-25T12:36:04.974799+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:36:04.974799+0000","last_undegraded":"2024-04-25T12:36:04.974799+0000","last_fullsized":"2024-04-25T12:36:04.974799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.974914+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:36:04.974914+0000","last_peered":"2024-04-25T12:36:04.974914+0000","last_clean":"2024-04-25T12:36:04.974914+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:36:04.974914+0000","last_undegraded":"2024-04-25T12:36:04.974914+0000","last_fullsized":"2024-04-25T12:36:04.974914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.628860+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:36:04.628860+0000","last_peered":"2024-04-25T12:36:04.628860+0000","last_clean":"2024-04-25T12:36:04.628860+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:36:04.628860+0000","last_undegraded":"2024-04-25T12:36:04.628860+0000","last_fullsized":"2024-04-25T12:36:04.628860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.624640+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:36:04.624640+0000","last_peered":"2024-04-25T12:36:04.624640+0000","last_clean":"2024-04-25T12:36:04.624640+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:36:04.624640+0000","last_undegraded":"2024-04-25T12:36:04.624640+0000","last_fullsized":"2024-04-25T12:36:04.624640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.629821+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:36:04.629821+0000","last_peered":"2024-04-25T12:36:04.629821+0000","last_clean":"2024-04-25T12:36:04.629821+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:36:04.629821+0000","last_undegraded":"2024-04-25T12:36:04.629821+0000","last_fullsized":"2024-04-25T12:36:04.629821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":69,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.978441+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:36:04.978441+0000","last_peered":"2024-04-25T12:36:04.978441+0000","last_clean":"2024-04-25T12:36:04.978441+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:36:04.978441+0000","last_undegraded":"2024-04-25T12:36:04.978441+0000","last_fullsized":"2024-04-25T12:36:04.978441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973484+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:36:04.973484+0000","last_peered":"2024-04-25T12:36:04.973484+0000","last_clean":"2024-04-25T12:36:04.973484+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:36:04.973484+0000","last_undegraded":"2024-04-25T12:36:04.973484+0000","last_fullsized":"2024-04-25T12:36:04.973484+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627187+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:36:04.627187+0000","last_peered":"2024-04-25T12:36:04.627187+0000","last_clean":"2024-04-25T12:36:04.627187+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:36:04.627187+0000","last_undegraded":"2024-04-25T12:36:04.627187+0000","last_fullsized":"2024-04-25T12:36:04.627187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627136+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:36:04.627136+0000","last_peered":"2024-04-25T12:36:04.627136+0000","last_clean":"2024-04-25T12:36:04.627136+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:36:04.627136+0000","last_undegraded":"2024-04-25T12:36:04.627136+0000","last_fullsized":"2024-04-25T12:36:04.627136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":75,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.625919+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:36:04.625919+0000","last_peered":"2024-04-25T12:36:04.625919+0000","last_clean":"2024-04-25T12:36:04.625919+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:36:04.625919+0000","last_undegraded":"2024-04-25T12:36:04.625919+0000","last_fullsized":"2024-04-25T12:36:04.625919+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.627769+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:36:04.627769+0000","last_peered":"2024-04-25T12:36:04.627769+0000","last_clean":"2024-04-25T12:36:04.627769+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:36:04.627769+0000","last_undegraded":"2024-04-25T12:36:04.627769+0000","last_fullsized":"2024-04-25T12:36:04.627769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973914+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:36:04.973914+0000","last_peered":"2024-04-25T12:36:04.973914+0000","last_clean":"2024-04-25T12:36:04.973914+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:36:04.973914+0000","last_undegraded":"2024-04-25T12:36:04.973914+0000","last_fullsized":"2024-04-25T12:36:04.973914+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.973264+0000","last_change":"2024-04-25T12:36:00.630044+0000","last_active":"2024-04-25T12:36:04.973264+0000","last_peered":"2024-04-25T12:36:04.973264+0000","last_clean":"2024-04-25T12:36:04.973264+0000","last_became_active":"2024-04-25T12:36:00.629698+0000","last_became_peered":"2024-04-25T12:36:00.629698+0000","last_unstale":"2024-04-25T12:36:04.973264+0000","last_undegraded":"2024-04-25T12:36:04.973264+0000","last_fullsized":"2024-04-25T12:36:04.973264+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:20.463389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":68,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T12:36:04.630317+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:36:04.630317+0000","last_peered":"2024-04-25T12:36:04.630317+0000","last_clean":"2024-04-25T12:36:04.630317+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:36:04.630317+0000","last_undegraded":"2024-04-25T12:36:04.630317+0000","last_fullsized":"2024-04-25T12:36:04.630317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11517,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":23034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":278018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":278108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":589824,"data_stored":277700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":277996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462915,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28808,"kb_used_data":572,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94343032,"statfs":{"total":96636764160,"available":96607264768,"internally_reserved":0,"allocated":585728,"data_stored":277596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":876635,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":284733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":867320,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:36:11.567 INFO:tasks.thrashosds.thrasher:Setting 2.8 to [3, 1] 2024-04-25T12:36:11.567 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.8', '3', '1'] 2024-04-25T12:36:11.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.8 3 1 2024-04-25T12:36:11.599 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:11.599+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:11.659 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:11.660+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:11.761 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:11.761+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:11.862 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:11.863+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:11.923 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:11.923 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:11.923 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:11.923 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:11.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.943+0000 7f7abd800640 1 Processor -- start 2024-04-25T12:36:11.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.944+0000 7f7abd800640 1 -- start start 2024-04-25T12:36:11.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.945+0000 7f7abd800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ab80b2a10 0x7f7ab815d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.945+0000 7f7abd800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab8073c10 con 0x7f7ab8073460 2024-04-25T12:36:11.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.945+0000 7f7abd800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab8074100 con 0x7f7ab80ca060 2024-04-25T12:36:11.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.945+0000 7f7abd800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab815d7b0 con 0x7f7ab80b2a10 2024-04-25T12:36:11.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.946+0000 7f7ab6a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7ab8073460 0x7f7ab80c3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:37054/0 (socket says 172.21.15.123:37054) 2024-04-25T12:36:11.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.946+0000 7f7ab6a00640 1 -- 172.21.15.123:0/653158928 learned_addr learned my addr 172.21.15.123:0/653158928 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:11.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.946+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742185519 0 0) 0x7f7ab8073c10 con 0x7f7ab8073460 2024-04-25T12:36:11.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.946+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a9c003520 con 0x7f7ab8073460 2024-04-25T12:36:11.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.946+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476229958 0 0) 0x7f7ab8074100 con 0x7f7ab80ca060 2024-04-25T12:36:11.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ab8073c10 con 0x7f7ab80ca060 2024-04-25T12:36:11.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3182730007 0 0) 0x7f7ab8073c10 con 0x7f7ab80ca060 2024-04-25T12:36:11.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 >> v2:172.21.15.123:3300/0 conn(0x7f7ab80b2a10 msgr2=0x7f7ab815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:11.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab6000640 1 --2- 172.21.15.123:0/653158928 >> v2:172.21.15.123:3300/0 conn(0x7f7ab80b2a10 0x7f7ab815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 --2- 172.21.15.123:0/653158928 >> v2:172.21.15.123:3300/0 conn(0x7f7ab80b2a10 0x7f7ab815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 >> v1:172.21.15.123:6789/0 conn(0x7f7ab8073460 legacy=0x7f7ab80c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab8070a20 con 0x7f7ab80ca060 2024-04-25T12:36:11.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f7ab00019f0 con 0x7f7ab80ca060 2024-04-25T12:36:11.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 >> v1:172.21.15.153:6789/0 conn(0x7f7ab80ca060 legacy=0x7f7ab80ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a9c004ba0 0x7f7a9c004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0037a0 0x7f7a9c003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a9c004210 0x7f7a9c0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab815d7b0 con 0x7f7a9c0037a0 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c014a10 con 0x7f7a9c004210 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.947+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9c014c10 con 0x7f7a9c004ba0 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.948+0000 7f7ab6000640 1 --2- 172.21.15.123:0/653158928 >> v2:172.21.15.123:3300/0 conn(0x7f7ab80b2a10 0x7f7ab815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.948+0000 7f7ab7400640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a9c004210 0x7f7a9c0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.948+0000 7f7ab6a00640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a9c004ba0 0x7f7a9c004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.948+0000 7f7ab7400640 1 -- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a9c004ba0 msgr2=0x7f7a9c004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:11.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.948+0000 7f7ab7400640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a9c004ba0 0x7f7a9c004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.948+0000 7f7ab7400640 1 -- 172.21.15.123:0/653158928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0037a0 msgr2=0x7f7a9c003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:36:11.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.949+0000 7f7ab6000640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0037a0 0x7f7a9c003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.949+0000 7f7ab7400640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0037a0 0x7f7a9c003bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.949+0000 7f7ab7400640 1 -- 172.21.15.123:0/653158928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab8070a20 con 0x7f7a9c004210 2024-04-25T12:36:11.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.950+0000 7f7ab7400640 1 -- 172.21.15.123:0/653158928 reap_dead start 2024-04-25T12:36:11.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.950+0000 7f7ab7400640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a9c004210 0x7f7a9c0141d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c590 tx=0x7f7a9c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e62ca4cd6144214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:11.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.951+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000c160 con 0x7f7a9c004210 2024-04-25T12:36:11.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.951+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab002c7b0 con 0x7f7a9c004210 2024-04-25T12:36:11.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.951+0000 7f7ab5600640 1 -- 172.21.15.123:0/653158928 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab002cba0 con 0x7f7a9c004210 2024-04-25T12:36:11.953 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:11.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.952+0000 7f7abd800640 1 -- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a9c004210 msgr2=0x7f7a9c0141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.952+0000 7f7abd800640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a9c004210 0x7f7a9c0141d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c590 tx=0x7f7a9c005830 comp rx=0 tx=0).stop 2024-04-25T12:36:11.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.952+0000 7f7abd800640 1 -- 172.21.15.123:0/653158928 shutdown_connections 2024-04-25T12:36:11.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.952+0000 7f7abd800640 1 --2- 172.21.15.123:0/653158928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a9c004210 0x7f7a9c0141d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.952+0000 7f7abd800640 1 -- 172.21.15.123:0/653158928 >> 172.21.15.123:0/653158928 conn(0x7f7ab806f440 msgr2=0x7f7ab806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:11.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.952+0000 7f7abd800640 1 -- 172.21.15.123:0/653158928 shutdown_connections 2024-04-25T12:36:11.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.952+0000 7f7abd800640 1 -- 172.21.15.123:0/653158928 wait complete. 2024-04-25T12:36:11.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.955+0000 7f7abd800640 1 Processor -- start 2024-04-25T12:36:11.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.956+0000 7f7abd800640 1 -- start start 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7abd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c004210 0x7f7ab81ee390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7abd800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab80cd5a0 0x7f7ab814a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7abd800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ab80cd9b0 0x7f7ab814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7abd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8071310 con 0x7f7a9c004210 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7abd800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8156fb0 con 0x7f7ab80cd5a0 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7abd800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab8073930 con 0x7f7ab80cd9b0 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7ab6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c004210 0x7f7ab81ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7ab6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c004210 0x7f7ab81ee390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53870/0 (socket says 172.21.15.123:53870) 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7ab6a00640 1 -- 172.21.15.123:0/4132497205 learned_addr learned my addr 172.21.15.123:0/4132497205 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7ab6000640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab80cd5a0 0x7f7ab814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.958+0000 7f7ab7400640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ab80cd9b0 0x7f7ab814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.959+0000 7f7ab6a00640 1 -- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ab80cd9b0 msgr2=0x7f7ab814a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.959+0000 7f7ab6a00640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ab80cd9b0 0x7f7ab814a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.959+0000 7f7ab6a00640 1 -- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab80cd5a0 msgr2=0x7f7ab814a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.959+0000 7f7ab6a00640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab80cd5a0 0x7f7ab814a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.959+0000 7f7ab6a00640 1 -- 172.21.15.123:0/4132497205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab81f2150 con 0x7f7a9c004210 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.959+0000 7f7ab6a00640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c004210 0x7f7ab81ee390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8004f10 tx=0x7f7aa8007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.960+0000 7f7aaea00640 1 -- 172.21.15.123:0/4132497205 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800f730 con 0x7f7a9c004210 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.960+0000 7f7aaea00640 1 -- 172.21.15.123:0/4132497205 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa8018070 con 0x7f7a9c004210 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.960+0000 7f7aaea00640 1 -- 172.21.15.123:0/4132497205 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800faf0 con 0x7f7a9c004210 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.960+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab81ee8d0 con 0x7f7a9c004210 2024-04-25T12:36:11.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.960+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab80cc960 con 0x7f7a9c004210 2024-04-25T12:36:11.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.962+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab8076700 con 0x7f7a9c004210 2024-04-25T12:36:11.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.963+0000 7f7aaea00640 1 -- 172.21.15.123:0/4132497205 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7aa800fc90 con 0x7f7a9c004210 2024-04-25T12:36:11.964 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:11.964+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:11.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.965+0000 7f7aaea00640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a980744b0 0x7f7a980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:11.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.965+0000 7f7aaea00640 1 -- 172.21.15.123:0/4132497205 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6619+0+0 (secure 0 0 0) 0x7f7aa80952d0 con 0x7f7a9c004210 2024-04-25T12:36:11.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.965+0000 7f7ab6000640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a980744b0 0x7f7a980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:11.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.966+0000 7f7ab6000640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a980744b0 0x7f7a980769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ab814cf30 tx=0x7f7aa4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:11.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:11.973+0000 7f7aaea00640 1 -- 172.21.15.123:0/4132497205 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ab81f2150 con 0x7f7a9c004210 2024-04-25T12:36:12.065 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:12.065+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:12.165 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:12.165+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:12.213 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:12.213 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:12.213 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:12.232 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:12.275 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:12.275+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:12.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.309+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "2.8", "id": [3, 1]} v 0) -- 0x7f7ab80c9b90 con 0x7f7a9c004210 2024-04-25T12:36:12.367 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:12.368+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:12.467 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:12.469+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:12.568 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:12.569+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:12.588 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 124.562310993, 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53759151699999996, 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024334+0000", 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7179999999999999e-06 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024345+0000", 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0608000000000001e-05 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024373+0000", 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7966e-05 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024392+0000", 2024-04-25T12:36:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024490+0000", 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7823000000000002e-05 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for peered", 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024520+0000", 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0056000000000002e-05 2024-04-25T12:36:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.559770+0000", 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53525036000000004 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560001+0000", 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023139100000000001 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560330+0000", 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032824500000000001 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561700+0000", 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013704730000000001 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561742+0000", 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1826000000000002e-05 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561855+0000", 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011287599999999999 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561924+0000", 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9484000000000001e-05 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.999387774, 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095393000000000006, 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587257+0000", 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9100000000000003e-07 2024-04-25T12:36:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587490+0000", 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000233186 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587502+0000", 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2499e-05 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587512+0000", 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7699999999999996e-06 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.591891+0000", 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043783770000000001 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592036+0000", 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014543999999999999 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592306+0000", 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026964399999999998 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595185+0000", 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028791730000000001 2024-04-25T12:36:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596722+0000", 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001537131 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596762+0000", 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0491000000000002e-05 2024-04-25T12:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596795+0000", 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2798e-05 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.999114855, 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012300247, 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 24 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587530+0000", 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3799999999999996e-07 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587688+0000", 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015835999999999999 2024-04-25T12:36:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587699+0000", 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0727000000000001e-05 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587710+0000", 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0987e-05 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593399+0000", 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005688935 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593685+0000", 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028662199999999999 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593893+0000", 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020758500000000001 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595599+0000", 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017058209999999999 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599668+0000", 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040696439999999999 2024-04-25T12:36:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599769+0000", 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100752 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599829+0000", 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0075999999999999e-05 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.998892797, 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012351434, 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 25 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587752+0000", 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587758+0000", 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2199999999999997e-06 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587770+0000", 2024-04-25T12:36:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2129e-05 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587778+0000", 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594340+0000", 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065618270000000001 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594415+0000", 2024-04-25T12:36:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4535000000000004e-05 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594510+0000", 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5100999999999996e-05 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595607+0000", 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010968239999999999 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599926+0000", 2024-04-25T12:36:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043194469999999997 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600027+0000", 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100961 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600102+0000", 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5339000000000005e-05 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.793943607, 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091591999999999993, 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792701+0000", 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.17e-06 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792911+0000", 2024-04-25T12:36:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000210163 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792932+0000", 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0472e-05 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792952+0000", 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0553000000000001e-05 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799157+0000", 2024-04-25T12:36:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062046030000000004 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799246+0000", 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8505999999999996e-05 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799389+0000", 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014319100000000001 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801232+0000", 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018427599999999999 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801753+0000", 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052153699999999998 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801814+0000", 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1014000000000002e-05 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801859+0000", 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5231000000000001e-05 2024-04-25T12:36:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.793505259, 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092910930000000003, 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 52 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793140+0000", 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-06 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793151+0000", 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0966e-05 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793168+0000", 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793184+0000", 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6200000000000001e-05 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801271+0000", 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080863189999999998 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801360+0000", 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9248999999999999e-05 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801515+0000", 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000154883 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802197+0000", 2024-04-25T12:36:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068243299999999998 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802339+0000", 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014159 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802393+0000", 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4215000000000002e-05 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802430+0000", 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6434999999999999e-05 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.33550198899999, 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012798129, 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 70 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251143+0000", 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2729999999999999e-06 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251153+0000", 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8439999999999993e-06 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251172+0000", 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8906000000000002e-05 2024-04-25T12:36:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251183+0000", 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1653e-05 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257332+0000", 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061486459999999998 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257490+0000", 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000157947 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257724+0000", 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023408200000000001 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259695+0000", 2024-04-25T12:36:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001971169 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263731+0000", 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035683 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263842+0000", 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110985 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263940+0000", 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7941000000000003e-05 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.33544508, 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013164844, 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 71 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251200+0000", 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2999999999999999e-07 2024-04-25T12:36:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251206+0000", 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9399999999999999e-06 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251215+0000", 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251223+0000", 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2260000000000005e-06 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259955+0000", 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087315629999999995 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260077+0000", 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122263 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260310+0000", 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232702 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263699+0000", 2024-04-25T12:36:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033888080000000001 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264218+0000", 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051973199999999996 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264304+0000", 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5433999999999996e-05 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264363+0000", 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9743e-05 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.33540347100001, 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017184050999999999, 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 72 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251241+0000", 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.23e-07 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251477+0000", 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023584499999999999 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251504+0000", 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7633000000000001e-05 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251516+0000", 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.164e-05 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260993+0000", 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094771119999999993 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261122+0000", 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012831400000000001 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261319+0000", 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019727199999999999 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264078+0000", 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027589120000000001 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268258+0000", 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041806569999999999 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268357+0000", 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8721000000000006e-05 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268424+0000", 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7222000000000001e-05 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.33510852800001, 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017105597, 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 73 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251536+0000", 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4300000000000001e-07 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251543+0000", 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3840000000000002e-06 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251552+0000", 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6860000000000001e-06 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251560+0000", 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0499999999999992e-06 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262615+0000", 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011054494 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262757+0000", 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142521 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262999+0000", 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024147400000000001 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264190+0000", 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011911840000000001 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268472+0000", 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042823940000000001 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268571+0000", 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8218000000000001e-05 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268641+0000", 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0248999999999998e-05 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.33507154199999, 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017309201999999999, 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 74 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251573+0000", 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9999999999999997e-07 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251579+0000", 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6330000000000001e-06 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251592+0000", 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3869999999999999e-05 2024-04-25T12:36:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251621+0000", 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8966e-05 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264505+0000", 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012883307 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264627+0000", 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012204 2024-04-25T12:36:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264832+0000", 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020571899999999999 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268227+0000", 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003394308 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268663+0000", 2024-04-25T12:36:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043630699999999998 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268761+0000", 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7774000000000003e-05 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268881+0000", 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120578 2024-04-25T12:36:12.659 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.335001433, 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01751983, 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.660 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 75 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251643+0000", 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5300000000000003e-07 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251849+0000", 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000206345 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251863+0000", 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3077e-05 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251873+0000", 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.08e-05 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265525+0000", 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013651876 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265667+0000", 2024-04-25T12:36:12.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014156699999999999 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265939+0000", 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027219700000000001 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268931+0000", 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029921990000000001 2024-04-25T12:36:12.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269008+0000", 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6619000000000006e-05 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269082+0000", 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4401999999999997e-05 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269162+0000", 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9895000000000007e-05 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.334753765, 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018357248, 2024-04-25T12:36:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 76 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251891+0000", 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7300000000000005e-07 2024-04-25T12:36:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251905+0000", 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4279e-05 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251915+0000", 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251935+0000", 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.961e-05 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267152+0000", 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015217341000000001 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267276+0000", 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012396 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267509+0000", 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023329500000000001 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268988+0000", 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014781709999999999 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270041+0000", 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001053806 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270161+0000", 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119494 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270247+0000", 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6382000000000001e-05 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 123.315529256, 2024-04-25T12:36:12.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099439890000000003, 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 80 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271115+0000", 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-04-25T12:36:12.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271221+0000", 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105713 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271236+0000", 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5415999999999999e-05 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271240+0000", 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3449999999999999e-06 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275441+0000", 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042009090000000001 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275579+0000", 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013740700000000001 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275795+0000", 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021634900000000001 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.277926+0000", 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021311220000000001 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.280958+0000", 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030320490000000002 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281016+0000", 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8232000000000001e-05 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281058+0000", 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1993999999999998e-05 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.96051289099999, 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094693499999999996, 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 92 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626132+0000", 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4399999999999999e-07 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626137+0000", 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4929999999999998e-06 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626147+0000", 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5300000000000002e-06 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626155+0000", 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4130000000000006e-06 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629532+0000", 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033774830000000001 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629626+0000", 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3394999999999996e-05 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629792+0000", 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016611100000000001 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633343+0000", 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00355055 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635469+0000", 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021262009999999999 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635548+0000", 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9167999999999998e-05 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635600+0000", 2024-04-25T12:36:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2262000000000002e-05 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.960476555, 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096152330000000008, 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 93 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626168+0000", 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9699999999999995e-07 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626290+0000", 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121941 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626304+0000", 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3729e-05 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626321+0000", 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7062999999999999e-05 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630147+0000", 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003826 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630229+0000", 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1953000000000006e-05 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630372+0000", 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014345199999999999 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633348+0000", 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029755789999999999 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635672+0000", 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023242829999999999 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635733+0000", 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0899000000000001e-05 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635782+0000", 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9636999999999999e-05 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.691 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.96030263500001, 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097157200000000006, 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 94 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.692 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626342+0000", 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4300000000000002e-07 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626348+0000", 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0059999999999998e-06 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626358+0000", 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626367+0000", 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8939999999999999e-06 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631146+0000", 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047795429999999998 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631307+0000", 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016041699999999999 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:12.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631539+0000", 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232258 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635089+0000", 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035503980000000002 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635880+0000", 2024-04-25T12:36:12.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079070200000000001 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635984+0000", 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104221 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.636057+0000", 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2597999999999998e-05 2024-04-25T12:36:12.698 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.468593083000002, 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010749583, 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118054+0000", 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118075+0000", 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118119+0000", 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4372000000000001e-05 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118163+0000", 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3766000000000001e-05 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119484+0000", 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001320494 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119803+0000", 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031975599999999998 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:12.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120121+0000", 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031758999999999997 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128491+0000", 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083702940000000003 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128605+0000", 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113158 2024-04-25T12:36:12.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128733+0000", 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012861099999999999 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128800+0000", 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7086000000000001e-05 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.705 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.468394097000001, 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011301215999999999, 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118251+0000", 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6160000000000001e-06 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118263+0000", 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2243e-05 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118305+0000", 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1838999999999997e-05 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118327+0000", 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2082999999999998e-05 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120981+0000", 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002653201 2024-04-25T12:36:12.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121221+0000", 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000240058 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121655+0000", 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043405400000000002 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128501+0000", 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068465519999999997 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128984+0000", 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048240400000000003 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129487+0000", 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050289499999999995 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129551+0000", 2024-04-25T12:36:12.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4270999999999995e-05 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.468286665999997, 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012520414000000001, 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118358+0000", 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3230000000000001e-06 2024-04-25T12:36:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118368+0000", 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3899999999999999e-06 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2481999999999999e-05 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118410+0000", 2024-04-25T12:36:12.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9304e-05 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126108+0000", 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076979409999999998 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126379+0000", 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027136700000000001 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126653+0000", 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274238 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129707+0000", 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030541800000000001 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130686+0000", 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097908900000000004 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130791+0000", 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104824 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:12.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130877+0000", 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6275999999999999e-05 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:12.717 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:12.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:36:12.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:12.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.646+0000 7f7aaea00640 1 -- 172.21.15.123:0/4132497205 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "2.8", "id": [3, 1]}]=0 set 2.8 pg_upmap mapping to [3,1] v53) ==== 122+0+0 (secure 0 0 0) 0x7f7aa805f7f0 con 0x7f7a9c004210 2024-04-25T12:36:12.720 INFO:teuthology.orchestra.run.smithi123.stderr:set 2.8 pg_upmap mapping to [3,1] 2024-04-25T12:36:12.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a980744b0 msgr2=0x7f7a980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:12.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a980744b0 0x7f7a980769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ab814cf30 tx=0x7f7aa4008040 comp rx=0 tx=0).stop 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c004210 msgr2=0x7f7ab81ee390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c004210 0x7f7ab81ee390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8004f10 tx=0x7f7aa8007430 comp rx=0 tx=0).stop 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 shutdown_connections 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a980744b0 0x7f7a980769c0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ab814cf30 tx=0x7f7aa4008040 comp rx=0 tx=0).stop 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ab80cd9b0 0x7f7ab814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab80cd5a0 0x7f7ab814a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 --2- 172.21.15.123:0/4132497205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c004210 0x7f7ab81ee390 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8004f10 tx=0x7f7aa8007430 comp rx=0 tx=0).stop 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.651+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 >> 172.21.15.123:0/4132497205 conn(0x7f7ab806f440 msgr2=0x7f7ab80cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.652+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 shutdown_connections 2024-04-25T12:36:12.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:12.652+0000 7f7abd800640 1 -- 172.21.15.123:0/4132497205 wait complete. 2024-04-25T12:36:12.722 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:12.718+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:12.819 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:12.819+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:12.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:12.935 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:12.935+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:13.024 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:13.024+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:13.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.090+0000 7f4f22800640 1 Processor -- start 2024-04-25T12:36:13.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.091+0000 7f4f22800640 1 -- start start 2024-04-25T12:36:13.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.094+0000 7f4f22800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4f1c0c9510 0x7f4f1c14efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.094+0000 7f4f21400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4f1c06f740 0x7f4f1c0c38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36830/0 (socket says 172.21.15.123:36830) 2024-04-25T12:36:13.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.095+0000 7f4f21400640 1 -- 172.21.15.123:0/2919969077 learned_addr learned my addr 172.21.15.123:0/2919969077 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:13.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.095+0000 7f4f20a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4f1c0c9510 0x7f4f1c14efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.095+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f1c158ec0 con 0x7f4f1c06f740 2024-04-25T12:36:13.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.095+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f1c1591a0 con 0x7f4f1c158480 2024-04-25T12:36:13.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.096+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4f1c072de0 con 0x7f4f1c0c9510 2024-04-25T12:36:13.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.096+0000 7f4f1be00640 1 -- 172.21.15.123:0/2919969077 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892970983 0 0) 0x7f4f1c158ec0 con 0x7f4f1c06f740 2024-04-25T12:36:13.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.096+0000 7f4f1be00640 1 -- 172.21.15.123:0/2919969077 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f04003520 con 0x7f4f1c06f740 2024-04-25T12:36:13.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.096+0000 7f4f1be00640 1 -- 172.21.15.123:0/2919969077 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140458764 0 0) 0x7f4f1c1591a0 con 0x7f4f1c158480 2024-04-25T12:36:13.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.096+0000 7f4f1be00640 1 -- 172.21.15.123:0/2919969077 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f1c158ec0 con 0x7f4f1c158480 2024-04-25T12:36:13.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.097+0000 7f4f20a00640 1 -- 172.21.15.123:0/2919969077 >> v1:172.21.15.153:6789/0 conn(0x7f4f1c158480 legacy=0x7f4f1c0c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.097+0000 7f4f20a00640 1 -- 172.21.15.123:0/2919969077 >> v1:172.21.15.123:6789/0 conn(0x7f4f1c06f740 legacy=0x7f4f1c0c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.097+0000 7f4f20a00640 1 -- 172.21.15.123:0/2919969077 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c06ec60 con 0x7f4f1c0c9510 2024-04-25T12:36:13.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.098+0000 7f4f20a00640 1 --2- 172.21.15.123:0/2919969077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c14efc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009400 tx=0x7f4f08005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f5145dd84508db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.098+0000 7f4f1be00640 1 -- 172.21.15.123:0/2919969077 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08004230 con 0x7f4f1c0c9510 2024-04-25T12:36:13.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.098+0000 7f4f1be00640 1 -- 172.21.15.123:0/2919969077 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f080043d0 con 0x7f4f1c0c9510 2024-04-25T12:36:13.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.098+0000 7f4f1be00640 1 -- 172.21.15.123:0/2919969077 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08003830 con 0x7f4f1c0c9510 2024-04-25T12:36:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.100+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 msgr2=0x7f4f1c14efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.100+0000 7f4f22800640 1 --2- 172.21.15.123:0/2919969077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c14efc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009400 tx=0x7f4f08005dc0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.100+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 shutdown_connections 2024-04-25T12:36:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.100+0000 7f4f22800640 1 --2- 172.21.15.123:0/2919969077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c14efc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.100+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 >> 172.21.15.123:0/2919969077 conn(0x7f4f1c0cd600 msgr2=0x7f4f1c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.100+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 shutdown_connections 2024-04-25T12:36:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.100+0000 7f4f22800640 1 -- 172.21.15.123:0/2919969077 wait complete. 2024-04-25T12:36:13.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.104+0000 7f4f22800640 1 Processor -- start 2024-04-25T12:36:13.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.104+0000 7f4f22800640 1 -- start start 2024-04-25T12:36:13.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f22800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f1c06f740 0x7f4f1c0c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f22800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c1f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f22800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c158480 0x7f4f1c14b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f22800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c0c8ea0 con 0x7f4f1c0c9510 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f22800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c06f220 con 0x7f4f1c158480 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f22800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1c0c9020 con 0x7f4f1c06f740 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f20a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c1f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f20a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c1f1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44262/0 (socket says 172.21.15.123:44262) 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f20a00640 1 -- 172.21.15.123:0/4111272067 learned_addr learned my addr 172.21.15.123:0/4111272067 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.106+0000 7f4f21400640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f1c06f740 0x7f4f1c0c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.107+0000 7f4f21e00640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c158480 0x7f4f1c14b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.107+0000 7f4f20a00640 1 -- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f1c06f740 msgr2=0x7f4f1c0c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.107+0000 7f4f20a00640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f1c06f740 0x7f4f1c0c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.107+0000 7f4f20a00640 1 -- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c158480 msgr2=0x7f4f1c14b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.107+0000 7f4f20a00640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c158480 0x7f4f1c14b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.107+0000 7f4f20a00640 1 -- 172.21.15.123:0/4111272067 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c0c61f0 con 0x7f4f1c0c9510 2024-04-25T12:36:13.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.107+0000 7f4f20a00640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c1f1320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4f08033040 tx=0x7f4f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.108+0000 7f4f19600640 1 -- 172.21.15.123:0/4111272067 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08040070 con 0x7f4f1c0c9510 2024-04-25T12:36:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.108+0000 7f4f19600640 1 -- 172.21.15.123:0/4111272067 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f0803c430 con 0x7f4f1c0c9510 2024-04-25T12:36:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.108+0000 7f4f19600640 1 -- 172.21.15.123:0/4111272067 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0803c800 con 0x7f4f1c0c9510 2024-04-25T12:36:13.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.108+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c152910 con 0x7f4f1c0c9510 2024-04-25T12:36:13.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.108+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1f1d00 con 0x7f4f1c0c9510 2024-04-25T12:36:13.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.110+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c152670 con 0x7f4f1c0c9510 2024-04-25T12:36:13.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.112+0000 7f4f19600640 1 -- 172.21.15.123:0/4111272067 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f08043070 con 0x7f4f1c0c9510 2024-04-25T12:36:13.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.113+0000 7f4f19600640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4efc0744b0 0x7f4efc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.113+0000 7f4f19600640 1 -- 172.21.15.123:0/4111272067 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6648+0+0 (secure 0 0 0) 0x7f4f080b5ff0 con 0x7f4f1c0c9510 2024-04-25T12:36:13.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.113+0000 7f4f21400640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4efc0744b0 0x7f4efc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.114+0000 7f4f21400640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4efc0744b0 0x7f4efc0769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f10000f60 tx=0x7f4f10009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.119 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:13.119 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:13.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.121+0000 7f4f19600640 1 -- 172.21.15.123:0/4111272067 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f1c152670 con 0x7f4f1c0c9510 2024-04-25T12:36:13.124 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:13.126+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:13.150 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:13.238 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:13.238+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.256+0000 7f31ea600640 1 Processor -- start 2024-04-25T12:36:13.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.256+0000 7f31ea600640 1 -- start start 2024-04-25T12:36:13.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31ea600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31e40c4500 0x7f31e40cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31ea600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40b23b0 0x7f31e40cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31ea600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e415b0b0 con 0x7f31e41543f0 2024-04-25T12:36:13.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31ea600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f31e40cb370 con 0x7f31e40c4500 2024-04-25T12:36:13.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31ea600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40cc010 con 0x7f31e40b23b0 2024-04-25T12:36:13.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31e9c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40b23b0 0x7f31e40cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31e9c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40b23b0 0x7f31e40cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:59986/0 (socket says 172.21.15.123:59986) 2024-04-25T12:36:13.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.258+0000 7f31e9c00640 1 -- 172.21.15.123:0/2411416784 learned_addr learned my addr 172.21.15.123:0/2411416784 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:13.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.259+0000 7f31e9c00640 1 -- 172.21.15.123:0/2411416784 >> v2:172.21.15.123:3300/0 conn(0x7f31e40c4500 msgr2=0x7f31e40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:13.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.259+0000 7f31e9c00640 1 --2- 172.21.15.123:0/2411416784 >> v2:172.21.15.123:3300/0 conn(0x7f31e40c4500 0x7f31e40cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.259+0000 7f31e9c00640 1 -- 172.21.15.123:0/2411416784 >> v1:172.21.15.123:6789/0 conn(0x7f31e41543f0 legacy=0x7f31e40cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:13.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.259+0000 7f31e9c00640 1 -- 172.21.15.123:0/2411416784 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40cadb0 con 0x7f31e40b23b0 2024-04-25T12:36:13.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.260+0000 7f31e9c00640 1 --2- 172.21.15.123:0/2411416784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40b23b0 0x7f31e40cc9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009e50 tx=0x7f31dc02a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea60db91508b4df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.260+0000 7f31e3400640 1 -- 172.21.15.123:0/2411416784 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc036070 con 0x7f31e40b23b0 2024-04-25T12:36:13.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.260+0000 7f31e3400640 1 -- 172.21.15.123:0/2411416784 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31dc031600 con 0x7f31e40b23b0 2024-04-25T12:36:13.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.260+0000 7f31e3400640 1 -- 172.21.15.123:0/2411416784 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0319d0 con 0x7f31e40b23b0 2024-04-25T12:36:13.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.261+0000 7f31ea600640 1 -- 172.21.15.123:0/2411416784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40b23b0 msgr2=0x7f31e40cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.261+0000 7f31ea600640 1 --2- 172.21.15.123:0/2411416784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40b23b0 0x7f31e40cc9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009e50 tx=0x7f31dc02a760 comp rx=0 tx=0).stop 2024-04-25T12:36:13.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.261+0000 7f31ea600640 1 -- 172.21.15.123:0/2411416784 shutdown_connections 2024-04-25T12:36:13.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.261+0000 7f31ea600640 1 --2- 172.21.15.123:0/2411416784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40b23b0 0x7f31e40cc9e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.261+0000 7f31ea600640 1 --2- 172.21.15.123:0/2411416784 >> v2:172.21.15.123:3300/0 conn(0x7f31e40c4500 0x7f31e40cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.261+0000 7f31ea600640 1 -- 172.21.15.123:0/2411416784 >> 172.21.15.123:0/2411416784 conn(0x7f31e4068810 msgr2=0x7f31e4068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:13.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.261+0000 7f31ea600640 1 -- 172.21.15.123:0/2411416784 shutdown_connections 2024-04-25T12:36:13.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.262+0000 7f31ea600640 1 -- 172.21.15.123:0/2411416784 wait complete. 2024-04-25T12:36:13.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.264+0000 7f31ea600640 1 Processor -- start 2024-04-25T12:36:13.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.264+0000 7f31ea600640 1 -- start start 2024-04-25T12:36:13.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31ea600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e40b23b0 0x7f31e4155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31ea600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40c4500 0x7f31e4156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31ea600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31e41543f0 0x7f31e40c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31ea600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40cb590 con 0x7f31e40b23b0 2024-04-25T12:36:13.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31ea600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40cbca0 con 0x7f31e41543f0 2024-04-25T12:36:13.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31ea600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40cb710 con 0x7f31e40c4500 2024-04-25T12:36:13.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40c4500 0x7f31e4156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31e9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31e41543f0 0x7f31e40c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31e9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31e41543f0 0x7f31e40c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:56494/0 (socket says 172.21.15.123:56494) 2024-04-25T12:36:13.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40c4500 0x7f31e4156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:59996/0 (socket says 172.21.15.123:59996) 2024-04-25T12:36:13.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.266+0000 7f31e9c00640 1 -- 172.21.15.123:0/998102351 learned_addr learned my addr 172.21.15.123:0/998102351 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:13.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e40b23b0 0x7f31e4155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e3e00640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31e41543f0 msgr2=0x7f31e40c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e3e00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31e41543f0 0x7f31e40c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e3e00640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e40b23b0 msgr2=0x7f31e4155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e3e00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e40b23b0 0x7f31e4155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e3e00640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e4157310 con 0x7f31e40c4500 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e9c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31e41543f0 0x7f31e40c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.267+0000 7f31e3e00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40c4500 0x7f31e4156dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b440 tx=0x7f31d000b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.268+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0011830 con 0x7f31e40c4500 2024-04-25T12:36:13.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.268+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d0016070 con 0x7f31e40c4500 2024-04-25T12:36:13.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.268+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0011bf0 con 0x7f31e40c4500 2024-04-25T12:36:13.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.268+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e40c69a0 con 0x7f31e40c4500 2024-04-25T12:36:13.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.268+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e41562c0 con 0x7f31e40c4500 2024-04-25T12:36:13.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.269+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f31e4076da0 con 0x7f31e40c4500 2024-04-25T12:36:13.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.271+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f31d0025430 con 0x7f31e40c4500 2024-04-25T12:36:13.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.272+0000 7f31e0c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31c40744b0 0x7f31c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.272+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 6648+0+0 (secure 0 0 0) 0x7f31d0095b80 con 0x7f31e40c4500 2024-04-25T12:36:13.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.272+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7f31d00960a0 con 0x7f31e40c4500 2024-04-25T12:36:13.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.273+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f31e40c97d0 0x7f31e4150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.273+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31e406f690 con 0x7f31e40c97d0 2024-04-25T12:36:13.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.273+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31c40744b0 0x7f31c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.274+0000 7f31e9c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f31e40c97d0 0x7f31e4150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.275+0000 7f31e9c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f31e40c97d0 0x7f31e4150f90 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.275+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31c40744b0 0x7f31c40769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31d800bbf0 tx=0x7f31d8006dd0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.276+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e406f690 con 0x7f31e40c97d0 2024-04-25T12:36:13.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.295+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e40cb8e0 con 0x7f31e40c97d0 2024-04-25T12:36:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.298+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e40cb8e0 con 0x7f31e40c97d0 2024-04-25T12:36:13.298 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:13.299 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:13.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.298+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f31e4156490 0x7f31e41f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.298+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f31e40c4ed0 con 0x7f31e4156490 2024-04-25T12:36:13.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.299+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f31e4156490 0x7f31e41f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.301+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f31e4156490 0x7f31e41f3dd0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.303+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e40c4ed0 con 0x7f31e4156490 2024-04-25T12:36:13.326 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:13.327+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:13.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.327+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e40c4bf0 con 0x7f31e4156490 2024-04-25T12:36:13.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.331+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e40c4bf0 con 0x7f31e4156490 2024-04-25T12:36:13.335 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.331+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f31e40c9180 0x7f31e41f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.331+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f31e40c8af0 con 0x7f31e40c9180 2024-04-25T12:36:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.332+0000 7f31e3e00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f31e40c9180 0x7f31e41f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.333+0000 7f31e3e00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f31e40c9180 0x7f31e41f4820 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.334+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e40c8af0 con 0x7f31e40c9180 2024-04-25T12:36:13.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.362+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e4062fe0 con 0x7f31e40c9180 2024-04-25T12:36:13.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.365+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e4062fe0 con 0x7f31e40c9180 2024-04-25T12:36:13.365 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:13.366 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.365+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f31e4063ac0 0x7f31e40cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.365+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f31e4152430 con 0x7f31e4063ac0 2024-04-25T12:36:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.366+0000 7f31e9c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f31e4063ac0 0x7f31e40cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.366+0000 7f31e9c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f31e4063ac0 0x7f31e40cd3e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.368+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e4152430 con 0x7f31e4063ac0 2024-04-25T12:36:13.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.396+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e40cd920 con 0x7f31e4063ac0 2024-04-25T12:36:13.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.399+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e40cd920 con 0x7f31e4063ac0 2024-04-25T12:36:13.400 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:13.400 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:13.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.400+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f31e40ce040 0x7f31e402c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.400+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f31e40cd920 con 0x7f31e40ce040 2024-04-25T12:36:13.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.400+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f31e40ce040 0x7f31e402c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.401+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f31e40ce040 0x7f31e402c9d0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.402+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e40cd920 con 0x7f31e40ce040 2024-04-25T12:36:13.427 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:13.428+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:13.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.430+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e402cf10 con 0x7f31e40ce040 2024-04-25T12:36:13.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.432+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e402cf10 con 0x7f31e40ce040 2024-04-25T12:36:13.432 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:13.433 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:13.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.432+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f31e402d630 0x7f31e402da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.432+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f31e4207a20 con 0x7f31e402d630 2024-04-25T12:36:13.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.433+0000 7f31e3e00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f31e402d630 0x7f31e402da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.434+0000 7f31e3e00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f31e402d630 0x7f31e402da40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.435+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e4207a20 con 0x7f31e402d630 2024-04-25T12:36:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.462+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e4205190 con 0x7f31e402d630 2024-04-25T12:36:13.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.465+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e4205190 con 0x7f31e402d630 2024-04-25T12:36:13.465 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:13.467 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:13.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.465+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f31e4207040 0x7f31e4207490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.465+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f31e420ae70 con 0x7f31e4207040 2024-04-25T12:36:13.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.466+0000 7f31e9c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f31e4207040 0x7f31e4207490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.467+0000 7f31e9c00640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f31e4207040 0x7f31e4207490 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.467+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e420ae70 con 0x7f31e4207040 2024-04-25T12:36:13.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.484+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f4f1c0c58f0 con 0x7f4f1c0c9510 2024-04-25T12:36:13.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.487+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e4209cf0 con 0x7f31e4207040 2024-04-25T12:36:13.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.489+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e4209cf0 con 0x7f31e4207040 2024-04-25T12:36:13.490 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:13.490 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:13.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.490+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f31e420a410 0x7f31e420a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:13.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.490+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f31e420e1e0 con 0x7f31e420a410 2024-04-25T12:36:13.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.490+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f31e420a410 0x7f31e420a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:13.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.492+0000 7f31e9200640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f31e420a410 0x7f31e420a820 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:13.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.493+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31e420e1e0 con 0x7f31e420a410 2024-04-25T12:36:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.508+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f31e420d060 con 0x7f31e420a410 2024-04-25T12:36:13.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.510+0000 7f31e0c00640 1 -- 172.21.15.123:0/998102351 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31e420d060 con 0x7f31e420a410 2024-04-25T12:36:13.510 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.510+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f31e40c97d0 msgr2=0x7f31e4150f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.510+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f31e40c97d0 0x7f31e4150f90 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f31e4156490 msgr2=0x7f31e41f3dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f31e4156490 0x7f31e41f3dd0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f31e40c9180 msgr2=0x7f31e41f4820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f31e40c9180 0x7f31e41f4820 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f31e4063ac0 msgr2=0x7f31e40cd3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f31e4063ac0 0x7f31e40cd3e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f31e40ce040 msgr2=0x7f31e402c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f31e40ce040 0x7f31e402c9d0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f31e402d630 msgr2=0x7f31e402da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f31e402d630 0x7f31e402da40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f31e4207040 msgr2=0x7f31e4207490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f31e4207040 0x7f31e4207490 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f31e420a410 msgr2=0x7f31e420a820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f31e420a410 0x7f31e420a820 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31e9c00640 1 -- 172.21.15.123:0/998102351 reap_dead start 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31e9c00640 1 -- 172.21.15.123:0/998102351 reap_dead start 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31e9c00640 1 -- 172.21.15.123:0/998102351 reap_dead start 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31e9c00640 1 -- 172.21.15.123:0/998102351 reap_dead start 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31e9c00640 1 -- 172.21.15.123:0/998102351 reap_dead start 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31c40744b0 msgr2=0x7f31c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31c40744b0 0x7f31c40769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31d800bbf0 tx=0x7f31d8006dd0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40c4500 msgr2=0x7f31e4156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40c4500 0x7f31e4156dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b440 tx=0x7f31d000b900 comp rx=0 tx=0).stop 2024-04-25T12:36:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 shutdown_connections 2024-04-25T12:36:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31c40744b0 0x7f31c40769c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 --2- 172.21.15.123:0/998102351 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31e40c4500 0x7f31e4156dd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 >> 172.21.15.123:0/998102351 conn(0x7f31e4068810 msgr2=0x7f31e4068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 shutdown_connections 2024-04-25T12:36:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.511+0000 7f31ea600640 1 -- 172.21.15.123:0/998102351 wait complete. 2024-04-25T12:36:13.528 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:13.529+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:13.546 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:13.547 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:13.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:13.629 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:13.630+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:13.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.655+0000 7f4f19600640 1 -- 172.21.15.123:0/4111272067 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v54) ==== 87+0+0 (secure 0 0 0) 0x7f4f0807fea0 con 0x7f4f1c0c9510 2024-04-25T12:36:13.655 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.658+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4efc0744b0 msgr2=0x7f4efc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.658+0000 7f4f22800640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4efc0744b0 0x7f4efc0769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f10000f60 tx=0x7f4f10009040 comp rx=0 tx=0).stop 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.658+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 msgr2=0x7f4f1c1f1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.658+0000 7f4f22800640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c1f1320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4f08033040 tx=0x7f4f080046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 shutdown_connections 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4efc0744b0 0x7f4efc0769c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c158480 0x7f4f1c14b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c0c9510 0x7f4f1c1f1320 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 --2- 172.21.15.123:0/4111272067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4f1c06f740 0x7f4f1c0c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:13.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 >> 172.21.15.123:0/4111272067 conn(0x7f4f1c0cd600 msgr2=0x7f4f1c0cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:13.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 shutdown_connections 2024-04-25T12:36:13.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:13.659+0000 7f4f22800640 1 -- 172.21.15.123:0/4111272067 wait complete. 2024-04-25T12:36:13.733 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:13.733+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:13.830 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:13.830+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:13.871 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:13.872 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:13.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:13.933 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:13.933+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:14.032 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:14.032+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:14.133 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:14.134+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:14.234 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:14.234+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:14.316 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:14.316 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:14.334 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:14.335+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:14.348 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:14.439 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:14.439+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:14.536 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:14.537+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:14.637 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:14.637+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:14.657 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:14.657 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:14.692 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:14.739 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:14.739+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:14.838 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:14.838+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:14.938 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:14.940+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:14.996 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:14.996 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:15.015 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:15.060 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:15.061+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:15.140 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:15.140+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:15.140 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:15.140 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:15.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:15.241 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:15.242+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:15.303 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:15.304 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:15.312 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:15.342 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:15.342+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:15.444 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:15.443+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:15.464 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:15.464 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:15.474 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:15.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:15.545 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:15.545+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:15.626 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:15.627 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:15.636 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:15.679 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:15.680+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:15.692 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:36:15.745 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:15.746+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:15.747 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:15.747 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:15.756 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:15.847 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:15.847+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:15.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.891+0000 7f2bdf600640 1 Processor -- start 2024-04-25T12:36:15.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.891+0000 7f2bdf600640 1 -- start start 2024-04-25T12:36:15.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.893+0000 7f2bdf600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd80ca770 0x7f2bd815d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.893+0000 7f2bdf600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd8073c10 con 0x7f2bd80b2a10 2024-04-25T12:36:15.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.893+0000 7f2bdf600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd8074100 con 0x7f2bd8073460 2024-04-25T12:36:15.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.893+0000 7f2bdf600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd815d7b0 con 0x7f2bd80ca770 2024-04-25T12:36:15.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.894+0000 7f2bdcc00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2bd8073460 0x7f2bd80c3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:37054/0 (socket says 172.21.15.123:37054) 2024-04-25T12:36:15.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.894+0000 7f2bdcc00640 1 -- 172.21.15.123:0/3646545640 learned_addr learned my addr 172.21.15.123:0/3646545640 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:15.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.894+0000 7f2bdd600640 1 --2- 172.21.15.123:0/3646545640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd80ca770 0x7f2bd815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.894+0000 7f2bdd600640 1 -- 172.21.15.123:0/3646545640 >> v1:172.21.15.153:6789/0 conn(0x7f2bd8073460 legacy=0x7f2bd80c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:15.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.894+0000 7f2bdd600640 1 -- 172.21.15.123:0/3646545640 >> v1:172.21.15.123:6789/0 conn(0x7f2bd80b2a10 legacy=0x7f2bd80ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:15.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.895+0000 7f2bdd600640 1 -- 172.21.15.123:0/3646545640 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd8070a20 con 0x7f2bd80ca770 2024-04-25T12:36:15.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.896+0000 7f2bdd600640 1 --2- 172.21.15.123:0/3646545640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd80ca770 0x7f2bd815d270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009dc0 tx=0x7f2bd002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb9ba2e0bf9224e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:15.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.896+0000 7f2bd7400640 1 -- 172.21.15.123:0/3646545640 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0007740 con 0x7f2bd80ca770 2024-04-25T12:36:15.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.896+0000 7f2bd7400640 1 -- 172.21.15.123:0/3646545640 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd00078e0 con 0x7f2bd80ca770 2024-04-25T12:36:15.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.896+0000 7f2bd7400640 1 -- 172.21.15.123:0/3646545640 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0007cb0 con 0x7f2bd80ca770 2024-04-25T12:36:15.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.897+0000 7f2bdf600640 1 -- 172.21.15.123:0/3646545640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd80ca770 msgr2=0x7f2bd815d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:15.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.897+0000 7f2bdf600640 1 --2- 172.21.15.123:0/3646545640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd80ca770 0x7f2bd815d270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009dc0 tx=0x7f2bd002a6d0 comp rx=0 tx=0).stop 2024-04-25T12:36:15.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.897+0000 7f2bdf600640 1 -- 172.21.15.123:0/3646545640 shutdown_connections 2024-04-25T12:36:15.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.897+0000 7f2bdf600640 1 --2- 172.21.15.123:0/3646545640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd80ca770 0x7f2bd815d270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:15.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.897+0000 7f2bdf600640 1 -- 172.21.15.123:0/3646545640 >> 172.21.15.123:0/3646545640 conn(0x7f2bd806f440 msgr2=0x7f2bd806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:15.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.897+0000 7f2bdf600640 1 -- 172.21.15.123:0/3646545640 shutdown_connections 2024-04-25T12:36:15.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.897+0000 7f2bdf600640 1 -- 172.21.15.123:0/3646545640 wait complete. 2024-04-25T12:36:15.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.900+0000 7f2bdf600640 1 Processor -- start 2024-04-25T12:36:15.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.900+0000 7f2bdf600640 1 -- start start 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.901+0000 7f2bdf600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd8073460 0x7f2bd81558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.901+0000 7f2bdf600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd80b2a10 0x7f2bd80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdf600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd80ca770 0x7f2bd814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdf600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8071310 con 0x7f2bd80ca770 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdf600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8156fb0 con 0x7f2bd80b2a10 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdf600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd8073930 con 0x7f2bd8073460 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd80ca770 0x7f2bd814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd80ca770 0x7f2bd814d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44282/0 (socket says 172.21.15.123:44282) 2024-04-25T12:36:15.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 learned_addr learned my addr 172.21.15.123:0/4030856468 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:15.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd8073460 0x7f2bd81558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.902+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd80b2a10 0x7f2bd80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd8073460 msgr2=0x7f2bd81558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bdd600640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd8073460 0x7f2bd81558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd80b2a10 msgr2=0x7f2bd80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bdd600640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd80b2a10 0x7f2bd80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80cd230 con 0x7f2bd80ca770 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bd8073460 0x7f2bd81558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:15.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd80b2a10 0x7f2bd80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:15.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.903+0000 7f2bdd600640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd80ca770 0x7f2bd814d2e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2bd002acc0 tx=0x7f2bd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:15.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.904+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd003f070 con 0x7f2bd80ca770 2024-04-25T12:36:15.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.904+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd0031bc0 con 0x7f2bd80ca770 2024-04-25T12:36:15.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.904+0000 7f2bdf600640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd80ce320 con 0x7f2bd80ca770 2024-04-25T12:36:15.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.904+0000 7f2bdf600640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd8154f80 con 0x7f2bd80ca770 2024-04-25T12:36:15.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.904+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0044460 con 0x7f2bd80ca770 2024-04-25T12:36:15.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.906+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f2bd8076700 con 0x7f2bd80ca770 2024-04-25T12:36:15.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.907+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2bd003b410 con 0x7f2bd80ca770 2024-04-25T12:36:15.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.908+0000 7f2bd4c00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2bb80744b0 0x7f2bb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.908+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6648+0+0 (secure 0 0 0) 0x7f2bd00b7060 con 0x7f2bd80ca770 2024-04-25T12:36:15.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.908+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2bb80744b0 0x7f2bb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.908+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v54) ==== 54+0+16 (secure 0 0 0) 0x7f2bd00b9600 con 0x7f2bd80ca770 2024-04-25T12:36:15.909 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:15.909 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:15.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.909+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f2bd81551c0 0x7f2bd80cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.909+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bd81f82c0 con 0x7f2bd81551c0 2024-04-25T12:36:15.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.909+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f2bd81551c0 0x7f2bd80cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.909+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2bb80744b0 0x7f2bb80769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005880 tx=0x7f2bcc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:15.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.910+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f2bd81551c0 0x7f2bd80cba80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:15.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.911+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd81f82c0 con 0x7f2bd81551c0 2024-04-25T12:36:15.917 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:15.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.927+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd80cd400 con 0x7f2bd81551c0 2024-04-25T12:36:15.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.929+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd80cd400 con 0x7f2bd81551c0 2024-04-25T12:36:15.929 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:15.929 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:15.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.929+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f2bd81fa880 0x7f2bd81fac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.929+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2bd80aef00 con 0x7f2bd81fa880 2024-04-25T12:36:15.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.930+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f2bd81fa880 0x7f2bd81fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.931+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f2bd81fa880 0x7f2bd81fac90 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:15.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.933+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd80aef00 con 0x7f2bd81fa880 2024-04-25T12:36:15.961 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:15.962+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:15.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.963+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd80c9b90 con 0x7f2bd81fa880 2024-04-25T12:36:15.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.966+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd80c9b90 con 0x7f2bd81fa880 2024-04-25T12:36:15.967 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:15.967 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:15.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.967+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f2bd80634f0 0x7f2bd8063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:15.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.967+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2bd8073e00 con 0x7f2bd80634f0 2024-04-25T12:36:15.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.967+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f2bd80634f0 0x7f2bd8063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:15.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.969+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f2bd80634f0 0x7f2bd8063900 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:15.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.970+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd8073e00 con 0x7f2bd80634f0 2024-04-25T12:36:15.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:15.999+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd80c9600 con 0x7f2bd80634f0 2024-04-25T12:36:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.002+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd80c9600 con 0x7f2bd80634f0 2024-04-25T12:36:16.002 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.002+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f2bd815c520 0x7f2bd815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.002+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f2bd80c9320 con 0x7f2bd815c520 2024-04-25T12:36:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.003+0000 7f2bdd600640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f2bd815c520 0x7f2bd815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.003+0000 7f2bdd600640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f2bd815c520 0x7f2bd815c930 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.005+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd80c9320 con 0x7f2bd815c520 2024-04-25T12:36:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.026+0000 7fdd2a200640 1 Processor -- start 2024-04-25T12:36:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.026+0000 7fdd2a200640 1 -- start start 2024-04-25T12:36:16.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd2a200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd240c90d0 0x7fdd240c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd2a200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd240c4240 0x7fdd240cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd2a200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdd2415a730 0x7fdd2415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd2a200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2415b080 con 0x7fdd240c4240 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd2a200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fdd240c4810 con 0x7fdd2415a730 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd2a200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240c83f0 con 0x7fdd240c90d0 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd22a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd240c4240 0x7fdd240cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd23e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdd2415a730 0x7fdd2415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd22a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd240c4240 0x7fdd240cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44292/0 (socket says 172.21.15.123:44292) 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.028+0000 7fdd22a00640 1 -- 172.21.15.123:0/1800298936 learned_addr learned my addr 172.21.15.123:0/1800298936 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.029+0000 7fdd23400640 1 --2- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd240c90d0 0x7fdd240c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.029+0000 7fdd23e00640 1 -- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd240c90d0 msgr2=0x7fdd240c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:16.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.029+0000 7fdd23e00640 1 --2- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd240c90d0 0x7fdd240c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.029+0000 7fdd23e00640 1 -- 172.21.15.123:0/1800298936 >> v2:172.21.15.123:3300/0 conn(0x7fdd240c4240 msgr2=0x7fdd240cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:16.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.029+0000 7fdd23e00640 1 --2- 172.21.15.123:0/1800298936 >> v2:172.21.15.123:3300/0 conn(0x7fdd240c4240 0x7fdd240cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.029+0000 7fdd23e00640 1 -- 172.21.15.123:0/1800298936 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd240c86d0 con 0x7fdd2415a730 2024-04-25T12:36:16.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.030+0000 7fdd23e00640 1 --2- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd2415a730 0x7fdd2415ab40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd240c37c0 tx=0x7fdd1c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83c9bda65493a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.030+0000 7fdd22000640 1 -- 172.21.15.123:0/1800298936 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c007740 con 0x7fdd2415a730 2024-04-25T12:36:16.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.030+0000 7fdd22000640 1 -- 172.21.15.123:0/1800298936 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0078e0 con 0x7fdd2415a730 2024-04-25T12:36:16.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.030+0000 7fdd22000640 1 -- 172.21.15.123:0/1800298936 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c007cb0 con 0x7fdd2415a730 2024-04-25T12:36:16.031 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:16.032 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:16.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 -- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd2415a730 msgr2=0x7fdd2415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 --2- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd2415a730 0x7fdd2415ab40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd240c37c0 tx=0x7fdd1c02a660 comp rx=0 tx=0).stop 2024-04-25T12:36:16.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 -- 172.21.15.123:0/1800298936 shutdown_connections 2024-04-25T12:36:16.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 --2- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd2415a730 0x7fdd2415ab40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 --2- 172.21.15.123:0/1800298936 >> v2:172.21.15.123:3300/0 conn(0x7fdd240c4240 0x7fdd240cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 --2- 172.21.15.123:0/1800298936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd240c90d0 0x7fdd240c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 -- 172.21.15.123:0/1800298936 >> 172.21.15.123:0/1800298936 conn(0x7fdd2406c750 msgr2=0x7fdd2406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:16.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 -- 172.21.15.123:0/1800298936 shutdown_connections 2024-04-25T12:36:16.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.031+0000 7fdd2a200640 1 -- 172.21.15.123:0/1800298936 wait complete. 2024-04-25T12:36:16.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.033+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd80c9320 con 0x7f2bd815c520 2024-04-25T12:36:16.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.033+0000 7fdd2a200640 1 Processor -- start 2024-04-25T12:36:16.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.034+0000 7fdd2a200640 1 -- start start 2024-04-25T12:36:16.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7fdd2a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd240c4240 0x7fdd241fa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7fdd2a200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd240c90d0 0x7fdd24159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7fdd2a200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd2415a730 0x7fdd2414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7fdd2a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd240ce1b0 con 0x7fdd240c4240 2024-04-25T12:36:16.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7fdd2a200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd240c8c90 con 0x7fdd240c90d0 2024-04-25T12:36:16.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7fdd2a200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240ce330 con 0x7fdd2415a730 2024-04-25T12:36:16.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd80c9320 con 0x7f2bd815c520 2024-04-25T12:36:16.037 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:16.038 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:16.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.036+0000 7fdd23400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd240c4240 0x7fdd241fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd22a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd240c90d0 0x7fdd24159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd23400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd240c4240 0x7fdd241fa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44294/0 (socket says 172.21.15.123:44294) 2024-04-25T12:36:16.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd23400640 1 -- 172.21.15.123:0/2683443328 learned_addr learned my addr 172.21.15.123:0/2683443328 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:16.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd23e00640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd2415a730 0x7fdd2414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f2bd820a100 0x7f2bd820c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f2bd81549c0 con 0x7f2bd820a100 2024-04-25T12:36:16.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd23e00640 1 -- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd240c90d0 msgr2=0x7fdd24159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd23e00640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd240c90d0 0x7fdd24159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd23e00640 1 -- 172.21.15.123:0/2683443328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd240c4240 msgr2=0x7fdd241fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f2bd820a100 0x7f2bd820c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.037+0000 7fdd23e00640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd240c4240 0x7fdd241fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd23e00640 1 -- 172.21.15.123:0/2683443328 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd241fac70 con 0x7fdd2415a730 2024-04-25T12:36:16.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd22a00640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd240c90d0 0x7fdd24159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:16.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd23400640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd240c4240 0x7fdd241fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:16.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd23e00640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd2415a730 0x7fdd2414df00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd240c4990 tx=0x7fdd1c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f2bd820a100 0x7f2bd820c520 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd1b400640 1 -- 172.21.15.123:0/2683443328 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c03f070 con 0x7fdd2415a730 2024-04-25T12:36:16.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd1b400640 1 -- 172.21.15.123:0/2683443328 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c031bc0 con 0x7fdd2415a730 2024-04-25T12:36:16.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd1b400640 1 -- 172.21.15.123:0/2683443328 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c044460 con 0x7fdd2415a730 2024-04-25T12:36:16.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.038+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2414eee0 con 0x7fdd2415a730 2024-04-25T12:36:16.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.039+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd241f9f30 con 0x7fdd2415a730 2024-04-25T12:36:16.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.040+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd81549c0 con 0x7f2bd820a100 2024-04-25T12:36:16.044 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:16.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.040+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2406d500 con 0x7fdd2415a730 2024-04-25T12:36:16.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.041+0000 7fdd1b400640 1 -- 172.21.15.123:0/2683443328 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd1c007790 con 0x7fdd2415a730 2024-04-25T12:36:16.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.042+0000 7fdd1b400640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdd040744e0 0x7fdd040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.042+0000 7fdd1b400640 1 -- 172.21.15.123:0/2683443328 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 6648+0+0 (secure 0 0 0) 0x7fdd1c0b6a00 con 0x7fdd2415a730 2024-04-25T12:36:16.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.043+0000 7fdd23400640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdd040744e0 0x7fdd040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.043+0000 7fdd23400640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdd040744e0 0x7fdd040769f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd14007e50 tx=0x7fdd14008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.047+0000 7fdd1b400640 1 -- 172.21.15.123:0/2683443328 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd1c04b030 con 0x7fdd2415a730 2024-04-25T12:36:16.049 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:16.049+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:16.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.069+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd81549c0 con 0x7f2bd820a100 2024-04-25T12:36:16.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.071+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd81549c0 con 0x7f2bd820a100 2024-04-25T12:36:16.072 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:16.073 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:16.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.071+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2bd820cf70 0x7f2bd820f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.071+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f2bd8212160 con 0x7f2bd820cf70 2024-04-25T12:36:16.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.072+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2bd820cf70 0x7f2bd820f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.074+0000 7f2bd7e00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2bd820cf70 0x7f2bd820f390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.075+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd8212160 con 0x7f2bd820cf70 2024-04-25T12:36:16.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.103+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd820f8d0 con 0x7f2bd820cf70 2024-04-25T12:36:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.105+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd820f8d0 con 0x7f2bd820cf70 2024-04-25T12:36:16.106 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.106+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2bd820fff0 0x7f2bd8210400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.106+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f2bd8215500 con 0x7f2bd820fff0 2024-04-25T12:36:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.106+0000 7f2bdd600640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2bd820fff0 0x7f2bd8210400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.107+0000 7f2bdd600640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2bd820fff0 0x7f2bd8210400 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.108+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd8215500 con 0x7f2bd820fff0 2024-04-25T12:36:16.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.136+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd8210a50 con 0x7f2bd820fff0 2024-04-25T12:36:16.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.138+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd8210a50 con 0x7f2bd820fff0 2024-04-25T12:36:16.139 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:16.139 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:16.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.139+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f2bd8210f60 0x7f2bd8211370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:16.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.139+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f2bd8211930 con 0x7f2bd8210f60 2024-04-25T12:36:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.139+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f2bd8210f60 0x7f2bd8211370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:16.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.140+0000 7f2bdcc00640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f2bd8210f60 0x7f2bd8211370 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:16.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.141+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd8211930 con 0x7f2bd8210f60 2024-04-25T12:36:16.150 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:16.150+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:16.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.169+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2bd82118b0 con 0x7f2bd8210f60 2024-04-25T12:36:16.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.171+0000 7f2bd4c00640 1 -- 172.21.15.123:0/4030856468 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2bd82118b0 con 0x7f2bd8210f60 2024-04-25T12:36:16.171 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:16.172 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:16.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f2bd81551c0 msgr2=0x7f2bd80cba80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f2bd81551c0 0x7f2bd80cba80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f2bd81fa880 msgr2=0x7f2bd81fac90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f2bd81fa880 0x7f2bd81fac90 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f2bd80634f0 msgr2=0x7f2bd8063900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f2bd80634f0 0x7f2bd8063900 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f2bd815c520 msgr2=0x7f2bd815c930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f2bd815c520 0x7f2bd815c930 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f2bd820a100 msgr2=0x7f2bd820c520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f2bd820a100 0x7f2bd820c520 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2bd820cf70 msgr2=0x7f2bd820f390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2bd820cf70 0x7f2bd820f390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2bd820fff0 msgr2=0x7f2bd8210400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2bd820fff0 0x7f2bd8210400 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f2bd8210f60 msgr2=0x7f2bd8211370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f2bd8210f60 0x7f2bd8211370 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2bb80744b0 msgr2=0x7f2bb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2bb80744b0 0x7f2bb80769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005880 tx=0x7f2bcc009040 comp rx=0 tx=0).stop 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 reap_dead start 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd80ca770 msgr2=0x7f2bd814d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bca000640 1 --2- 172.21.15.123:0/4030856468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd80ca770 0x7f2bd814d2e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2bd002acc0 tx=0x7f2bd0004a40 comp rx=0 tx=0).stop 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 reap_dead start 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 reap_dead start 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 reap_dead start 2024-04-25T12:36:16.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.172+0000 7f2bdd600640 1 -- 172.21.15.123:0/4030856468 reap_dead start 2024-04-25T12:36:16.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.173+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 shutdown_connections 2024-04-25T12:36:16.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.173+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 >> 172.21.15.123:0/4030856468 conn(0x7f2bd806f440 msgr2=0x7f2bd80b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:16.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.174+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 shutdown_connections 2024-04-25T12:36:16.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.174+0000 7f2bca000640 1 -- 172.21.15.123:0/4030856468 wait complete. 2024-04-25T12:36:16.199 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:16.199 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:16.199 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:16.199 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:16.199 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:16.207 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:16.253 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:16.253+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:16.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.295+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fdd24076700 con 0x7fdd2415a730 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.372980372, 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:36:16.325 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.330 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.372892718, 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:16.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:36:16.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:36:16.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.335 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.37254815, 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:36:16.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:36:16.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:36:16.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.340 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.37247567999999, 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.341 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:36:16.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.372401702, 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.350 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.37036433, 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:36:16.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:36:16.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:36:16.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.369504194, 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.208818539000006, 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.359 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:36:16.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:36:16.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:36:16.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.208742424999997, 2024-04-25T12:36:16.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:36:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:36:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.208696755999995, 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:36:16.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:36:16.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.208651981000003, 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:16.378 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.202035557000002, 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:16.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.380 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:36:16.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.201976067000004, 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.171005774999998, 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.386 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:36:16.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:36:16.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:36:16.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:36:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.170928263999997, 2024-04-25T12:36:16.391 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.396 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.170880414999999, 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:16.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:36:16.399 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:36:16.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.399 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:36:16.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.404 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.405 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.170838524999994, 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:16.406 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:36:16.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.170804352999994, 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.411 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:36:16.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:36:16.413 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:36:16.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:36:16.415 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.170521610999998, 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.416 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.417 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.419 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 81.166381180000002, 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:16.420 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:36:16.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:36:16.422 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.423 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.423 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.423 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:16.423 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:16.423 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:16.423 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:16.424 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:16.407+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:16.507 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:16.507+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:16.559 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:16.560 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:16.560 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:16.560 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:16.569 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:16.611 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:16.608+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:16.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.672+0000 7fdd1b400640 1 -- 172.21.15.123:0/2683443328 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v55) ==== 97+0+0 (secure 0 0 0) 0x7fdd1c080dd0 con 0x7fdd2415a730 2024-04-25T12:36:16.673 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:36:16.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.675+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdd040744e0 msgr2=0x7fdd040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.675+0000 7fdd2a200640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdd040744e0 0x7fdd040769f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd14007e50 tx=0x7fdd14008040 comp rx=0 tx=0).stop 2024-04-25T12:36:16.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd2415a730 msgr2=0x7fdd2414df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:16.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd2415a730 0x7fdd2414df00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd240c4990 tx=0x7fdd1c004a40 comp rx=0 tx=0).stop 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 shutdown_connections 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdd040744e0 0x7fdd040769f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd2415a730 0x7fdd2414df00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd240c90d0 0x7fdd24159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 --2- 172.21.15.123:0/2683443328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd240c4240 0x7fdd241fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 >> 172.21.15.123:0/2683443328 conn(0x7fdd2406c750 msgr2=0x7fdd24160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 shutdown_connections 2024-04-25T12:36:16.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:16.676+0000 7fdd2a200640 1 -- 172.21.15.123:0/2683443328 wait complete. 2024-04-25T12:36:16.688 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:16.688 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:16.688 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:16.688 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:16.688 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:16.697 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:16.742 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:16.743+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:16.811 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:16.811 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:16.811 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:16.811 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [] 2024-04-25T12:36:16.811 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:16.812 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:16.812+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:16.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:16.913 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:16.914+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:17.015 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:17.015+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:17.116 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:17.117+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:17.173 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:17.173 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:17.173 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:17.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:17.219 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:17.220+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:17.320 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:17.320+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:17.423 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:17.423+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:17.524 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:17.524+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:17.624 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:17.625+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:17.632 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:17.632 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:17.633 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:17.666 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:17.722 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:17.722 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:17.722 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-04-25T12:36:17.722 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-25T12:36:17.722 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-25T12:36:17.723 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-04-25T12:36:17.753 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:17.753+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:17.825 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:17.825+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:17.844 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:17.844 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "" 2024-04-25T12:36:17.844 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:17.926 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:17.927+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:17.981 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:17.981 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 119.02850353700001, 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012628781, 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951638+0000", 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951659+0000", 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1027999999999999e-05 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951691+0000", 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2267000000000001e-05 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951710+0000", 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9063999999999999e-05 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952026+0000", 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000315719 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952367+0000", 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034072600000000002 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955799+0000", 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034325969999999999 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962852+0000", 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070528889999999997 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964076+0000", 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012234119999999999 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964178+0000", 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102166 2024-04-25T12:36:17.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964264+0000", 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6175999999999996e-05 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 119.028365706, 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012746265999999999, 2024-04-25T12:36:17.987 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951774+0000", 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-06 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951782+0000", 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2090000000000005e-06 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951796+0000", 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3559e-05 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951808+0000", 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2183e-05 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956591+0000", 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047823550000000003 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956696+0000", 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105696 2024-04-25T12:36:17.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956907+0000", 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000211165 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962907+0000", 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059995650000000001 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964283+0000", 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013758189999999999 2024-04-25T12:36:17.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964381+0000", 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7977000000000001e-05 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964519+0000", 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138675 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:31 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 119.02719698200001, 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076541370000000001, 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:17.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952943+0000", 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952999+0000", 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5968999999999999e-05 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:17.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953032+0000", 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2598999999999997e-05 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953058+0000", 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6585999999999999e-05 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953207+0000", 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014848300000000001 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:17.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953263+0000", 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6005999999999999e-05 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960434+0000", 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071716219999999999 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960596+0000", 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016173899999999999 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:17.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 119.02701506299999, 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010940785, 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953125+0000", 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.22e-06 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953138+0000", 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2996000000000001e-05 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953158+0000", 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0173000000000002e-05 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953176+0000", 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7927e-05 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954316+0000", 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011402599999999999 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:17.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954375+0000", 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963968+0000", 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095929269999999994 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964065+0000", 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6628999999999999e-05 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.000 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 119.026916239, 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080627230000000008, 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:18.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953225+0000", 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3640000000000002e-06 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953259+0000", 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3955999999999998e-05 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953292+0000", 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3549e-05 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953320+0000", 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7892e-05 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953495+0000", 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017444800000000001 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953705+0000", 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021011000000000001 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953862+0000", 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015660399999999999 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959861+0000", 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059993959999999997 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961071+0000", 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012101900000000001 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961192+0000", 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121279 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961285+0000", 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2935000000000004e-05 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 119.026739502, 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010652488999999999, 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.006 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953401+0000", 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.513e-06 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953412+0000", 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209000000000001e-05 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953473+0000", 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0668999999999998e-05 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953496+0000", 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3246000000000001e-05 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956356+0000", 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00286002 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956620+0000", 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026411099999999998 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.957005+0000", 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038473499999999998 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962757+0000", 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057517080000000003 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963927+0000", 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00117023 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964002+0000", 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5483999999999998e-05 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964052+0000", 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9564000000000002e-05 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 119.02661105200001, 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011827087999999999, 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.011 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953529+0000", 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953594+0000", 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4807999999999993e-05 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953617+0000", 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3220000000000001e-05 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953683+0000", 2024-04-25T12:36:18.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5328999999999997e-05 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954191+0000", 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000508428 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040280099999999999 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954922+0000", 2024-04-25T12:36:18.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032838300000000002 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960432+0000", 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055099989999999998 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965226+0000", 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047938700000000004 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965297+0000", 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0992000000000001e-05 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965355+0000", 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7695000000000002e-05 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:19 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.863707251999998, 2024-04-25T12:36:18.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010949354999999999, 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116434+0000", 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116475+0000", 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1486000000000002e-05 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116505+0000", 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9709e-05 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116529+0000", 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3839999999999999e-05 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116651+0000", 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122013 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116685+0000", 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4513e-05 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127273+0000", 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010587653000000001 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127381+0000", 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108204 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:20 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:18.020 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.863552124999998, 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014061374, 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116588+0000", 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0499999999999999e-06 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116602+0000", 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3793e-05 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116621+0000", 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9545999999999999e-05 2024-04-25T12:36:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116637+0000", 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5696e-05 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117243+0000", 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060668400000000002 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117280+0000", 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6619000000000003e-05 2024-04-25T12:36:18.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130559+0000", 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013278570999999999 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130648+0000", 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9414999999999997e-05 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:27 2.9 e25/18 2:91bd965c:::501.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.863125475000004, 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013688128000000001, 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117015+0000", 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3200000000000001e-06 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117121+0000", 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106451 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117145+0000", 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.404e-05 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117210+0000", 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4314999999999997e-05 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120564+0000", 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033548050000000002 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120624+0000", 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9234999999999998e-05 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130654+0000", 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010030437999999999 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130702+0000", 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7524000000000001e-05 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.028 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:18 2.13 e25/18 2:ca4dd7f5:::611.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.862772175999993, 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010076254, 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117369+0000", 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.88e-06 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117385+0000", 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6670000000000001e-05 2024-04-25T12:36:18.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117413+0000", 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7691999999999998e-05 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117435+0000", 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2416e-05 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117549+0000", 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113436 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1509e-05 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127347+0000", 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097767759999999992 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127443+0000", 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5874999999999999e-05 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.032 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.861560503000007, 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012295627, 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118582+0000", 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000001e-06 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118599+0000", 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7176e-05 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:18.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2505999999999997e-05 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118659+0000", 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7559000000000002e-05 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120040+0000", 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001380956 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120305+0000", 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026497700000000001 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120707+0000", 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040189499999999999 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128035+0000", 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073284049999999996 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130711+0000", 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026752910000000002 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130787+0000", 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6384999999999995e-05 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130874+0000", 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7020000000000004e-05 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:18.037 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.861404503000003, 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012319366, 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118747+0000", 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1286e-05 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118772+0000", 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5150000000000001e-05 2024-04-25T12:36:18.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118793+0000", 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0778999999999999e-05 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121911+0000", 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031179409999999999 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122331+0000", 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041955999999999997 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124441+0000", 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021107629999999999 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130205+0000", 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057636140000000002 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130888+0000", 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068289100000000003 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130961+0000", 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3026000000000002e-05 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131054+0000", 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2855999999999993e-05 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:21 2.8 2:1470b2f8:::613.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:18.042 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.861315774999994, 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012409474, 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:18.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118824+0000", 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118834+0000", 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118855+0000", 2024-04-25T12:36:18.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999999e-05 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118875+0000", 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9809999999999998e-05 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119612+0000", 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073729299999999998 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119912+0000", 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029931000000000001 2024-04-25T12:36:18.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120202+0000", 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000289907 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130342+0000", 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010140188 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131091+0000", 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074893900000000001 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131166+0000", 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5272999999999996e-05 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131233+0000", 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6625000000000007e-05 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.047 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.861207027999995, 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012992702, 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118933+0000", 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3069999999999999e-06 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118943+0000", 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7200000000000001e-06 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118967+0000", 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3828999999999998e-05 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118986+0000", 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9057000000000001e-05 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124788+0000", 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00580271 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125092+0000", 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030394999999999999 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125479+0000", 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038660000000000002 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131299+0000", 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058200559999999997 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131823+0000", 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052382700000000004 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131888+0000", 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4740999999999996e-05 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131925+0000", 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6905e-05 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:5 2.7 e25/18 2:e455cf45:::60b.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:18.053 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 87.861083579999999, 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011712752, 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119057+0000", 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.296e-06 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119068+0000", 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.189e-05 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119091+0000", 2024-04-25T12:36:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2802999999999998e-05 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119110+0000", 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8473000000000001e-05 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121085+0000", 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019750689999999999 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121149+0000", 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3979000000000005e-05 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130709+0000", 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009559817 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130768+0000", 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9425000000000001e-05 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:21 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.825294291999995, 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066623919999999996, 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154846+0000", 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.06e-06 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154856+0000", 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0713000000000001e-05 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154878+0000", 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1180000000000001e-05 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154896+0000", 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8343e-05 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155332+0000", 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043663000000000003 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155389+0000", 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6818999999999999e-05 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161381+0000", 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059914440000000003 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161507+0000", 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126203 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:22 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.825020520999999, 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064678330000000001, 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155120+0000", 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155131+0000", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1793e-05 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155155+0000", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3306e-05 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155174+0000", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9383000000000001e-05 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157101+0000", 2024-04-25T12:36:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019271570000000001 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157161+0000", 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9945000000000003e-05 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161512+0000", 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043506389999999999 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161586+0000", 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4250000000000002e-05 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:26 2.11 2:893cd530:::mds2_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.824083548000004, 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064127289999999998, 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156057+0000", 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8330000000000001e-06 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156090+0000", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2719000000000003e-05 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156116+0000", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6531999999999999e-05 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156140+0000", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3453999999999999e-05 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156350+0000", 2024-04-25T12:36:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021060999999999999 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156467+0000", 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116823 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156594+0000", 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012698900000000001 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161255+0000", 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046609570000000003 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162360+0000", 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001105167 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162416+0000", 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6076999999999999e-05 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162468+0000", 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1567999999999998e-05 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 82.823932898999999, 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066007829999999998, 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156208+0000", 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5570000000000001e-06 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156216+0000", 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0900000000000005e-06 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156237+0000", 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1452999999999999e-05 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156255+0000", 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8091999999999999e-05 2024-04-25T12:36:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156841+0000", 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00058573200000000005 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156983+0000", 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000141676 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162807+0000", 2024-04-25T12:36:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058241830000000001 2024-04-25T12:36:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:18.075 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:18.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:18.077 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:18.076+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:18.177 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:18.177+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:18.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:18.278 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:18.277+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:18.378 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:18.378+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:18.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.446+0000 7f8e4f000640 1 Processor -- start 2024-04-25T12:36:18.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.446+0000 7f8e4f000640 1 -- start start 2024-04-25T12:36:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4f000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e481543f0 0x7f8e480cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4f000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e480c4500 0x7f8e480d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4f000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4815b0b0 con 0x7f8e480b23b0 2024-04-25T12:36:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4f000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8e480cb370 con 0x7f8e480c4500 2024-04-25T12:36:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4f000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8e480cc010 con 0x7f8e481543f0 2024-04-25T12:36:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4d200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e480c4500 0x7f8e480d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e481543f0 0x7f8e480cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4d200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e480c4500 0x7f8e480d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:56530/0 (socket says 172.21.15.123:56530) 2024-04-25T12:36:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.448+0000 7f8e4d200640 1 -- 172.21.15.123:0/277337032 learned_addr learned my addr 172.21.15.123:0/277337032 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.449+0000 7f8e4dc00640 1 -- 172.21.15.123:0/277337032 >> v2:172.21.15.153:3300/0 conn(0x7f8e480c4500 msgr2=0x7f8e480d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.449+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/277337032 >> v2:172.21.15.153:3300/0 conn(0x7f8e480c4500 0x7f8e480d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.449+0000 7f8e4dc00640 1 -- 172.21.15.123:0/277337032 >> v1:172.21.15.123:6789/0 conn(0x7f8e480b23b0 legacy=0x7f8e480cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.449+0000 7f8e4dc00640 1 -- 172.21.15.123:0/277337032 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e480cadb0 con 0x7f8e481543f0 2024-04-25T12:36:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.449+0000 7f8e4d200640 1 --2- 172.21.15.123:0/277337032 >> v2:172.21.15.153:3300/0 conn(0x7f8e480c4500 0x7f8e480d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.451+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/277337032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e481543f0 0x7f8e480cfb50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00dbc0 tx=0x7f8e3c02e510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=250bba73b5cda92f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.451+0000 7f8e47e00640 1 -- 172.21.15.123:0/277337032 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c03a070 con 0x7f8e481543f0 2024-04-25T12:36:18.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.451+0000 7f8e47e00640 1 -- 172.21.15.123:0/277337032 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00b650 con 0x7f8e481543f0 2024-04-25T12:36:18.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.451+0000 7f8e47e00640 1 -- 172.21.15.123:0/277337032 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00ba20 con 0x7f8e481543f0 2024-04-25T12:36:18.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 -- 172.21.15.123:0/277337032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e481543f0 msgr2=0x7f8e480cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 --2- 172.21.15.123:0/277337032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e481543f0 0x7f8e480cfb50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00dbc0 tx=0x7f8e3c02e510 comp rx=0 tx=0).stop 2024-04-25T12:36:18.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 -- 172.21.15.123:0/277337032 shutdown_connections 2024-04-25T12:36:18.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 --2- 172.21.15.123:0/277337032 >> v2:172.21.15.153:3300/0 conn(0x7f8e480c4500 0x7f8e480d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 --2- 172.21.15.123:0/277337032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e481543f0 0x7f8e480cfb50 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00dbc0 tx=0x7f8e3c02e510 comp rx=0 tx=0).stop 2024-04-25T12:36:18.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 -- 172.21.15.123:0/277337032 >> 172.21.15.123:0/277337032 conn(0x7f8e48068810 msgr2=0x7f8e48068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:18.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 -- 172.21.15.123:0/277337032 shutdown_connections 2024-04-25T12:36:18.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.452+0000 7f8e4f000640 1 -- 172.21.15.123:0/277337032 wait complete. 2024-04-25T12:36:18.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.455+0000 7f8e4f000640 1 Processor -- start 2024-04-25T12:36:18.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.455+0000 7f8e4f000640 1 -- start start 2024-04-25T12:36:18.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.456+0000 7f8e4f000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e480b23b0 0x7f8e48157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.456+0000 7f8e4f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e480c4500 0x7f8e481494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.456+0000 7f8e4f000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e4815d320 0x7f8e48155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.456+0000 7f8e4f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e480cb590 con 0x7f8e480c4500 2024-04-25T12:36:18.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.456+0000 7f8e4f000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e480cbca0 con 0x7f8e4815d320 2024-04-25T12:36:18.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e480c4500 0x7f8e481494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e480c4500 0x7f8e481494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44302/0 (socket says 172.21.15.123:44302) 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 -- 172.21.15.123:0/4004887397 learned_addr learned my addr 172.21.15.123:0/4004887397 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e4815d320 0x7f8e48155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8e480cb710 con 0x7f8e480b23b0 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e480b23b0 0x7f8e48157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e480b23b0 msgr2=0x7f8e48157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e480b23b0 0x7f8e48157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e4815d320 msgr2=0x7f8e48155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e4815d320 0x7f8e48155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4d200640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e48155b50 con 0x7f8e480c4500 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.457+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e4815d320 0x7f8e48155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.458+0000 7f8e4d200640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e480c4500 0x7f8e481494f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e8d0 tx=0x7f8e3400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.458+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34004030 con 0x7f8e480c4500 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.458+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4815d9a0 con 0x7f8e480c4500 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.458+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e340041d0 con 0x7f8e480c4500 2024-04-25T12:36:18.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.458+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400c940 con 0x7f8e480c4500 2024-04-25T12:36:18.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.459+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e481550f0 con 0x7f8e480c4500 2024-04-25T12:36:18.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.461+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8e48076da0 con 0x7f8e480c4500 2024-04-25T12:36:18.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.462+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e3400cae0 con 0x7f8e480c4500 2024-04-25T12:36:18.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.463+0000 7f8e45600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e280744b0 0x7f8e280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.463+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6648+0+0 (secure 0 0 0) 0x7f8e34095800 con 0x7f8e480c4500 2024-04-25T12:36:18.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.463+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v55) ==== 54+0+16 (secure 0 0 0) 0x7f8e34017020 con 0x7f8e480c4500 2024-04-25T12:36:18.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.463+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e280744b0 0x7f8e280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.464+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8e48072860 0x7f8e48072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.464+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e48156020 con 0x7f8e48072860 2024-04-25T12:36:18.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.464+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8e48072860 0x7f8e48072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.466+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e280744b0 0x7f8e280769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c037000 tx=0x7f8e3c00d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.466+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8e48072860 0x7f8e48072c70 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.466+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e48156020 con 0x7f8e48072860 2024-04-25T12:36:18.470 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:18.470 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:18.470 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:18.470 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:18.479 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:18.480+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:18.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.482+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e481f22c0 con 0x7f8e48072860 2024-04-25T12:36:18.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.484+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e481f22c0 con 0x7f8e48072860 2024-04-25T12:36:18.484 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:18.485 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:18.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.484+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8e481ee500 0x7f8e481ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.485+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8e481f2510 con 0x7f8e481ee500 2024-04-25T12:36:18.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.485+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8e481ee500 0x7f8e481ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.486+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8e481ee500 0x7f8e481ee910 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.487+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e481f2510 con 0x7f8e481ee500 2024-04-25T12:36:18.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.503+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e48149a30 con 0x7f8e481ee500 2024-04-25T12:36:18.507 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.506+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e3c03b040 con 0x7f8e481ee500 2024-04-25T12:36:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.507+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8e480634f0 0x7f8e48063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.507+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8e481f1f90 con 0x7f8e480634f0 2024-04-25T12:36:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.507+0000 7f8e4d200640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8e480634f0 0x7f8e48063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.508+0000 7f8e4d200640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8e480634f0 0x7f8e48063900 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.509+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e481f1f90 con 0x7f8e480634f0 2024-04-25T12:36:18.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.524+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e480cb8e0 con 0x7f8e480634f0 2024-04-25T12:36:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.526+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e480cb8e0 con 0x7f8e480634f0 2024-04-25T12:36:18.526 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.526+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8e48152bb0 0x7f8e4802c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.526+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8e481eee50 con 0x7f8e48152bb0 2024-04-25T12:36:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.527+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8e48152bb0 0x7f8e4802c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.528+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8e48152bb0 0x7f8e4802c9d0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.529+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e481eee50 con 0x7f8e48152bb0 2024-04-25T12:36:18.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.544+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e48153290 con 0x7f8e48152bb0 2024-04-25T12:36:18.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.546+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e48153290 con 0x7f8e48152bb0 2024-04-25T12:36:18.546 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:18.547 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:18.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.546+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8e4802d420 0x7f8e4802d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.546+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8e480cf160 con 0x7f8e4802d420 2024-04-25T12:36:18.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.547+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8e4802d420 0x7f8e4802d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.547+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8e4802d420 0x7f8e4802d830 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.548+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e480cf160 con 0x7f8e4802d420 2024-04-25T12:36:18.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.564+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e480cf160 con 0x7f8e4802d420 2024-04-25T12:36:18.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.566+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e480cf160 con 0x7f8e4802d420 2024-04-25T12:36:18.566 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:18.567 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:18.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.566+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8e48202e90 0x7f8e482052b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.566+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8e4802ddf0 con 0x7f8e48202e90 2024-04-25T12:36:18.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.567+0000 7f8e4d200640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8e48202e90 0x7f8e482052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.567+0000 7f8e4d200640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8e48202e90 0x7f8e482052b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.568+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e4802ddf0 con 0x7f8e48202e90 2024-04-25T12:36:18.580 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:18.581+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:18.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.583+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e480008d0 con 0x7f8e48202e90 2024-04-25T12:36:18.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.585+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e480008d0 con 0x7f8e48202e90 2024-04-25T12:36:18.585 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:18.586 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:18.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.586+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8e48205d00 0x7f8e48208120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.586+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8e4820b050 con 0x7f8e48205d00 2024-04-25T12:36:18.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.586+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8e48205d00 0x7f8e48208120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.587+0000 7f8e4e600640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8e48205d00 0x7f8e48208120 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.587+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e4820b050 con 0x7f8e48205d00 2024-04-25T12:36:18.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.602+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e4820b020 con 0x7f8e48205d00 2024-04-25T12:36:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.605+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e4820b020 con 0x7f8e48205d00 2024-04-25T12:36:18.605 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.605+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8e48208b70 0x7f8e48208f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.605+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8e4820e420 con 0x7f8e48208b70 2024-04-25T12:36:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.605+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8e48208b70 0x7f8e48208f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.606+0000 7f8e4dc00640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8e48208b70 0x7f8e48208f80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.607+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e4820e420 con 0x7f8e48208b70 2024-04-25T12:36:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.622+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8e482095d0 con 0x7f8e48208b70 2024-04-25T12:36:18.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e45600640 1 -- 172.21.15.123:0/4004887397 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8e482095d0 con 0x7f8e48208b70 2024-04-25T12:36:18.624 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8e48072860 msgr2=0x7f8e48072c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8e48072860 0x7f8e48072c70 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8e481ee500 msgr2=0x7f8e481ee910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8e481ee500 0x7f8e481ee910 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8e480634f0 msgr2=0x7f8e48063900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8e480634f0 0x7f8e48063900 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8e48152bb0 msgr2=0x7f8e4802c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8e48152bb0 0x7f8e4802c9d0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8e4802d420 msgr2=0x7f8e4802d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8e4802d420 0x7f8e4802d830 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8e48202e90 msgr2=0x7f8e482052b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8e48202e90 0x7f8e482052b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8e48205d00 msgr2=0x7f8e48208120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8e48205d00 0x7f8e48208120 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8e48208b70 msgr2=0x7f8e48208f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.624+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8e48208b70 0x7f8e48208f80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e280744b0 msgr2=0x7f8e280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e280744b0 0x7f8e280769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c037000 tx=0x7f8e3c00d000 comp rx=0 tx=0).stop 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e480c4500 msgr2=0x7f8e481494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4f000640 1 --2- 172.21.15.123:0/4004887397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e480c4500 0x7f8e481494f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e8d0 tx=0x7f8e3400ed90 comp rx=0 tx=0).stop 2024-04-25T12:36:18.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4e600640 1 -- 172.21.15.123:0/4004887397 reap_dead start 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4e600640 1 -- 172.21.15.123:0/4004887397 reap_dead start 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4e600640 1 -- 172.21.15.123:0/4004887397 reap_dead start 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4e600640 1 -- 172.21.15.123:0/4004887397 reap_dead start 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4e600640 1 -- 172.21.15.123:0/4004887397 reap_dead start 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 shutdown_connections 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.625+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 >> 172.21.15.123:0/4004887397 conn(0x7f8e48068810 msgr2=0x7f8e480cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.626+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 shutdown_connections 2024-04-25T12:36:18.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.626+0000 7f8e4f000640 1 -- 172.21.15.123:0/4004887397 wait complete. 2024-04-25T12:36:18.702 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:36:18.703 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:18.702+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:18.768 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:18.769 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:18.782 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:18.782+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:18.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:18.883 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:18.883+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:18.985 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:18.985+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:18.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.987+0000 7fbdb4c00640 1 Processor -- start 2024-04-25T12:36:18.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.987+0000 7fbdb4c00640 1 -- start start 2024-04-25T12:36:18.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.989+0000 7fbdb4c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.989+0000 7fbdb4c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbdb015ad00 0x7fbdb00cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.989+0000 7fbdb4c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb00ca730 con 0x7fbdb00cc420 2024-04-25T12:36:18.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.989+0000 7fbdb4c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb00c4d20 con 0x7fbdb015ad00 2024-04-25T12:36:18.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.989+0000 7fbdb4c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb00c5000 con 0x7fbdb00cba00 2024-04-25T12:36:18.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdafe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbdb015ad00 0x7fbdb00cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdafe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbdb015ad00 0x7fbdb00cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:56558/0 (socket says 172.21.15.123:56558) 2024-04-25T12:36:18.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdafe00640 1 -- 172.21.15.123:0/1768905131 learned_addr learned my addr 172.21.15.123:0/1768905131 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:18.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdaf400640 1 --2- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdafe00640 1 -- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 msgr2=0x7fbdb00ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:18.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdafe00640 1 --2- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdafe00640 1 -- 172.21.15.123:0/1768905131 >> v1:172.21.15.123:6789/0 conn(0x7fbdb00cc420 legacy=0x7fbdb015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.990+0000 7fbdafe00640 1 -- 172.21.15.123:0/1768905131 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb00c52e0 con 0x7fbdb015ad00 2024-04-25T12:36:18.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.992+0000 7fbdafe00640 1 --2- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb015ad00 0x7fbdb00cc830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009e50 tx=0x7fbda802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b68b5a6213923cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:18.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.992+0000 7fbdae000640 1 -- 172.21.15.123:0/1768905131 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8036070 con 0x7fbdb015ad00 2024-04-25T12:36:18.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.992+0000 7fbdae000640 1 -- 172.21.15.123:0/1768905131 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda8031600 con 0x7fbdb015ad00 2024-04-25T12:36:18.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.992+0000 7fbdae000640 1 -- 172.21.15.123:0/1768905131 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80319d0 con 0x7fbdb015ad00 2024-04-25T12:36:18.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 -- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb015ad00 msgr2=0x7fbdb00cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb015ad00 0x7fbdb00cc830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009e50 tx=0x7fbda802a710 comp rx=0 tx=0).stop 2024-04-25T12:36:18.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 -- 172.21.15.123:0/1768905131 shutdown_connections 2024-04-25T12:36:18.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb015ad00 0x7fbdb00cc830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/1768905131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 -- 172.21.15.123:0/1768905131 >> 172.21.15.123:0/1768905131 conn(0x7fbdb0072cb0 msgr2=0x7fbdb00730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:18.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 -- 172.21.15.123:0/1768905131 shutdown_connections 2024-04-25T12:36:18.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.993+0000 7fbdb4c00640 1 -- 172.21.15.123:0/1768905131 wait complete. 2024-04-25T12:36:18.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.996+0000 7fbdb4c00640 1 Processor -- start 2024-04-25T12:36:18.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.996+0000 7fbdb4c00640 1 -- start start 2024-04-25T12:36:18.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.997+0000 7fbdb4c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.997+0000 7fbdb4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb00cc420 0x7fbdb00c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.997+0000 7fbdb4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdb015ad00 0x7fbdb0144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:18.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.997+0000 7fbdb4c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb00c66f0 con 0x7fbdb015ad00 2024-04-25T12:36:18.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.997+0000 7fbdb4c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb00c58a0 con 0x7fbdb00cc420 2024-04-25T12:36:18.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.997+0000 7fbdb4c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb0154540 con 0x7fbdb00cba00 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaf400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb00cc420 0x7fbdb00c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaf400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:60066/0 (socket says 172.21.15.123:60066) 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb00cc420 0x7fbdb00c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:56566/0 (socket says 172.21.15.123:56566) 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaf400640 1 -- 172.21.15.123:0/327416537 learned_addr learned my addr 172.21.15.123:0/327416537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdafe00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdb015ad00 0x7fbdb0144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaea00640 1 -- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 msgr2=0x7fbdb00c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaea00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaea00640 1 -- 172.21.15.123:0/327416537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdb015ad00 msgr2=0x7fbdb0144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaea00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdb015ad00 0x7fbdb0144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdaea00640 1 -- 172.21.15.123:0/327416537 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb01fb360 con 0x7fbdb00cc420 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.998+0000 7fbdafe00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdb015ad00 0x7fbdb0144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:18.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.999+0000 7fbdaea00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb00cc420 0x7fbdb00c7bc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c3c0 tx=0x7fbd9c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:19.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.999+0000 7fbda7400640 1 -- 172.21.15.123:0/327416537 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c017070 con 0x7fbdb00cc420 2024-04-25T12:36:19.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.999+0000 7fbda7400640 1 -- 172.21.15.123:0/327416537 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c0052d0 con 0x7fbdb00cc420 2024-04-25T12:36:19.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.999+0000 7fbda7400640 1 -- 172.21.15.123:0/327416537 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c003830 con 0x7fbdb00cc420 2024-04-25T12:36:19.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.999+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb00c90d0 con 0x7fbdb00cc420 2024-04-25T12:36:19.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:18.999+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb01fa880 con 0x7fbdb00cc420 2024-04-25T12:36:19.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.001+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb0076da0 con 0x7fbdb00cc420 2024-04-25T12:36:19.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.002+0000 7fbda7400640 1 -- 172.21.15.123:0/327416537 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd9c01a070 con 0x7fbdb00cc420 2024-04-25T12:36:19.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.003+0000 7fbda7400640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbd900744e0 0x7fbd900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:19.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.003+0000 7fbda7400640 1 -- 172.21.15.123:0/327416537 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6648+0+0 (secure 0 0 0) 0x7fbd9c094780 con 0x7fbdb00cc420 2024-04-25T12:36:19.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.003+0000 7fbdaf400640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbd900744e0 0x7fbd900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:19.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.004+0000 7fbdaf400640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbd900744e0 0x7fbd900769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbda0007a60 tx=0x7fbda0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:19.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.008+0000 7fbda7400640 1 -- 172.21.15.123:0/327416537 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd9c05e590 con 0x7fbdb00cc420 2024-04-25T12:36:19.086 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:19.086+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:19.186 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:19.186+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:19.196 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:19.197 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:19.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:19.318 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:19.318+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:19.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.329+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fbdb00c9a60 con 0x7fbdb00cc420 2024-04-25T12:36:19.388 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:19.388+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:19.489 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:19.489+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:19.590 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:19.590+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:19.594 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:19.594 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:19.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:19.691 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:19.690+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:19.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.693+0000 7fbda7400640 1 -- 172.21.15.123:0/327416537 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v56) ==== 91+0+0 (secure 0 0 0) 0x7fbd9c062890 con 0x7fbdb00cc420 2024-04-25T12:36:19.696 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:36:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbd900744e0 msgr2=0x7fbd900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbd900744e0 0x7fbd900769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbda0007a60 tx=0x7fbda0008040 comp rx=0 tx=0).stop 2024-04-25T12:36:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb00cc420 msgr2=0x7fbdb00c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb00cc420 0x7fbdb00c7bc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c3c0 tx=0x7fbd9c00c880 comp rx=0 tx=0).stop 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 shutdown_connections 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fbd900744e0 0x7fbd900769f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdb015ad00 0x7fbdb0144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb00cc420 0x7fbdb00c7bc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 --2- 172.21.15.123:0/327416537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbdb00cba00 0x7fbdb00c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 >> 172.21.15.123:0/327416537 conn(0x7fbdb0072cb0 msgr2=0x7fbdb01e4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 shutdown_connections 2024-04-25T12:36:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:19.698+0000 7fbdb4c00640 1 -- 172.21.15.123:0/327416537 wait complete. 2024-04-25T12:36:19.792 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:19.792+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:19.893 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:19.893+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:19.993 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:19.993+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:20.052 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:20.052 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:20.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:20.124 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:20.124+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:20.194 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:20.194+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:20.295 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:20.295+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:20.396 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:20.396+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:20.497 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:20.497+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:20.519 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:20.519 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:20.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:20.598 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:20.598+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:20.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:20.699 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:20.698+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:20.800 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:20.800+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:20.901 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:20.901+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:20.937 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:20.937 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:20.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.950+0000 7feafa600640 1 Processor -- start 2024-04-25T12:36:20.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.950+0000 7feafa600640 1 -- start start 2024-04-25T12:36:20.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.953+0000 7feafa600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf40c8170 0x7feaf40ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:20.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.953+0000 7feafa600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf4074400 con 0x7feaf40c8fd0 2024-04-25T12:36:20.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.953+0000 7feafa600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf41595e0 con 0x7feaf41426a0 2024-04-25T12:36:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.953+0000 7feafa600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feaf4074670 con 0x7feaf40c8170 2024-04-25T12:36:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.953+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf40c8170 0x7feaf40ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.953+0000 7feaf2a00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7feaf41426a0 0x7feaf4159dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:37070/0 (socket says 172.21.15.123:37070) 2024-04-25T12:36:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.953+0000 7feaf2a00640 1 -- 172.21.15.123:0/3548415505 learned_addr learned my addr 172.21.15.123:0/3548415505 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.954+0000 7feaf2000640 1 -- 172.21.15.123:0/3548415505 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065256962 0 0) 0x7feaf41595e0 con 0x7feaf41426a0 2024-04-25T12:36:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.954+0000 7feaf3e00640 1 -- 172.21.15.123:0/3548415505 >> v1:172.21.15.153:6789/0 conn(0x7feaf41426a0 legacy=0x7feaf4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:20.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.954+0000 7feaf3e00640 1 -- 172.21.15.123:0/3548415505 >> v1:172.21.15.123:6789/0 conn(0x7feaf40c8fd0 legacy=0x7feaf4153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:20.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.955+0000 7feaf3e00640 1 -- 172.21.15.123:0/3548415505 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf4074950 con 0x7feaf40c8170 2024-04-25T12:36:20.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.956+0000 7feaf3e00640 1 --2- 172.21.15.123:0/3548415505 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf40c8170 0x7feaf40ca200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feaec009dc0 tx=0x7feaec02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec5d4ea41250382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.956+0000 7feaf2000640 1 -- 172.21.15.123:0/3548415505 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec007740 con 0x7feaf40c8170 2024-04-25T12:36:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.956+0000 7feaf2000640 1 -- 172.21.15.123:0/3548415505 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec0078e0 con 0x7feaf40c8170 2024-04-25T12:36:20.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.956+0000 7feaf2000640 1 -- 172.21.15.123:0/3548415505 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec007cb0 con 0x7feaf40c8170 2024-04-25T12:36:20.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.957+0000 7feafa600640 1 -- 172.21.15.123:0/3548415505 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf40c8170 msgr2=0x7feaf40ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:20.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.957+0000 7feafa600640 1 --2- 172.21.15.123:0/3548415505 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf40c8170 0x7feaf40ca200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feaec009dc0 tx=0x7feaec02a6d0 comp rx=0 tx=0).stop 2024-04-25T12:36:20.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.957+0000 7feafa600640 1 -- 172.21.15.123:0/3548415505 shutdown_connections 2024-04-25T12:36:20.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.957+0000 7feafa600640 1 --2- 172.21.15.123:0/3548415505 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf40c8170 0x7feaf40ca200 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:20.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.957+0000 7feafa600640 1 -- 172.21.15.123:0/3548415505 >> 172.21.15.123:0/3548415505 conn(0x7feaf406f440 msgr2=0x7feaf406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:20.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.957+0000 7feafa600640 1 -- 172.21.15.123:0/3548415505 shutdown_connections 2024-04-25T12:36:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.957+0000 7feafa600640 1 -- 172.21.15.123:0/3548415505 wait complete. 2024-04-25T12:36:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.960+0000 7feafa600640 1 Processor -- start 2024-04-25T12:36:20.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.960+0000 7feafa600640 1 -- start start 2024-04-25T12:36:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.961+0000 7feafa600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf40c8170 0x7feaf40acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:20.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.961+0000 7feafa600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf40c8fd0 0x7feaf416c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.961+0000 7feafa600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf41426a0 0x7feaf40c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.961+0000 7feafa600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feaf40c8580 con 0x7feaf40c8fd0 2024-04-25T12:36:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.961+0000 7feafa600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feaf40c4880 con 0x7feaf40c8170 2024-04-25T12:36:20.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.961+0000 7feafa600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feaf40c8700 con 0x7feaf41426a0 2024-04-25T12:36:20.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf40c8fd0 0x7feaf416c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:20.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf40c8170 0x7feaf40acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:20.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf41426a0 0x7feaf40c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:20.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf41426a0 0x7feaf40c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:60092/0 (socket says 172.21.15.123:60092) 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf40c8170 0x7feaf40acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:56578/0 (socket says 172.21.15.123:56578) 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf3e00640 1 -- 172.21.15.123:0/4112335084 learned_addr learned my addr 172.21.15.123:0/4112335084 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf3400640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf41426a0 msgr2=0x7feaf40c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.962+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feaf41426a0 0x7feaf40c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feaf3400640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf40c8fd0 msgr2=0x7feaf416c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf40c8fd0 0x7feaf416c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feaf3400640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf416c690 con 0x7feaf40c8170 2024-04-25T12:36:20.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf40c8170 0x7feaf40acca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feae400c3c0 tx=0x7feae400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae40052d0 con 0x7feaf40c8170 2024-04-25T12:36:20.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae4005490 con 0x7feaf40c8170 2024-04-25T12:36:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4007260 con 0x7feaf40c8170 2024-04-25T12:36:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.963+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf4141ff0 con 0x7feaf40c8170 2024-04-25T12:36:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.964+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf40ac4a0 con 0x7feaf40c8170 2024-04-25T12:36:20.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.965+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7feaf4076700 con 0x7feaf40c8170 2024-04-25T12:36:20.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.967+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feae4007400 con 0x7feaf40c8170 2024-04-25T12:36:20.968 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.969+0000 7feaeb400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fead40744e0 0x7fead40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.969+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6648+0+0 (secure 0 0 0) 0x7feae4095490 con 0x7feaf40c8170 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.969+0000 7feaf2a00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fead40744e0 0x7fead40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.969+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v56) ==== 54+0+16 (secure 0 0 0) 0x7feae4095840 con 0x7feaf40c8170 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.969+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7feaf40ac6a0 0x7feaf40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.970+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feaf40cd7f0 con 0x7feaf40ac6a0 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.970+0000 7feaf3e00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7feaf40ac6a0 0x7feaf40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:20.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.970+0000 7feaf2a00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fead40744e0 0x7fead40769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feae0007fd0 tx=0x7feae0007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:20.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.971+0000 7feaf3e00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7feaf40ac6a0 0x7feaf40ca100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:20.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:20.972+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf40cd7f0 con 0x7feaf40ac6a0 2024-04-25T12:36:21.001 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:21.002+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.004+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf4138690 con 0x7feaf40ac6a0 2024-04-25T12:36:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.007+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf4138690 con 0x7feaf40ac6a0 2024-04-25T12:36:21.007 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.007+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7feaf41ff720 0x7feaf4203000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.007+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7feaf41383b0 con 0x7feaf41ff720 2024-04-25T12:36:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.008+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7feaf41ff720 0x7feaf4203000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.009+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7feaf41ff720 0x7feaf4203000 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:21.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.010+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf41383b0 con 0x7feaf41ff720 2024-04-25T12:36:21.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.040+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf40c8d90 con 0x7feaf41ff720 2024-04-25T12:36:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.044+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf40c8d90 con 0x7feaf41ff720 2024-04-25T12:36:21.044 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.044+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7feaf40ab9d0 0x7feaf4062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.044+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7feaf40c4200 con 0x7feaf40ab9d0 2024-04-25T12:36:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.045+0000 7feaf2a00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7feaf40ab9d0 0x7feaf4062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.045+0000 7feaf2a00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7feaf40ab9d0 0x7feaf4062fe0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.047+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf40c4200 con 0x7feaf40ab9d0 2024-04-25T12:36:21.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.076+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf40ab2d0 con 0x7feaf40ab9d0 2024-04-25T12:36:21.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.079+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf40ab2d0 con 0x7feaf40ab9d0 2024-04-25T12:36:21.079 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:21.080 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:21.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.079+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7feaf4063a30 0x7feaf41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:21.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.079+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7feaf40ab2d0 con 0x7feaf4063a30 2024-04-25T12:36:21.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.080+0000 7feaf3e00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7feaf4063a30 0x7feaf41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:21.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.081+0000 7feaf3e00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7feaf4063a30 0x7feaf41545d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:21.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.082+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf40ab2d0 con 0x7feaf4063a30 2024-04-25T12:36:21.102 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:21.102+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:21.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.111+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf40ab2d0 con 0x7feaf4063a30 2024-04-25T12:36:21.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.114+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf40ab2d0 con 0x7feaf4063a30 2024-04-25T12:36:21.114 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:21.115 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:21.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.114+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7feaf4155020 0x7feaf402c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:21.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.115+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7feaf40ab2d0 con 0x7feaf4155020 2024-04-25T12:36:21.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.115+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7feaf4155020 0x7feaf402c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:21.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.116+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7feaf4155020 0x7feaf402c9d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:21.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.117+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf40ab2d0 con 0x7feaf4155020 2024-04-25T12:36:21.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.146+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf4155430 con 0x7feaf4155020 2024-04-25T12:36:21.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.148+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf4155430 con 0x7feaf4155020 2024-04-25T12:36:21.148 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:21.149 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:21.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.149+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7feaf402d420 0x7feaf402d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:21.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.149+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7feaf402dd70 con 0x7feaf402d420 2024-04-25T12:36:21.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.149+0000 7feaf2a00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7feaf402d420 0x7feaf402d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:21.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.150+0000 7feaf2a00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7feaf402d420 0x7feaf402d830 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:21.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.151+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf402dd70 con 0x7feaf402d420 2024-04-25T12:36:21.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.180+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf402dd70 con 0x7feaf402d420 2024-04-25T12:36:21.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.182+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf402dd70 con 0x7feaf402d420 2024-04-25T12:36:21.183 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:21.183 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:21.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.183+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7feaf42163d0 0x7feaf42187f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:21.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.183+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7feaf421b670 con 0x7feaf42163d0 2024-04-25T12:36:21.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.183+0000 7feaf3e00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7feaf42163d0 0x7feaf42187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:21.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.184+0000 7feaf3e00640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7feaf42163d0 0x7feaf42187f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:21.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.185+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf421b670 con 0x7feaf42163d0 2024-04-25T12:36:21.203 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:21.203+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:21.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.214+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf4219000 con 0x7feaf42163d0 2024-04-25T12:36:21.214 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:21.214 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:21.214 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:21.215 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:21.215 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:21.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.216+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf4219000 con 0x7feaf42163d0 2024-04-25T12:36:21.217 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:21.217 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:21.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.217+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7feaf4219720 0x7feaf4219b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:21.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.217+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7feaf421eac0 con 0x7feaf4219720 2024-04-25T12:36:21.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.217+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7feaf4219720 0x7feaf4219b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:21.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.218+0000 7feaf3400640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7feaf4219720 0x7feaf4219b30 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.219+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf421eac0 con 0x7feaf4219720 2024-04-25T12:36:21.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:21.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.248+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7feaf421d8c0 con 0x7feaf4219720 2024-04-25T12:36:21.253 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:21.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.250+0000 7feaeb400640 1 -- 172.21.15.123:0/4112335084 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feaf421d8c0 con 0x7feaf4219720 2024-04-25T12:36:21.253 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:21.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7feaf40ac6a0 msgr2=0x7feaf40ca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7feaf40ac6a0 0x7feaf40ca100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7feaf41ff720 msgr2=0x7feaf4203000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7feaf41ff720 0x7feaf4203000 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7feaf40ab9d0 msgr2=0x7feaf4062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7feaf40ab9d0 0x7feaf4062fe0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7feaf4063a30 msgr2=0x7feaf41545d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7feaf4063a30 0x7feaf41545d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7feaf4155020 msgr2=0x7feaf402c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7feaf4155020 0x7feaf402c9d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7feaf402d420 msgr2=0x7feaf402d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7feaf402d420 0x7feaf402d830 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7feaf42163d0 msgr2=0x7feaf42187f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7feaf42163d0 0x7feaf42187f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7feaf4219720 msgr2=0x7feaf4219b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7feaf4219720 0x7feaf4219b30 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fead40744e0 msgr2=0x7fead40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fead40744e0 0x7fead40769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feae0007fd0 tx=0x7feae0007de0 comp rx=0 tx=0).stop 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf40c8170 msgr2=0x7feaf40acca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feafa600640 1 --2- 172.21.15.123:0/4112335084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feaf40c8170 0x7feaf40acca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feae400c3c0 tx=0x7feae400c880 comp rx=0 tx=0).stop 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feaf3e00640 1 -- 172.21.15.123:0/4112335084 reap_dead start 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feaf3e00640 1 -- 172.21.15.123:0/4112335084 reap_dead start 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feaf3e00640 1 -- 172.21.15.123:0/4112335084 reap_dead start 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feaf3e00640 1 -- 172.21.15.123:0/4112335084 reap_dead start 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.251+0000 7feaf3e00640 1 -- 172.21.15.123:0/4112335084 reap_dead start 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.252+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 shutdown_connections 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.252+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 >> 172.21.15.123:0/4112335084 conn(0x7feaf406f440 msgr2=0x7feaf4137900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.252+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 shutdown_connections 2024-04-25T12:36:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.252+0000 7feafa600640 1 -- 172.21.15.123:0/4112335084 wait complete. 2024-04-25T12:36:21.304 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:21.304+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:21.405 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:21.405+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:21.498 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.54707643, 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.499 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:36:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:36:21.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.546392248, 2024-04-25T12:36:21.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:36:21.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.507 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.54494170700001, 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.511 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.544680892, 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.540830505, 2024-04-25T12:36:21.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:36:21.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:36:21.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:36:21.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.54060813700001, 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:36:21.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:36:21.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:36:21.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:36:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.540254348, 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.529 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.379635399999998, 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:36:21.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:36:21.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:36:21.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:36:21.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.379427907999997, 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.379283418, 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:36:21.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:21.545 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.379151624000002, 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:21.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:36:21.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:36:21.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:36:21.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.379077328999998, 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:36:21.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.378977587999998, 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:36:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:36:21.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:21.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.378908125999999, 2024-04-25T12:36:21.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:36:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:36:21.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:36:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.378808011999993, 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:36:21.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:36:21.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.378710921000007, 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.572 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.378620515999998, 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.577 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.378472543000001, 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.378365138000007, 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 86.344113840000006, 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:36:21.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:36:21.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:36:21.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:36:21.593 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.594 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.594 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.594 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:21.594 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:21.594 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:21.595 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:21.596 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:21.595+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:21.696 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:21.696+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:21.723 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:21.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:21.723 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:21.724 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:21.732 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:21.737 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:36:21.796 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:21.796+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:21.844 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:21.844 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:21.844 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:21.844 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:21.844 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:21.853 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:21.898 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:21.898+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:21.968 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:21.968 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:21.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:21.968 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:21.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.968 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.968 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.01832369500001, 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:36:21.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:36:21.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.01823598599999, 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:21.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:36:21.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.980 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.018110058, 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:21.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:21.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:36:21.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:21.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:36:21.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.01804839899999, 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:36:21.987 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:21.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:36:21.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.991 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.017991636, 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:36:21.993 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:21.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:36:21.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:21.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:21.999 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.017928166, 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.001 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:36:22.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:22.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:36:22.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.005 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.01787621699999, 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.006 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:36:22.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:36:22.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:36:22.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.01782426699999, 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.012 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.01770715799999, 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:36:22.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:36:22.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:22.024 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.017668711, 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:36:22.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:36:22.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 91.852533657999999, 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.029 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:36:22.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:36:22.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:36:22.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 91.852475607000002, 2024-04-25T12:36:22.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:36:22.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:36:22.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 91.852415914000005, 2024-04-25T12:36:22.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:22.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:36:22.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.815495831999996, 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:36:22.047 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:22.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:36:22.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.053 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.815406866000004, 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:36:22.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:36:22.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:36:22.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.059 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.815363024999996, 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:36:22.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:36:22.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.813849121000004, 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:36:22.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:36:22.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:36:22.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.812929905000004, 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:36:22.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.812474551999998, 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.078 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:36:22.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 86.812445573999995, 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:36:22.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.086 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.087 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:22.089 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:22.089 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:22.088+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.978+0000 7f89a4600640 1 Processor -- start 2024-04-25T12:36:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.978+0000 7f89a4600640 1 -- start start 2024-04-25T12:36:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.980+0000 7f89a4600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f899c0c90d0 0x7f899c0c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.980+0000 7f89a4600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f899c15a730 0x7f899c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.980+0000 7f89a4600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f899c0c4810 con 0x7f899c0c4240 2024-04-25T12:36:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.980+0000 7f89a4600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f899c15b080 con 0x7f899c0c90d0 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.980+0000 7f89a4600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f899c0ce380 con 0x7f899c15a730 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.981+0000 7f89a1200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f899c0c4240 0x7f899c0cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36882/0 (socket says 172.21.15.123:36882) 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.981+0000 7f89a1200640 1 -- 172.21.15.123:0/4170258905 learned_addr learned my addr 172.21.15.123:0/4170258905 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.981+0000 7f89a2600640 1 --2- 172.21.15.123:0/4170258905 >> v2:172.21.15.153:3300/0 conn(0x7f899c15a730 0x7f899c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.981+0000 7f89a1c00640 1 --2- 172.21.15.123:0/4170258905 >> v2:172.21.15.123:3300/0 conn(0x7f899c0c90d0 0x7f899c0c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.981+0000 7f89a2600640 1 -- 172.21.15.123:0/4170258905 >> v2:172.21.15.123:3300/0 conn(0x7f899c0c90d0 msgr2=0x7f899c0c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.981+0000 7f89a2600640 1 --2- 172.21.15.123:0/4170258905 >> v2:172.21.15.123:3300/0 conn(0x7f899c0c90d0 0x7f899c0c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.981+0000 7f89a2600640 1 -- 172.21.15.123:0/4170258905 >> v1:172.21.15.123:6789/0 conn(0x7f899c0c4240 legacy=0x7f899c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.982+0000 7f89a2600640 1 -- 172.21.15.123:0/4170258905 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c0c83f0 con 0x7f899c15a730 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.983+0000 7f89a2600640 1 --2- 172.21.15.123:0/4170258905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 0x7f899c15ab40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8994009d90 tx=0x7f899402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dcde45959602b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.983+0000 7f899be00640 1 -- 172.21.15.123:0/4170258905 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994007740 con 0x7f899c15a730 2024-04-25T12:36:22.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.983+0000 7f899be00640 1 -- 172.21.15.123:0/4170258905 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89940078e0 con 0x7f899c15a730 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.983+0000 7f899be00640 1 -- 172.21.15.123:0/4170258905 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994007cb0 con 0x7f899c15a730 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 -- 172.21.15.123:0/4170258905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 msgr2=0x7f899c15ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 --2- 172.21.15.123:0/4170258905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 0x7f899c15ab40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8994009d90 tx=0x7f899402a6a0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 -- 172.21.15.123:0/4170258905 shutdown_connections 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 --2- 172.21.15.123:0/4170258905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 0x7f899c15ab40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 --2- 172.21.15.123:0/4170258905 >> v2:172.21.15.123:3300/0 conn(0x7f899c0c90d0 0x7f899c0c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 -- 172.21.15.123:0/4170258905 >> 172.21.15.123:0/4170258905 conn(0x7f899c06c750 msgr2=0x7f899c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 -- 172.21.15.123:0/4170258905 shutdown_connections 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.984+0000 7f89a4600640 1 -- 172.21.15.123:0/4170258905 wait complete. 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.987+0000 7f89a4600640 1 Processor -- start 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.987+0000 7f89a4600640 1 -- start start 2024-04-25T12:36:22.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.988+0000 7f89a4600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f899c0c4240 0x7f899c154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.988+0000 7f89a4600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f899c0c90d0 0x7f899c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a4600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 0x7f899c1fa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a4600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f899c0c8c90 con 0x7f899c0c90d0 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a4600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f899c0c89b0 con 0x7f899c15a730 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a4600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f899c073db0 con 0x7f899c0c4240 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f899c0c4240 0x7f899c154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f899c0c4240 0x7f899c154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:60104/0 (socket says 172.21.15.123:60104) 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 -- 172.21.15.123:0/818142286 learned_addr learned my addr 172.21.15.123:0/818142286 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1200640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f899c0c90d0 0x7f899c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a2600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 0x7f899c1fa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 -- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 msgr2=0x7f899c1fa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 0x7f899c1fa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 -- 172.21.15.123:0/818142286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f899c0c90d0 msgr2=0x7f899c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f899c0c90d0 0x7f899c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1c00640 1 -- 172.21.15.123:0/818142286 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c0cd510 con 0x7f899c0c4240 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.989+0000 7f89a1200640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f899c0c90d0 0x7f899c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.990+0000 7f89a1c00640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f899c0c4240 0x7f899c154e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f899000d5f0 tx=0x7f899000dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.990+0000 7f8999600640 1 -- 172.21.15.123:0/818142286 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900125f0 con 0x7f899c0c4240 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.990+0000 7f8999600640 1 -- 172.21.15.123:0/818142286 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8990019070 con 0x7f899c0c4240 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.990+0000 7f8999600640 1 -- 172.21.15.123:0/818142286 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900129b0 con 0x7f899c0c4240 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.990+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c1f7b10 con 0x7f899c0c4240 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.990+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c158170 con 0x7f899c0c4240 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.992+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899c06d500 con 0x7f899c0c4240 2024-04-25T12:36:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.993+0000 7f8999600640 1 -- 172.21.15.123:0/818142286 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8990012b50 con 0x7f899c0c4240 2024-04-25T12:36:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.994+0000 7f8999600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f897c0744e0 0x7f897c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.995+0000 7f8999600640 1 -- 172.21.15.123:0/818142286 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 6648+0+0 (secure 0 0 0) 0x7f89900956b0 con 0x7f899c0c4240 2024-04-25T12:36:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.995+0000 7f89a1200640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f897c0744e0 0x7f897c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.995+0000 7f89a1200640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f897c0744e0 0x7f897c0769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f899c14ba40 tx=0x7f8988009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:21.997+0000 7f8999600640 1 -- 172.21.15.123:0/818142286 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f899005fb80 con 0x7f899c0c4240 2024-04-25T12:36:22.189 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:22.189+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:22.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.200+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f899c076700 con 0x7f899c0c4240 2024-04-25T12:36:22.242 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:22.242 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:22.242 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:22.243 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:22.251 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:22.290 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:22.290+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:22.364 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:22.364 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:22.364 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:22.364 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:22.364 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:22.372 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:22.416 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:22.416+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 24/18 rep_tid 1 entries 24'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:22.486 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.531409563, 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075768199999999997, 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.487 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954840+0000", 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6869999999999999e-06 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954864+0000", 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3824000000000001e-05 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954902+0000", 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500999999999999e-05 2024-04-25T12:36:22.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954919+0000", 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7618e-05 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955011+0000", 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2250000000000006e-05 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955596+0000", 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058480200000000004 2024-04-25T12:36:22.489 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:25 2.1b e25/18 2:dbae76b5:::mds1_inotable:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.369724481000006, 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036130300000000001, 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.491 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116525+0000", 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.455e-06 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116539+0000", 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4e-05 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116563+0000", 2024-04-25T12:36:22.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4111999999999999e-05 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116577+0000", 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4059999999999999e-05 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116689+0000", 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011209000000000001 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116719+0000", 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.989e-05 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120082+0000", 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033628130000000001 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120137+0000", 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4610000000000001e-05 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.494 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.495 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.495 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 4 entries 25'3 (0'0) error 2:db457a8a:::201.00000001:head by mds.1.9:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:22.498 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:22.498 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.368667939999995, 2024-04-25T12:36:22.498 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030622480000000001, 2024-04-25T12:36:22.498 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117581+0000", 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0279999999999999e-06 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117596+0000", 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5109999999999999e-05 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117631+0000", 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4230000000000003e-05 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117655+0000", 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4987e-05 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117751+0000", 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5311999999999998e-05 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120642+0000", 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028915809999999998 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 5 entries 25'4 (0'0) error 2:d80f8093:::201.00000002:head by mds.1.9:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.368166708999993, 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0025905469999999999, 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.502 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118082+0000", 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4000000000000001e-07 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118091+0000", 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9919999999999996e-06 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118108+0000", 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7207999999999999e-05 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.504 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118124+0000", 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5378000000000001e-05 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118203+0000", 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8929000000000003e-05 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120672+0000", 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024692999999999998 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.505 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:34 2.1b e25/18 2:db457a8a:::201.00000001:head v 25'5, mlcod=25'3)", 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.364692375000004, 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016516759999999999, 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121557+0000", 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.423e-06 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.507 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121568+0000", 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0499000000000001e-05 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121585+0000", 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7283e-05 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121597+0000", 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1821000000000001e-05 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121653+0000", 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6533000000000002e-05 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121671+0000", 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7285000000000001e-05 2024-04-25T12:36:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123158+0000", 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0014869519999999999 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123207+0000", 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9879999999999997e-05 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.510 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:35 2.a 2:54ba6923:::mds1_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.342580057999996, 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016436459999999999, 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.511 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143670+0000", 2024-04-25T12:36:22.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4040000000000002e-06 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143685+0000", 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4768e-05 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143703+0000", 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7424000000000001e-05 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143721+0000", 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8685000000000002e-05 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143824+0000", 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010286800000000001 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.514 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144032+0000", 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020812700000000001 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144205+0000", 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017276999999999999 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144982+0000", 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077719300000000002 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145196+0000", 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000213308 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145254+0000", 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8619000000000002e-05 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145312+0000", 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7479999999999999e-05 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.1.9:35 2.a e25/18 ondisk, result = 0)", 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.341153491, 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00022686700000000001, 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.517 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.518 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145096+0000", 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2309999999999999e-06 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145102+0000", 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1639999999999998e-06 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145115+0000", 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2809000000000001e-05 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145124+0000", 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9989999999999997e-06 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145178+0000", 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4287999999999999e-05 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145189+0000", 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.084e-05 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145321+0000", 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013253599999999999 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:11 2.c 2:36924187:::618.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.333904224999998, 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070050249999999998, 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152345+0000", 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6270000000000001e-06 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152355+0000", 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4069999999999999e-06 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152384+0000", 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9479e-05 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152399+0000", 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152478+0000", 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9702000000000004e-05 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152615+0000", 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013665500000000001 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152730+0000", 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011504499999999999 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156394+0000", 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036643259999999999 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.526 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028370299999999999 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159289+0000", 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7268999999999997e-05 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159349+0000", 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0161999999999998e-05 2024-04-25T12:36:22.527 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.333817651000004, 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070895239999999998, 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.528 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152431+0000", 2024-04-25T12:36:22.529 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6899999999999996e-07 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152440+0000", 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3259999999999996e-06 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152450+0000", 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0874e-05 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.530 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152470+0000", 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9658999999999999e-05 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153265+0000", 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079454099999999995 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153349+0000", 2024-04-25T12:36:22.531 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4381000000000004e-05 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153485+0000", 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136365 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156400+0000", 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029144990000000001 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.532 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159418+0000", 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030182299999999998 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159477+0000", 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159520+0000", 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3127000000000002e-05 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.333754037000006, 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0068584939999999997, 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.534 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.535 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152495+0000", 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.16e-07 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152501+0000", 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7819999999999999e-06 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.536 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152510+0000", 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9199999999999993e-06 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152518+0000", 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9090000000000006e-06 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152651+0000", 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000132349 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.537 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152795+0000", 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014392800000000001 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152930+0000", 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013561399999999999 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.538 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156365+0000", 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034348830000000001 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159266+0000", 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029006959999999999 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.539 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159296+0000", 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0026e-05 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159353+0000", 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6570999999999998e-05 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.540 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.333668074000002, 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069239999999999996, 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.541 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152581+0000", 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3399999999999998e-07 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.542 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152587+0000", 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4049999999999997e-06 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152598+0000", 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1043000000000001e-05 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152607+0000", 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9409999999999996e-06 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.543 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153448+0000", 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00084054699999999997 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153563+0000", 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000115584 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.544 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153719+0000", 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015566199999999999 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156373+0000", 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002653617 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159423+0000", 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030508060000000001 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159472+0000", 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9042000000000002e-05 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159504+0000", 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1519000000000001e-05 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.546 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.333585190999997, 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0086667419999999998, 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.547 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152664+0000", 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0100000000000003e-07 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.548 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152671+0000", 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8870000000000003e-06 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152679+0000", 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2730000000000002e-06 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152696+0000", 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7373000000000001e-05 2024-04-25T12:36:22.549 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153747+0000", 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001050647 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153967+0000", 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00021982699999999999 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.550 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154122+0000", 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015472399999999999 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156405+0000", 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022838110000000002 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:22.551 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161248+0000", 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048425669999999999 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161283+0000", 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5166000000000002e-05 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161330+0000", 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6566000000000001e-05 2024-04-25T12:36:22.552 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.2 epoch 25/18 rep_tid 20 entries 25'1 (0'0) error 2:40dbbc66:::202.00000003:head by mds.2.13:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.331979309999994, 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0021562719999999999, 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.553 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154270+0000", 2024-04-25T12:36:22.554 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.638e-06 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154290+0000", 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392e-05 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154315+0000", 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4858000000000002e-05 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.555 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154335+0000", 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0044e-05 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154711+0000", 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037622099999999999 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156425+0000", 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017141190000000001 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.556 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:11 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.327123416000006, 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00023418899999999999, 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.557 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159126+0000", 2024-04-25T12:36:22.558 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7699999999999992e-07 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159131+0000", 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8109999999999999e-06 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159143+0000", 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1745e-05 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.559 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159153+0000", 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6520000000000001e-06 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159211+0000", 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8582999999999998e-05 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159225+0000", 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3088000000000001e-05 2024-04-25T12:36:22.560 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159359+0000", 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013433299999999999 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:15 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.32707302, 2024-04-25T12:36:22.561 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000189507, 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.562 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159176+0000", 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6899999999999998e-07 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159180+0000", 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1320000000000004e-06 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.563 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159188+0000", 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9319999999999996e-06 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159199+0000", 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1165999999999999e-05 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:36:22.564 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5071000000000002e-05 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159262+0000", 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0160000000000008e-06 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159365+0000", 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010272099999999999 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.565 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:12 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.327034089999998, 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000315354, 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.566 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2099999999999998e-07 2024-04-25T12:36:22.567 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159217+0000", 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.875e-06 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159223+0000", 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2999999999999998e-06 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.568 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159229+0000", 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6740000000000002e-06 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159401+0000", 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017201599999999999 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159412+0000", 2024-04-25T12:36:22.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0349e-05 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159529+0000", 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000117819 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:16 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:36:22.570 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.327005821, 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00026996799999999999, 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159243+0000", 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0100000000000001e-07 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159246+0000", 2024-04-25T12:36:22.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.687e-06 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.072e-06 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159261+0000", 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2169999999999999e-06 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.573 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159408+0000", 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000147637 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159417+0000", 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999996e-06 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159512+0000", 2024-04-25T12:36:22.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5163999999999999e-05 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:23 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.325091666000006, 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000182072, 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:22.575 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161157+0000", 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7400000000000002e-07 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161164+0000", 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161180+0000", 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6189999999999999e-05 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161191+0000", 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0981e-05 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161238+0000", 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7509999999999997e-05 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:22.578 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161244+0000", 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.694e-06 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161338+0000", 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4432999999999995e-05 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:22.579 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:22.582 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:22.582 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:22.581+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:22.682 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:22.682+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:22.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.713+0000 7f8999600640 1 -- 172.21.15.123:0/818142286 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v57) ==== 101+0+0 (secure 0 0 0) 0x7f899005f480 con 0x7f899c0c4240 2024-04-25T12:36:22.713 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f897c0744e0 msgr2=0x7f897c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f897c0744e0 0x7f897c0769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f899c14ba40 tx=0x7f8988009040 comp rx=0 tx=0).stop 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f899c0c4240 msgr2=0x7f899c154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f899c0c4240 0x7f899c154e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f899000d5f0 tx=0x7f899000dab0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 shutdown_connections 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f897c0744e0 0x7f897c0769f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c15a730 0x7f899c1fa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f899c0c90d0 0x7f899c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 --2- 172.21.15.123:0/818142286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f899c0c4240 0x7f899c154e20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:22.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 >> 172.21.15.123:0/818142286 conn(0x7f899c06c750 msgr2=0x7f899c0730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:22.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 shutdown_connections 2024-04-25T12:36:22.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:22.717+0000 7f89a4600640 1 -- 172.21.15.123:0/818142286 wait complete. 2024-04-25T12:36:22.738 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:22.739 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:22.739 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:22.739 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:22.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:22.789 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:22.789+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:22.853 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:22.854 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:22.854 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-04-25T12:36:22.854 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-25T12:36:22.854 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-25T12:36:22.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-04-25T12:36:22.900 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:22.900 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:22.900 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:22.903 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:22.903+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:22.909 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:22.984 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:22.984+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:23.023 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:23.023 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:23.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:23.023 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:23.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 124.07205278000001, 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:36:23.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:23.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.028 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:23.030 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 124.07196512599999, 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:36:23.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:23.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 124.07162055800001, 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:23.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.039 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:36:23.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 124.071548088, 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:36:23.043 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:23.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:36:23.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.049 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 124.07147411, 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:23.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:36:23.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.055 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 124.06943673799999, 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:36:23.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:36:23.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.060 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 124.06857660199999, 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:36:23.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:36:23.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.907890946999999, 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.066 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:36:23.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.907814833000003, 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:36:23.072 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:36:23.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.078 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.907769164000001, 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:23.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:36:23.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.084 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.907724388999995, 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.085 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.086 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:36:23.087 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.088 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.901107964999994, 2024-04-25T12:36:23.089 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.090 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:36:23.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.092 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:36:23.093 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.094 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 92.901048474999996, 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:23.095 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.096 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:36:23.097 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.098 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.870078183000004, 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.099 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:36:23.100 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.101 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:36:23.102 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.103 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:36:23.104 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.870000672000003, 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.105 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:23.106 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.107 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:36:23.108 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.109 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.110 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.869952823000006, 2024-04-25T12:36:23.111 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.112 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.113 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:36:23.114 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.115 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.116 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.117 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.869910933, 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:36:23.120 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.121 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.122 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:36:23.123 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:23.124 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.869876761, 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.125 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:36:23.126 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.127 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:23.128 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:36:23.129 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.130 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.869594019000004, 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.131 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.132 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.133 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:36:23.134 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 87.865453587999994, 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:23.135 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.136 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:36:23.137 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:36:23.138 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:23.139 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:23.141 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:23.142 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-25T12:36:23.142 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:23.142 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:23.143 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:23.141+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:23.242 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:23.241+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:23.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:23.294 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:23.294 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:23.294 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:23.294 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:23.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:23.344 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:23.344+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:23.443 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:23.443+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:23.454 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:23.454 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:23.454 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:23.455 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:23.455 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:23.464 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.522+0000 7f8ba5a00640 1 Processor -- start 2024-04-25T12:36:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.522+0000 7f8ba5a00640 1 -- start start 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8ba5a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba00c7f70 0x7f8ba0072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8ba5a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba0074230 con 0x7f8ba0154510 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8ba5a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba0074510 con 0x7f8ba0073220 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8ba5a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba0154920 con 0x7f8ba00c7f70 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba00c7f70 0x7f8ba0072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba00c7f70 0x7f8ba0072ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:34498/0 (socket says 172.21.15.123:34498) 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8b9f400640 1 -- 172.21.15.123:0/3331212266 learned_addr learned my addr 172.21.15.123:0/3331212266 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8ba5000640 1 --1- 172.21.15.123:0/3331212266 >> v1:172.21.15.153:6789/0 conn(0x7f8ba0073220 0x7f8ba00c8760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60228/0 (socket says 172.21.15.123:60228) 2024-04-25T12:36:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.524+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3331212266 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3589240542 0 0) 0x7f8ba0074510 con 0x7f8ba0073220 2024-04-25T12:36:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.525+0000 7f8b9f400640 1 -- 172.21.15.123:0/3331212266 >> v1:172.21.15.153:6789/0 conn(0x7f8ba0073220 legacy=0x7f8ba00c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.525+0000 7f8b9f400640 1 -- 172.21.15.123:0/3331212266 >> v1:172.21.15.123:6789/0 conn(0x7f8ba0154510 legacy=0x7f8ba015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.525+0000 7f8b9f400640 1 -- 172.21.15.123:0/3331212266 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba00747f0 con 0x7f8ba00c7f70 2024-04-25T12:36:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.526+0000 7f8b9f400640 1 --2- 172.21.15.123:0/3331212266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba00c7f70 0x7f8ba0072ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098a0 tx=0x7f8b8c02a3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394510dde5993837 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.527+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3331212266 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c037070 con 0x7f8ba00c7f70 2024-04-25T12:36:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.527+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3331212266 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c032640 con 0x7f8ba00c7f70 2024-04-25T12:36:23.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.527+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3331212266 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0329f0 con 0x7f8ba00c7f70 2024-04-25T12:36:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.528+0000 7f8ba5a00640 1 -- 172.21.15.123:0/3331212266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba00c7f70 msgr2=0x7f8ba0072ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.528+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/3331212266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba00c7f70 0x7f8ba0072ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098a0 tx=0x7f8b8c02a3b0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.528+0000 7f8ba5a00640 1 -- 172.21.15.123:0/3331212266 shutdown_connections 2024-04-25T12:36:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.528+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/3331212266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba00c7f70 0x7f8ba0072ce0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.528+0000 7f8ba5a00640 1 -- 172.21.15.123:0/3331212266 >> 172.21.15.123:0/3331212266 conn(0x7f8ba0070b90 msgr2=0x7f8ba0070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.528+0000 7f8ba5a00640 1 -- 172.21.15.123:0/3331212266 shutdown_connections 2024-04-25T12:36:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.528+0000 7f8ba5a00640 1 -- 172.21.15.123:0/3331212266 wait complete. 2024-04-25T12:36:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.530+0000 7f8ba5a00640 1 Processor -- start 2024-04-25T12:36:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.530+0000 7f8ba5a00640 1 -- start start 2024-04-25T12:36:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ba0073220 0x7f8ba00cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00c7f70 0x7f8ba013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba0154510 0x7f8ba013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba00c8380 con 0x7f8ba00c7f70 2024-04-25T12:36:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba015db00 con 0x7f8ba0073220 2024-04-25T12:36:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba0074ad0 con 0x7f8ba0154510 2024-04-25T12:36:23.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00c7f70 0x7f8ba013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba0154510 0x7f8ba013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ba0073220 0x7f8ba00cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba0154510 0x7f8ba013f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:34500/0 (socket says 172.21.15.123:34500) 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00c7f70 0x7f8ba013ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60854/0 (socket says 172.21.15.123:60854) 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.532+0000 7f8ba5000640 1 -- 172.21.15.123:0/513931893 learned_addr learned my addr 172.21.15.123:0/513931893 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b9fe00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba0154510 msgr2=0x7f8ba013f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8ba0154510 0x7f8ba013f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b9fe00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00c7f70 msgr2=0x7f8ba013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00c7f70 0x7f8ba013ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b9fe00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba0145220 con 0x7f8ba0073220 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b9f400640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00c7f70 0x7f8ba013ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ba0073220 0x7f8ba00cc640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000c390 tx=0x7f8b9000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90017070 con 0x7f8ba0073220 2024-04-25T12:36:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b900052d0 con 0x7f8ba0073220 2024-04-25T12:36:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90003830 con 0x7f8ba0073220 2024-04-25T12:36:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.533+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba01f61d0 con 0x7f8ba0073220 2024-04-25T12:36:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.534+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba0154e50 con 0x7f8ba0073220 2024-04-25T12:36:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.535+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8ba0076da0 con 0x7f8ba0073220 2024-04-25T12:36:23.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.537+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8b9001a070 con 0x7f8ba0073220 2024-04-25T12:36:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.538+0000 7f8b97e00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8b800744b0 0x7f8b800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.539+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6648+0+0 (secure 0 0 0) 0x7f8b90094790 con 0x7f8ba0073220 2024-04-25T12:36:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.539+0000 7f8b9f400640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8b800744b0 0x7f8b800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.539+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v57) ==== 54+0+16 (secure 0 0 0) 0x7f8b9009a170 con 0x7f8ba0073220 2024-04-25T12:36:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.539+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8ba0156a70 0x7f8ba0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.539+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ba00736d0 con 0x7f8ba0156a70 2024-04-25T12:36:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.540+0000 7f8b9f400640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8b800744b0 0x7f8b800769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c034000 tx=0x7f8b8c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.540+0000 7f8ba5000640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8ba0156a70 0x7f8ba0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.541+0000 7f8ba5000640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8ba0156a70 0x7f8ba0156e80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.541+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba00736d0 con 0x7f8ba0156a70 2024-04-25T12:36:23.543 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:23.543+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:23.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.557+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba0147930 con 0x7f8ba0156a70 2024-04-25T12:36:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.559+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba0147930 con 0x7f8ba0156a70 2024-04-25T12:36:23.559 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.559+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8ba015ca10 0x7f8ba015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.559+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8ba015d360 con 0x7f8ba015ca10 2024-04-25T12:36:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.560+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8ba015ca10 0x7f8ba015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.560+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8ba015ca10 0x7f8ba015ce20 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.561+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba015d360 con 0x7f8ba015ca10 2024-04-25T12:36:23.576 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:23.576 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:23.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:23.576 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [] 2024-04-25T12:36:23.577 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:23.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:23.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.591+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba0147370 con 0x7f8ba015ca10 2024-04-25T12:36:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.594+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba0147370 con 0x7f8ba015ca10 2024-04-25T12:36:23.595 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.595+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8ba00634f0 0x7f8ba0063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.595+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8ba0063e40 con 0x7f8ba00634f0 2024-04-25T12:36:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.595+0000 7f8b9f400640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8ba00634f0 0x7f8ba0063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.596+0000 7f8b9f400640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8ba00634f0 0x7f8ba0063900 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.597+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba0063e40 con 0x7f8ba00634f0 2024-04-25T12:36:23.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.627+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba00ccb80 con 0x7f8ba00634f0 2024-04-25T12:36:23.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.630+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba00ccb80 con 0x7f8ba00634f0 2024-04-25T12:36:23.630 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:23.630 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:23.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.630+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8ba00cb7f0 0x7f8ba00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.630+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8ba0157530 con 0x7f8ba00cb7f0 2024-04-25T12:36:23.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.630+0000 7f8ba5000640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8ba00cb7f0 0x7f8ba00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.631+0000 7f8ba5000640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8ba00cb7f0 0x7f8ba00cbc00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.633+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba0157530 con 0x7f8ba00cb7f0 2024-04-25T12:36:23.645 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:23.645+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:23.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.662+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba0154ab0 con 0x7f8ba00cb7f0 2024-04-25T12:36:23.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.665+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba0154ab0 con 0x7f8ba00cb7f0 2024-04-25T12:36:23.665 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:23.666 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:23.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.665+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8ba002cee0 0x7f8ba002d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.665+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8ba0154ab0 con 0x7f8ba002cee0 2024-04-25T12:36:23.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.666+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8ba002cee0 0x7f8ba002d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.667+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8ba002cee0 0x7f8ba002d2f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.668+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba0154ab0 con 0x7f8ba002cee0 2024-04-25T12:36:23.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.697+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba0154ab0 con 0x7f8ba002cee0 2024-04-25T12:36:23.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.699+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba0154ab0 con 0x7f8ba002cee0 2024-04-25T12:36:23.699 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:23.699 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:23.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.699+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8ba0207300 0x7f8ba0209720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.699+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8ba0154ab0 con 0x7f8ba0207300 2024-04-25T12:36:23.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.700+0000 7f8b9f400640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8ba0207300 0x7f8ba0209720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.700+0000 7f8b9f400640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8ba0207300 0x7f8ba0209720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.701+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba0154ab0 con 0x7f8ba0207300 2024-04-25T12:36:23.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.731+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba0154ab0 con 0x7f8ba0207300 2024-04-25T12:36:23.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.734+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba0154ab0 con 0x7f8ba0207300 2024-04-25T12:36:23.734 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:23.734 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:23.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.734+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8ba020a170 0x7f8ba020c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.734+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8ba002de50 con 0x7f8ba020a170 2024-04-25T12:36:23.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.735+0000 7f8ba5000640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8ba020a170 0x7f8ba020c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.735+0000 7f8ba5000640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8ba020a170 0x7f8ba020c590 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.736+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba002de50 con 0x7f8ba020a170 2024-04-25T12:36:23.745 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:23.745+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:23.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.764+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba01f6cb0 con 0x7f8ba020a170 2024-04-25T12:36:23.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.767+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba01f6cb0 con 0x7f8ba020a170 2024-04-25T12:36:23.767 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.767+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8ba020cfe0 0x7f8ba020f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.767+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8ba02121d0 con 0x7f8ba020cfe0 2024-04-25T12:36:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.767+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8ba020cfe0 0x7f8ba020f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:23.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.768+0000 7f8b9fe00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8ba020cfe0 0x7f8ba020f400 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:23.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.769+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba02121d0 con 0x7f8ba020cfe0 2024-04-25T12:36:23.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.797+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ba020f940 con 0x7f8ba020cfe0 2024-04-25T12:36:23.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.799+0000 7f8b97e00640 1 -- 172.21.15.123:0/513931893 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ba020f940 con 0x7f8ba020cfe0 2024-04-25T12:36:23.800 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:23.801 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8ba0156a70 msgr2=0x7f8ba0156e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8ba0156a70 0x7f8ba0156e80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8ba015ca10 msgr2=0x7f8ba015ce20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8ba015ca10 0x7f8ba015ce20 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8ba00634f0 msgr2=0x7f8ba0063900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8ba00634f0 0x7f8ba0063900 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8ba00cb7f0 msgr2=0x7f8ba00cbc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8ba00cb7f0 0x7f8ba00cbc00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8ba002cee0 msgr2=0x7f8ba002d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8ba002cee0 0x7f8ba002d2f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8ba0207300 msgr2=0x7f8ba0209720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8ba0207300 0x7f8ba0209720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8ba020a170 msgr2=0x7f8ba020c590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8ba020a170 0x7f8ba020c590 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8ba020cfe0 msgr2=0x7f8ba020f400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8ba020cfe0 0x7f8ba020f400 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8b800744b0 msgr2=0x7f8b800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8b800744b0 0x7f8b800769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c034000 tx=0x7f8b8c02e000 comp rx=0 tx=0).stop 2024-04-25T12:36:23.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ba0073220 msgr2=0x7f8ba00cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5a00640 1 --2- 172.21.15.123:0/513931893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ba0073220 0x7f8ba00cc640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000c390 tx=0x7f8b9000c850 comp rx=0 tx=0).stop 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5000640 1 -- 172.21.15.123:0/513931893 reap_dead start 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.800+0000 7f8ba5000640 1 -- 172.21.15.123:0/513931893 reap_dead start 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5000640 1 -- 172.21.15.123:0/513931893 reap_dead start 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5000640 1 -- 172.21.15.123:0/513931893 reap_dead start 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5000640 1 -- 172.21.15.123:0/513931893 reap_dead start 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5000640 1 -- 172.21.15.123:0/513931893 reap_dead start 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 shutdown_connections 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 >> 172.21.15.123:0/513931893 conn(0x7f8ba0070b90 msgr2=0x7f8ba0075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 shutdown_connections 2024-04-25T12:36:23.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:23.801+0000 7f8ba5a00640 1 -- 172.21.15.123:0/513931893 wait complete. 2024-04-25T12:36:23.846 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:23.846+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:23.880 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:23.880 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:23.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:23.946 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:23.946+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:24.048 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:24.048+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:24.149 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:24.149+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:24.156 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:24.157 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:24.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:24.270 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:24.270+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:24.350 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:24.351+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:24.451 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:24.451+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:24.476 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:24.477 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:24.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:24.552 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:24.552+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:24.652 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:24.653+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:24.753 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:24.754+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:24.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:36:24.854 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:24.854+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:24.896 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:24.896 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:24.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:24.977 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:24.977+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:25.055 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:25.055+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:25.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.094+0000 7f32b3200640 1 Processor -- start 2024-04-25T12:36:25.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.094+0000 7f32b3200640 1 -- start start 2024-04-25T12:36:25.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.097+0000 7f32b3200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b40c6450 0x7f32b40cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.097+0000 7f32b3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f32b41602b0 0x7f32b40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.098+0000 7f32b3200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b406ecc0 con 0x7f32b40cb990 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.098+0000 7f32b3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f32b415d220 con 0x7f32b41602b0 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.098+0000 7f32b3200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f32b415c610 con 0x7f32b40c6450 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.098+0000 7f32b1400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f32b40cb990 0x7f32b415fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60244/0 (socket says 172.21.15.123:60244) 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.098+0000 7f32b1400640 1 -- 172.21.15.123:0/246358776 learned_addr learned my addr 172.21.15.123:0/246358776 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.098+0000 7f32b1e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b40c6450 0x7f32b40cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:25.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.098+0000 7f32b2800640 1 --2- 172.21.15.123:0/246358776 >> v2:172.21.15.123:3300/0 conn(0x7f32b41602b0 0x7f32b40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:25.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.099+0000 7f32b1e00640 1 -- 172.21.15.123:0/246358776 >> v2:172.21.15.123:3300/0 conn(0x7f32b41602b0 msgr2=0x7f32b40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:25.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.099+0000 7f32b0a00640 1 -- 172.21.15.123:0/246358776 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2567443329 0 0) 0x7f32b406ecc0 con 0x7f32b40cb990 2024-04-25T12:36:25.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.099+0000 7f32b1e00640 1 --2- 172.21.15.123:0/246358776 >> v2:172.21.15.123:3300/0 conn(0x7f32b41602b0 0x7f32b40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.099+0000 7f32b1e00640 1 -- 172.21.15.123:0/246358776 >> v1:172.21.15.153:6789/0 conn(0x7f32b40cb990 legacy=0x7f32b415fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:25.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.099+0000 7f32b1e00640 1 -- 172.21.15.123:0/246358776 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b415c8f0 con 0x7f32b40c6450 2024-04-25T12:36:25.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.099+0000 7f32b2800640 1 --2- 172.21.15.123:0/246358776 >> v2:172.21.15.123:3300/0 conn(0x7f32b41602b0 0x7f32b40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:25.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.100+0000 7f32b1e00640 1 --2- 172.21.15.123:0/246358776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b40c6450 0x7f32b40cc390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32a40097a0 tx=0x7f32a402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383fdce4f07fc631 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:25.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.101+0000 7f32b0a00640 1 -- 172.21.15.123:0/246358776 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4037070 con 0x7f32b40c6450 2024-04-25T12:36:25.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.101+0000 7f32b0a00640 1 -- 172.21.15.123:0/246358776 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a4032440 con 0x7f32b40c6450 2024-04-25T12:36:25.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.101+0000 7f32b0a00640 1 -- 172.21.15.123:0/246358776 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4032810 con 0x7f32b40c6450 2024-04-25T12:36:25.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 -- 172.21.15.123:0/246358776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b40c6450 msgr2=0x7f32b40cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:25.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 --2- 172.21.15.123:0/246358776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b40c6450 0x7f32b40cc390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32a40097a0 tx=0x7f32a402a1a0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 -- 172.21.15.123:0/246358776 shutdown_connections 2024-04-25T12:36:25.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 --2- 172.21.15.123:0/246358776 >> v2:172.21.15.123:3300/0 conn(0x7f32b41602b0 0x7f32b40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 --2- 172.21.15.123:0/246358776 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b40c6450 0x7f32b40cc390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 -- 172.21.15.123:0/246358776 >> 172.21.15.123:0/246358776 conn(0x7f32b4072cb0 msgr2=0x7f32b40730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:25.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 -- 172.21.15.123:0/246358776 shutdown_connections 2024-04-25T12:36:25.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.102+0000 7f32b3200640 1 -- 172.21.15.123:0/246358776 wait complete. 2024-04-25T12:36:25.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.106+0000 7f32b3200640 1 Processor -- start 2024-04-25T12:36:25.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.106+0000 7f32b3200640 1 -- start start 2024-04-25T12:36:25.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.108+0000 7f32b3200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b40c6450 0x7f32b4147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:25.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.108+0000 7f32b3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32b40cb990 0x7f32b41e5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:25.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b3200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b41602b0 0x7f32b4148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:25.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32b40ca2b0 con 0x7f32b40cb990 2024-04-25T12:36:25.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b3200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32b415ceb0 con 0x7f32b40c6450 2024-04-25T12:36:25.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b3200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f32b406e880 con 0x7f32b41602b0 2024-04-25T12:36:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32b40cb990 0x7f32b41e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32b40cb990 0x7f32b41e5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60864/0 (socket says 172.21.15.123:60864) 2024-04-25T12:36:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b1400640 1 -- 172.21.15.123:0/1678091157 learned_addr learned my addr 172.21.15.123:0/1678091157 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.109+0000 7f32b2800640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b41602b0 0x7f32b4148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.110+0000 7f32b1400640 1 -- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b41602b0 msgr2=0x7f32b4148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.110+0000 7f32b1400640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b41602b0 0x7f32b4148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.110+0000 7f32b1400640 1 -- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b40c6450 msgr2=0x7f32b4147f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T12:36:25.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.110+0000 7f32b1400640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b40c6450 0x7f32b4147f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.110+0000 7f32b1400640 1 -- 172.21.15.123:0/1678091157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b4148470 con 0x7f32b40cb990 2024-04-25T12:36:25.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.111+0000 7f32b1400640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32b40cb990 0x7f32b41e5840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c440 tx=0x7f32a0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:25.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.111+0000 7f32aa000640 1 -- 172.21.15.123:0/1678091157 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00074e0 con 0x7f32b40cb990 2024-04-25T12:36:25.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.111+0000 7f32aa000640 1 -- 172.21.15.123:0/1678091157 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a00047e0 con 0x7f32b40cb990 2024-04-25T12:36:25.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.111+0000 7f32aa000640 1 -- 172.21.15.123:0/1678091157 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0004bd0 con 0x7f32b40cb990 2024-04-25T12:36:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.113+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b414a380 con 0x7f32b40cb990 2024-04-25T12:36:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.113+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b414a940 con 0x7f32b40cb990 2024-04-25T12:36:25.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.114+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b414a0d0 con 0x7f32b40cb990 2024-04-25T12:36:25.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.115+0000 7f32aa000640 1 -- 172.21.15.123:0/1678091157 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f32a0011070 con 0x7f32b40cb990 2024-04-25T12:36:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.117+0000 7f32aa000640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f32940744e0 0x7f32940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.117+0000 7f32aa000640 1 -- 172.21.15.123:0/1678091157 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6648+0+0 (secure 0 0 0) 0x7f32a00950c0 con 0x7f32b40cb990 2024-04-25T12:36:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.117+0000 7f32b1e00640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f32940744e0 0x7f32940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.118+0000 7f32b1e00640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f32940744e0 0x7f32940769f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034000 tx=0x7f32a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:25.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.122+0000 7f32aa000640 1 -- 172.21.15.123:0/1678091157 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32a005f490 con 0x7f32b40cb990 2024-04-25T12:36:25.156 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:25.156+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:25.256 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:25.256+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:25.278 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:25.278 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:25.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:25.357 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:25.357+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:25.457 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:25.458+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:25.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.527+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f32b4076da0 con 0x7f32b40cb990 2024-04-25T12:36:25.559 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:25.559+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:25.588 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:25.588 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:25.608 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:25.659 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:25.660+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:25.760 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:25.760+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:25.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:25.861 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:25.862+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:25.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.891+0000 7f32aa000640 1 -- 172.21.15.123:0/1678091157 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v58) ==== 87+0+0 (secure 0 0 0) 0x7f32a005ed90 con 0x7f32b40cb990 2024-04-25T12:36:25.898 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:36:25.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.899+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f32940744e0 msgr2=0x7f32940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:25.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.899+0000 7f32b3200640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f32940744e0 0x7f32940769f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034000 tx=0x7f32a402e000 comp rx=0 tx=0).stop 2024-04-25T12:36:25.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.899+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32b40cb990 msgr2=0x7f32b41e5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:25.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.899+0000 7f32b3200640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32b40cb990 0x7f32b41e5840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c440 tx=0x7f32a0001e70 comp rx=0 tx=0).stop 2024-04-25T12:36:25.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 shutdown_connections 2024-04-25T12:36:25.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f32940744e0 0x7f32940769f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f32b41602b0 0x7f32b4148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32b40cb990 0x7f32b41e5840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 --2- 172.21.15.123:0/1678091157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b40c6450 0x7f32b4147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:25.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 >> 172.21.15.123:0/1678091157 conn(0x7f32b4072cb0 msgr2=0x7f32b40c5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:25.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 shutdown_connections 2024-04-25T12:36:25.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:25.900+0000 7f32b3200640 1 -- 172.21.15.123:0/1678091157 wait complete. 2024-04-25T12:36:25.910 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:25.910 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:25.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:25.983 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:25.983+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:26.062 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:26.062+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:26.163 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:26.163+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:26.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.179+0000 7faaefc00640 1 Processor -- start 2024-04-25T12:36:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.182+0000 7faaefc00640 1 -- start start 2024-04-25T12:36:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faaefc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faae8152c70 0x7faae8074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faaefc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faae80c8170 0x7faae815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faaefc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae80c8580 con 0x7faae80c2790 2024-04-25T12:36:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faaefc00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7faae8158bd0 con 0x7faae8152c70 2024-04-25T12:36:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faaefc00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7faae8158eb0 con 0x7faae80c8170 2024-04-25T12:36:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faae7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7faae80c2790 0x7faae80c3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35820/0 (socket says 172.21.15.123:35820) 2024-04-25T12:36:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faae7e00640 1 -- 172.21.15.123:0/4059438221 learned_addr learned my addr 172.21.15.123:0/4059438221 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.185+0000 7faaed200640 1 --2- 172.21.15.123:0/4059438221 >> v2:172.21.15.123:3300/0 conn(0x7faae8152c70 0x7faae8074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.186+0000 7faaedc00640 1 --2- 172.21.15.123:0/4059438221 >> v2:172.21.15.153:3300/0 conn(0x7faae80c8170 0x7faae815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.186+0000 7faae7400640 1 -- 172.21.15.123:0/4059438221 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978366154 0 0) 0x7faae80c8580 con 0x7faae80c2790 2024-04-25T12:36:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.186+0000 7faae7400640 1 -- 172.21.15.123:0/4059438221 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faad0003520 con 0x7faae80c2790 2024-04-25T12:36:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.187+0000 7faaed200640 1 -- 172.21.15.123:0/4059438221 >> v2:172.21.15.153:3300/0 conn(0x7faae80c8170 msgr2=0x7faae815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.187+0000 7faaed200640 1 --2- 172.21.15.123:0/4059438221 >> v2:172.21.15.153:3300/0 conn(0x7faae80c8170 0x7faae815c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.187+0000 7faaed200640 1 -- 172.21.15.123:0/4059438221 >> v1:172.21.15.123:6789/0 conn(0x7faae80c2790 legacy=0x7faae80c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.187+0000 7faaed200640 1 -- 172.21.15.123:0/4059438221 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae8159190 con 0x7faae8152c70 2024-04-25T12:36:26.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.189+0000 7faaed200640 1 --2- 172.21.15.123:0/4059438221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae8152c70 0x7faae8074820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faae80c4b20 tx=0x7faadc02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2be0b16ea9ebe2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.190+0000 7faae7400640 1 -- 172.21.15.123:0/4059438221 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc037070 con 0x7faae8152c70 2024-04-25T12:36:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.190+0000 7faae7400640 1 -- 172.21.15.123:0/4059438221 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc032440 con 0x7faae8152c70 2024-04-25T12:36:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.190+0000 7faae7400640 1 -- 172.21.15.123:0/4059438221 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc032830 con 0x7faae8152c70 2024-04-25T12:36:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.191+0000 7faaefc00640 1 -- 172.21.15.123:0/4059438221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae8152c70 msgr2=0x7faae8074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.191+0000 7faaefc00640 1 --2- 172.21.15.123:0/4059438221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae8152c70 0x7faae8074820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faae80c4b20 tx=0x7faadc02a150 comp rx=0 tx=0).stop 2024-04-25T12:36:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.192+0000 7faaefc00640 1 -- 172.21.15.123:0/4059438221 shutdown_connections 2024-04-25T12:36:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.192+0000 7faaefc00640 1 --2- 172.21.15.123:0/4059438221 >> v2:172.21.15.153:3300/0 conn(0x7faae80c8170 0x7faae815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.192+0000 7faaefc00640 1 --2- 172.21.15.123:0/4059438221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae8152c70 0x7faae8074820 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.192+0000 7faaefc00640 1 -- 172.21.15.123:0/4059438221 >> 172.21.15.123:0/4059438221 conn(0x7faae806f440 msgr2=0x7faae806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.192+0000 7faaefc00640 1 -- 172.21.15.123:0/4059438221 shutdown_connections 2024-04-25T12:36:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.193+0000 7faaefc00640 1 -- 172.21.15.123:0/4059438221 wait complete. 2024-04-25T12:36:26.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.196+0000 7faaefc00640 1 Processor -- start 2024-04-25T12:36:26.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.196+0000 7faaefc00640 1 -- start start 2024-04-25T12:36:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaefc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80c2790 0x7faae8165ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaefc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae80c8170 0x7faae81f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaefc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faae8152c70 0x7faae8146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaefc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faae80c46f0 con 0x7faae80c2790 2024-04-25T12:36:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaefc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faae8159750 con 0x7faae80c8170 2024-04-25T12:36:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaefc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faae80c4870 con 0x7faae8152c70 2024-04-25T12:36:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaed200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80c2790 0x7faae8165ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaed200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80c2790 0x7faae8165ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60884/0 (socket says 172.21.15.123:60884) 2024-04-25T12:36:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faaed200640 1 -- 172.21.15.123:0/1283935285 learned_addr learned my addr 172.21.15.123:0/1283935285 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.199+0000 7faae7e00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae80c8170 0x7faae81f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.200+0000 7faaedc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faae8152c70 0x7faae8146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.200+0000 7faaed200640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faae8152c70 msgr2=0x7faae8146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.200+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faae8152c70 0x7faae8146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.200+0000 7faaed200640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae80c8170 msgr2=0x7faae81f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.200+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae80c8170 0x7faae81f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.200+0000 7faaed200640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae81f6ea0 con 0x7faae80c2790 2024-04-25T12:36:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.200+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80c2790 0x7faae8165ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faadc009890 tx=0x7faadc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.201+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc037040 con 0x7faae80c2790 2024-04-25T12:36:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.201+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc041d50 con 0x7faae80c2790 2024-04-25T12:36:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.201+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae8165500 con 0x7faae80c2790 2024-04-25T12:36:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.201+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae80c5170 con 0x7faae80c2790 2024-04-25T12:36:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.201+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc0039d0 con 0x7faae80c2790 2024-04-25T12:36:26.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.204+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7faae80c7e60 con 0x7faae80c2790 2024-04-25T12:36:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.205+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faadc003b70 con 0x7faae80c2790 2024-04-25T12:36:26.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.206+0000 7faae4c00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faac80744e0 0x7faac80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.206+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6648+0+0 (secure 0 0 0) 0x7faadc0bdef0 con 0x7faae80c2790 2024-04-25T12:36:26.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.207+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v58) ==== 54+0+16 (secure 0 0 0) 0x7faae80c7e60 con 0x7faae80c2790 2024-04-25T12:36:26.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.207+0000 7faae7e00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faac80744e0 0x7faac80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.207+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7faae80c9250 0x7faae8143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.207+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faae81652b0 con 0x7faae80c9250 2024-04-25T12:36:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.208+0000 7faae7e00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faac80744e0 0x7faac80769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faad40011e0 tx=0x7faad4007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.208+0000 7faaedc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7faae80c9250 0x7faae8143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.208+0000 7faaedc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7faae80c9250 0x7faae8143cf0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.210+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae81652b0 con 0x7faae80c9250 2024-04-25T12:36:26.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.228+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae8076ed0 con 0x7faae80c9250 2024-04-25T12:36:26.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.230+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae8076ed0 con 0x7faae80c9250 2024-04-25T12:36:26.233 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.234+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7faae8062fe0 0x7faae80633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.234+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7faae8144740 con 0x7faae8062fe0 2024-04-25T12:36:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.234+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7faae8062fe0 0x7faae80633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.235+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7faae8062fe0 0x7faae80633f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.236+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae8144740 con 0x7faae8062fe0 2024-04-25T12:36:26.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.252+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae8074600 con 0x7faae8062fe0 2024-04-25T12:36:26.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.254+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae8074600 con 0x7faae8062fe0 2024-04-25T12:36:26.254 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:26.255 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:26.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.254+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7faae8150fc0 0x7faae81513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.254+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7faae81479b0 con 0x7faae8150fc0 2024-04-25T12:36:26.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.255+0000 7faae7e00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7faae8150fc0 0x7faae81513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.256+0000 7faae7e00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7faae8150fc0 0x7faae81513d0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.257+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae81479b0 con 0x7faae8150fc0 2024-04-25T12:36:26.264 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:26.264+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:26.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.274+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae81663a0 con 0x7faae8150fc0 2024-04-25T12:36:26.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.277+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae81663a0 con 0x7faae8150fc0 2024-04-25T12:36:26.277 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:26.278 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:26.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.278+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faae8157230 0x7faae8157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.278+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7faae80cb6e0 con 0x7faae8157230 2024-04-25T12:36:26.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.278+0000 7faaedc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faae8157230 0x7faae8157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.279+0000 7faaedc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faae8157230 0x7faae8157640 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.280+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae80cb6e0 con 0x7faae8157230 2024-04-25T12:36:26.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.296+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae80cb6e0 con 0x7faae8157230 2024-04-25T12:36:26.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.299+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae80cb6e0 con 0x7faae8157230 2024-04-25T12:36:26.299 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:26.300 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:26.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.299+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7faae8158090 0x7faae802c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.300+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7faae81584a0 con 0x7faae8158090 2024-04-25T12:36:26.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.300+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7faae8158090 0x7faae802c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.301+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7faae8158090 0x7faae802c9d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.301+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae81584a0 con 0x7faae8158090 2024-04-25T12:36:26.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.317+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae81584a0 con 0x7faae8158090 2024-04-25T12:36:26.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.321+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae81584a0 con 0x7faae8158090 2024-04-25T12:36:26.321 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.321+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7faae802d420 0x7faae802d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.321+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7faae802dd70 con 0x7faae802d420 2024-04-25T12:36:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.322+0000 7faae7e00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7faae802d420 0x7faae802d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.322+0000 7faae7e00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7faae802d420 0x7faae802d830 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.323+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae802dd70 con 0x7faae802d420 2024-04-25T12:36:26.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.338+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae802dd70 con 0x7faae802d420 2024-04-25T12:36:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.341+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae802dd70 con 0x7faae802d420 2024-04-25T12:36:26.341 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.341+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7faae82059a0 0x7faae8207dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.341+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7faae8165780 con 0x7faae82059a0 2024-04-25T12:36:26.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.341+0000 7faaedc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7faae82059a0 0x7faae8207dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.342+0000 7faaedc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7faae82059a0 0x7faae8207dc0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.343+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae8165780 con 0x7faae82059a0 2024-04-25T12:36:26.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.360+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae8165750 con 0x7faae82059a0 2024-04-25T12:36:26.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.362+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae8165750 con 0x7faae82059a0 2024-04-25T12:36:26.362 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:26.363 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:26.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.362+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7faae8208810 0x7faae820ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:26.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.362+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7faae820db70 con 0x7faae8208810 2024-04-25T12:36:26.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.363+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7faae8208810 0x7faae820ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:26.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.363+0000 7faaed200640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7faae8208810 0x7faae820ac30 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:26.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.364+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faae820db70 con 0x7faae8208810 2024-04-25T12:36:26.368 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:26.368+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:26.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.379+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7faae820c8b0 con 0x7faae8208810 2024-04-25T12:36:26.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.381+0000 7faae4c00640 1 -- 172.21.15.123:0/1283935285 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7faae820c8b0 con 0x7faae8208810 2024-04-25T12:36:26.381 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7faae80c9250 msgr2=0x7faae8143cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7faae80c9250 0x7faae8143cf0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7faae8062fe0 msgr2=0x7faae80633f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7faae8062fe0 0x7faae80633f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7faae8150fc0 msgr2=0x7faae81513d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7faae8150fc0 0x7faae81513d0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faae8157230 msgr2=0x7faae8157640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faae8157230 0x7faae8157640 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7faae8158090 msgr2=0x7faae802c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7faae8158090 0x7faae802c9d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7faae802d420 msgr2=0x7faae802d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7faae802d420 0x7faae802d830 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7faae82059a0 msgr2=0x7faae8207dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7faae82059a0 0x7faae8207dc0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7faae8208810 msgr2=0x7faae820ac30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7faae8208810 0x7faae820ac30 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaedc00640 1 -- 172.21.15.123:0/1283935285 reap_dead start 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaedc00640 1 -- 172.21.15.123:0/1283935285 reap_dead start 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaedc00640 1 -- 172.21.15.123:0/1283935285 reap_dead start 2024-04-25T12:36:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaedc00640 1 -- 172.21.15.123:0/1283935285 reap_dead start 2024-04-25T12:36:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaedc00640 1 -- 172.21.15.123:0/1283935285 reap_dead start 2024-04-25T12:36:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faac80744e0 msgr2=0x7faac80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faac80744e0 0x7faac80769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faad40011e0 tx=0x7faad4007d30 comp rx=0 tx=0).stop 2024-04-25T12:36:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80c2790 msgr2=0x7faae8165ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80c2790 0x7faae8165ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faadc009890 tx=0x7faadc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 shutdown_connections 2024-04-25T12:36:26.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faac80744e0 0x7faac80769f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 --2- 172.21.15.123:0/1283935285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80c2790 0x7faae8165ab0 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faadc009890 tx=0x7faadc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:26.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.382+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 >> 172.21.15.123:0/1283935285 conn(0x7faae806f440 msgr2=0x7faae80ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:26.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.383+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 shutdown_connections 2024-04-25T12:36:26.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:26.383+0000 7faaefc00640 1 -- 172.21.15.123:0/1283935285 wait complete. 2024-04-25T12:36:26.385 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:26.385 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:26.386 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:26.386 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:26.386 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:26.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:26.466 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:26.467+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:26.567 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:26.567+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:26.667 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:26.668+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:26.768 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:26.769+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:26.859 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:26.859 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:26.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:26.859 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 127.906268469, 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:36:26.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:36:26.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:36:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.864 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 127.906156526, 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.869 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 127.906107726, 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 127.906036566, 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:26.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:36:26.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:36:26.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 127.905988154, 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 127.89779222600001, 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.884 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:26.888 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 110.791548228, 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:36:26.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:36:26.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:36:26.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.893 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 110.791366227, 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:36:26.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:36:26.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.739332239999996, 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.739256284000007, 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.734827543999998, 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:36:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:36:26.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:36:26.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:36:26.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.734645481000001, 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.918 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.734541117000006, 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:26.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:36:26.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:36:26.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:36:26.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.733265560000007, 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 96.732211336000006, 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 91.703345696, 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 77.981834200999998, 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 73.939273649, 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:36:26.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:36:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:48 2.14 2:2e2fa760:::200.00000001:head [write 5505~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.066881717999998, 2024-04-25T12:36:26.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010227916, 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790256+0000", 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7570000000000002e-06 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790280+0000", 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4456000000000001e-05 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790319+0000", 2024-04-25T12:36:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9103000000000002e-05 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790344+0000", 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4998999999999999e-05 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790557+0000", 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021251199999999999 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-25T12:36:26.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790572+0000", 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5003e-05 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.796879+0000", 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063072359999999999 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797129+0000", 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025002800000000001 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797609+0000", 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047965200000000001 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800188+0000", 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002578922 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800281+0000", 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3561999999999996e-05 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800387+0000", 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010545800000000001 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800480+0000", 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3227999999999996e-05 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.951 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:49 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 40.066612137, 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011186097000000001, 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790524+0000", 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999998e-06 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790541+0000", 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7567e-05 2024-04-25T12:36:26.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790566+0000", 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4338999999999999e-05 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790581+0000", 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5123000000000001e-05 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790708+0000", 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012676699999999999 2024-04-25T12:36:26.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790718+0000", 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.798785+0000", 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080674500000000003 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799024+0000", 2024-04-25T12:36:26.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000239044 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799376+0000", 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035158299999999998 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800980+0000", 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016040939999999999 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801537+0000", 2024-04-25T12:36:26.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055677400000000001 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801639+0000", 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102388 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801708+0000", 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8504999999999995e-05 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:26.957 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:26.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:26.958 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:26.912+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:27.012 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:27.013+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:27.113 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:27.113+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:27.214 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:27.214+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:27.315 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:27.315+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:27.334 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:27.334 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:27.334 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:27.334 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:27.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:27.415 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:27.416+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:27.516 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:27.516+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:27.616 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:27.617+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:27.649 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:27.649 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:27.649 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:27.649 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:27.649 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:27.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:27.717 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:27.718+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:27.818 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:27.818+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:27.919 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:27.919+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:27.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:36:27.982 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 129.032107381, 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:36:27.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 129.03142319899999, 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:36:27.987 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:27.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:36:27.991 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 129.02997265799999, 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:27.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:36:27.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:36:27.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 129.029711843, 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 129.025861456, 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:27.999 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:36:28.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:36:28.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:36:28.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 129.02563908799999, 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:36:28.004 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.009 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 129.02528529899999, 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:36:28.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:36:28.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.864666350999997, 2024-04-25T12:36:28.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:36:28.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:28.018 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.864458858999996, 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:28.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:36:28.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:36:28.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.023 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.864314368999999, 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:28.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:36:28.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.864182575000001, 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.033 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.864108279999996, 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:36:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.864008538999997, 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:28.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:36:28.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:36:28.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:36:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:36:28.043 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.863939076999998, 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:36:28.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:36:28.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.048 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.863838963000006, 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:36:28.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:36:28.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:36:28.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:28.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:36:28.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:36:28.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.863741872000006, 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.062 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.062 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:28.062 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.863651466999997, 2024-04-25T12:36:28.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:36:28.063 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.063 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.063 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:28.063 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:36:28.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:36:28.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:36:28.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.863503494, 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:36:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:36:28.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 97.863396089000005, 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.074 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 92.829144791000004, 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:28.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:36:28.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:36:28.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.080 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.080 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:28.080 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:28.080 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:28.064+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:28.143 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:28.143 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:28.143 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-25T12:36:28.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-25T12:36:28.191 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:28.191+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:28.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.191+0000 7ff0b6600640 1 Processor -- start 2024-04-25T12:36:28.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.191+0000 7ff0b6600640 1 -- start start 2024-04-25T12:36:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0b6600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b00c9b80 0x7ff0b00742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0b6600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0b00c8250 0x7ff0b00c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0b6600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b0156e60 con 0x7ff0b0159820 2024-04-25T12:36:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0b6600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b00c8ba0 con 0x7ff0b00c8250 2024-04-25T12:36:28.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0b6600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b015ca70 con 0x7ff0b00c9b80 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0af400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b00c9b80 0x7ff0b00742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0afe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff0b0159820 0x7ff0b0159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35828/0 (socket says 172.21.15.123:35828) 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0af400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b00c9b80 0x7ff0b00742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:34550/0 (socket says 172.21.15.123:34550) 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.194+0000 7ff0afe00640 1 -- 172.21.15.123:0/341830599 learned_addr learned my addr 172.21.15.123:0/341830599 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.195+0000 7ff0aea00640 1 --2- 172.21.15.123:0/341830599 >> v2:172.21.15.123:3300/0 conn(0x7ff0b00c8250 0x7ff0b00c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.195+0000 7ff0af400640 1 -- 172.21.15.123:0/341830599 >> v2:172.21.15.123:3300/0 conn(0x7ff0b00c8250 msgr2=0x7ff0b00c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.195+0000 7ff0af400640 1 --2- 172.21.15.123:0/341830599 >> v2:172.21.15.123:3300/0 conn(0x7ff0b00c8250 0x7ff0b00c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.195+0000 7ff0af400640 1 -- 172.21.15.123:0/341830599 >> v1:172.21.15.123:6789/0 conn(0x7ff0b0159820 legacy=0x7ff0b0159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.195+0000 7ff0af400640 1 -- 172.21.15.123:0/341830599 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b00c7130 con 0x7ff0b00c9b80 2024-04-25T12:36:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.195+0000 7ff0ae000640 1 -- 172.21.15.123:0/341830599 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708702804 0 0) 0x7ff0b0156e60 con 0x7ff0b0159820 2024-04-25T12:36:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.196+0000 7ff0af400640 1 --2- 172.21.15.123:0/341830599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b00c9b80 0x7ff0b00742b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00097a0 tx=0x7ff0a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9b5211f5e75a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.196+0000 7ff0ae000640 1 -- 172.21.15.123:0/341830599 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0037070 con 0x7ff0b00c9b80 2024-04-25T12:36:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.196+0000 7ff0ae000640 1 -- 172.21.15.123:0/341830599 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a0032440 con 0x7ff0b00c9b80 2024-04-25T12:36:28.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.196+0000 7ff0ae000640 1 -- 172.21.15.123:0/341830599 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0032810 con 0x7ff0b00c9b80 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.197+0000 7ff0b6600640 1 -- 172.21.15.123:0/341830599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b00c9b80 msgr2=0x7ff0b00742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.197+0000 7ff0b6600640 1 --2- 172.21.15.123:0/341830599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b00c9b80 0x7ff0b00742b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00097a0 tx=0x7ff0a002a180 comp rx=0 tx=0).stop 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.198+0000 7ff0b6600640 1 -- 172.21.15.123:0/341830599 shutdown_connections 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.198+0000 7ff0b6600640 1 --2- 172.21.15.123:0/341830599 >> v2:172.21.15.123:3300/0 conn(0x7ff0b00c8250 0x7ff0b00c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.198+0000 7ff0b6600640 1 --2- 172.21.15.123:0/341830599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b00c9b80 0x7ff0b00742b0 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00097a0 tx=0x7ff0a002a180 comp rx=0 tx=0).stop 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.198+0000 7ff0b6600640 1 -- 172.21.15.123:0/341830599 >> 172.21.15.123:0/341830599 conn(0x7ff0b006c750 msgr2=0x7ff0b006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.198+0000 7ff0b6600640 1 -- 172.21.15.123:0/341830599 shutdown_connections 2024-04-25T12:36:28.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.198+0000 7ff0b6600640 1 -- 172.21.15.123:0/341830599 wait complete. 2024-04-25T12:36:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.200+0000 7ff0b6600640 1 Processor -- start 2024-04-25T12:36:28.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.200+0000 7ff0b6600640 1 -- start start 2024-04-25T12:36:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0b6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00c8250 0x7ff0b01420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0b6600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0b0159820 0x7ff0b0156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0b6600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b0143ca0 0x7ff0b00cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0b6600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b015d310 con 0x7ff0b00c8250 2024-04-25T12:36:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0b6600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b015d030 con 0x7ff0b0159820 2024-04-25T12:36:28.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0b6600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b015d5f0 con 0x7ff0b0143ca0 2024-04-25T12:36:28.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0aea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0b0159820 0x7ff0b0156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0aea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0b0159820 0x7ff0b0156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34930/0 (socket says 172.21.15.123:34930) 2024-04-25T12:36:28.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.202+0000 7ff0aea00640 1 -- 172.21.15.123:0/296161366 learned_addr learned my addr 172.21.15.123:0/296161366 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0afe00640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b0143ca0 0x7ff0b00cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0af400640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00c8250 0x7ff0b01420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0aea00640 1 -- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b0143ca0 msgr2=0x7ff0b00cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0aea00640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b0143ca0 0x7ff0b00cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0aea00640 1 -- 172.21.15.123:0/296161366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00c8250 msgr2=0x7ff0b01420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0aea00640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00c8250 0x7ff0b01420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0aea00640 1 -- 172.21.15.123:0/296161366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b0142600 con 0x7ff0b0159820 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0af400640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00c8250 0x7ff0b01420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0aea00640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0b0159820 0x7ff0b0156130 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00d630 tx=0x7ff09c00daf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0a7400640 1 -- 172.21.15.123:0/296161366 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0125f0 con 0x7ff0b0159820 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0a7400640 1 -- 172.21.15.123:0/296161366 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c019070 con 0x7ff0b0159820 2024-04-25T12:36:28.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.203+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b0141130 con 0x7ff0b0159820 2024-04-25T12:36:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.204+0000 7ff0a7400640 1 -- 172.21.15.123:0/296161366 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0129d0 con 0x7ff0b0159820 2024-04-25T12:36:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.204+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b00ccc10 con 0x7ff0b0159820 2024-04-25T12:36:28.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.205+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b0076700 con 0x7ff0b0159820 2024-04-25T12:36:28.205 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:28.206 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:28.206 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:28.206 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:28.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.206+0000 7ff0a7400640 1 -- 172.21.15.123:0/296161366 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff09c012b70 con 0x7ff0b0159820 2024-04-25T12:36:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.208+0000 7ff0a7400640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff0900744b0 0x7ff0900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.208+0000 7ff0a7400640 1 -- 172.21.15.123:0/296161366 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6648+0+0 (secure 0 0 0) 0x7ff09c095a10 con 0x7ff0b0159820 2024-04-25T12:36:28.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.209+0000 7ff0af400640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff0900744b0 0x7ff0900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.209+0000 7ff0af400640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff0900744b0 0x7ff0900769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0034000 tx=0x7ff0a002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.212+0000 7ff0a7400640 1 -- 172.21.15.123:0/296161366 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff09c05fdf0 con 0x7ff0b0159820 2024-04-25T12:36:28.215 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:28.264 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:28.265+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:28.327 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:28.327 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:28.327 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:28.327 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:28.327 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:28.335 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:28.377 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:28.378+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:28.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:28.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.422+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7ff0b00c9590 con 0x7ff0b0159820 2024-04-25T12:36:28.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.454+0000 7f783b600640 1 Processor -- start 2024-04-25T12:36:28.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.454+0000 7f783b600640 1 -- start start 2024-04-25T12:36:28.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f783b600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f78340c9b80 0x7f78340742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f783b600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78340c8250 0x7f78340c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f783b600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7834156e60 con 0x7f7834159820 2024-04-25T12:36:28.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f783b600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f78340c8ba0 con 0x7f78340c8250 2024-04-25T12:36:28.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f783b600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f783415ca70 con 0x7f78340c9b80 2024-04-25T12:36:28.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f7838c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f78340c9b80 0x7f78340742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f7838c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f78340c9b80 0x7f78340742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34936/0 (socket says 172.21.15.123:34936) 2024-04-25T12:36:28.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.457+0000 7f7838c00640 1 -- 172.21.15.123:0/2325827787 learned_addr learned my addr 172.21.15.123:0/2325827787 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:28.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.458+0000 7f7833e00640 1 --2- 172.21.15.123:0/2325827787 >> v2:172.21.15.123:3300/0 conn(0x7f78340c8250 0x7f78340c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.458+0000 7f7838c00640 1 -- 172.21.15.123:0/2325827787 >> v2:172.21.15.123:3300/0 conn(0x7f78340c8250 msgr2=0x7f78340c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:28.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.458+0000 7f7838c00640 1 --2- 172.21.15.123:0/2325827787 >> v2:172.21.15.123:3300/0 conn(0x7f78340c8250 0x7f78340c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.458+0000 7f7838c00640 1 -- 172.21.15.123:0/2325827787 >> v1:172.21.15.153:6789/0 conn(0x7f7834159820 legacy=0x7f7834159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:28.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.458+0000 7f7838c00640 1 -- 172.21.15.123:0/2325827787 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78340c7130 con 0x7f78340c9b80 2024-04-25T12:36:28.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.458+0000 7f7833e00640 1 --2- 172.21.15.123:0/2325827787 >> v2:172.21.15.123:3300/0 conn(0x7f78340c8250 0x7f78340c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:28.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.460+0000 7f7838c00640 1 --2- 172.21.15.123:0/2325827787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78340c9b80 0x7f78340742b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7834074b80 tx=0x7f782802c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ce49a67f53dfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.460+0000 7f7833400640 1 -- 172.21.15.123:0/2325827787 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828038070 con 0x7f78340c9b80 2024-04-25T12:36:28.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.460+0000 7f7833400640 1 -- 172.21.15.123:0/2325827787 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7828031710 con 0x7f78340c9b80 2024-04-25T12:36:28.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.460+0000 7f7833400640 1 -- 172.21.15.123:0/2325827787 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828031ae0 con 0x7f78340c9b80 2024-04-25T12:36:28.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.461+0000 7f783b600640 1 -- 172.21.15.123:0/2325827787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78340c9b80 msgr2=0x7f78340742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.461+0000 7f783b600640 1 --2- 172.21.15.123:0/2325827787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78340c9b80 0x7f78340742b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7834074b80 tx=0x7f782802c440 comp rx=0 tx=0).stop 2024-04-25T12:36:28.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.461+0000 7f783b600640 1 -- 172.21.15.123:0/2325827787 shutdown_connections 2024-04-25T12:36:28.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.461+0000 7f783b600640 1 --2- 172.21.15.123:0/2325827787 >> v2:172.21.15.123:3300/0 conn(0x7f78340c8250 0x7f78340c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.461+0000 7f783b600640 1 --2- 172.21.15.123:0/2325827787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78340c9b80 0x7f78340742b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.461+0000 7f783b600640 1 -- 172.21.15.123:0/2325827787 >> 172.21.15.123:0/2325827787 conn(0x7f783406c750 msgr2=0x7f783406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:28.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.461+0000 7f783b600640 1 -- 172.21.15.123:0/2325827787 shutdown_connections 2024-04-25T12:36:28.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.462+0000 7f783b600640 1 -- 172.21.15.123:0/2325827787 wait complete. 2024-04-25T12:36:28.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.465+0000 7f783b600640 1 Processor -- start 2024-04-25T12:36:28.466 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:28.466+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:28.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.465+0000 7f783b600640 1 -- start start 2024-04-25T12:36:28.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f783b600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78340c8250 0x7f78341fab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f783b600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78340c9b80 0x7f783414e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f783b600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7834159820 0x7f783414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f783b600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f783415d310 con 0x7f78340c8250 2024-04-25T12:36:28.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f783b600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f783415d030 con 0x7f7834159820 2024-04-25T12:36:28.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f783b600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f783415d5f0 con 0x7f78340c9b80 2024-04-25T12:36:28.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f7839600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7834159820 0x7f783414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f7833e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78340c9b80 0x7f783414e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f7839600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7834159820 0x7f783414ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:34944/0 (socket says 172.21.15.123:34944) 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f7839600640 1 -- 172.21.15.123:0/1691834014 learned_addr learned my addr 172.21.15.123:0/1691834014 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.468+0000 7f7838c00640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78340c8250 0x7f78341fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.469+0000 7f7833e00640 1 -- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7834159820 msgr2=0x7f783414ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.469+0000 7f7833e00640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7834159820 0x7f783414ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.469+0000 7f7833e00640 1 -- 172.21.15.123:0/1691834014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78340c8250 msgr2=0x7f78341fab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.469+0000 7f7833e00640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78340c8250 0x7f78341fab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.469+0000 7f7833e00640 1 -- 172.21.15.123:0/1691834014 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78341545e0 con 0x7f78340c9b80 2024-04-25T12:36:28.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.469+0000 7f7833e00640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78340c9b80 0x7f783414e3d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7834070ac0 tx=0x7f782000da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.470+0000 7f7830c00640 1 -- 172.21.15.123:0/1691834014 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820018020 con 0x7f78340c9b80 2024-04-25T12:36:28.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.470+0000 7f7830c00640 1 -- 172.21.15.123:0/1691834014 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7820014620 con 0x7f78340c9b80 2024-04-25T12:36:28.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.470+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78341fb0c0 con 0x7f78340c9b80 2024-04-25T12:36:28.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.470+0000 7f7830c00640 1 -- 172.21.15.123:0/1691834014 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820014a10 con 0x7f78340c9b80 2024-04-25T12:36:28.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.470+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7834154300 con 0x7f78340c9b80 2024-04-25T12:36:28.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.471+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7834076700 con 0x7f78340c9b80 2024-04-25T12:36:28.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.472+0000 7f7830c00640 1 -- 172.21.15.123:0/1691834014 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f782001b020 con 0x7f78340c9b80 2024-04-25T12:36:28.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.474+0000 7f7830c00640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78140744b0 0x7f78140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.474+0000 7f7830c00640 1 -- 172.21.15.123:0/1691834014 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 6648+0+0 (secure 0 0 0) 0x7f7820095850 con 0x7f78340c9b80 2024-04-25T12:36:28.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.474+0000 7f7838c00640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78140744b0 0x7f78140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.475+0000 7f7838c00640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78140744b0 0x7f78140769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7828035000 tx=0x7f7828008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.478+0000 7f7830c00640 1 -- 172.21.15.123:0/1691834014 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7820022020 con 0x7f78340c9b80 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.542395758, 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.492 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:36:28.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:36:28.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:28.495 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:36:28.496 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.54230804900001, 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:36:28.497 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.498 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.499 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.542182121, 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:36:28.502 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.504 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:36:28.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:36:28.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.54212046200001, 2024-04-25T12:36:28.507 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:36:28.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:36:28.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:36:28.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.512 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.54206369900001, 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.513 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.515 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.517 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.542000229, 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:28.518 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:36:28.519 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:36:28.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:36:28.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.522 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.54194828000001, 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:36:28.526 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:36:28.527 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.54189632999999, 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.528 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:28.529 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.530 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:28.531 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.532 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.54177922100001, 2024-04-25T12:36:28.533 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.534 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:36:28.535 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.536 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.537 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.538 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 129.541740774, 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:28.539 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:36:28.540 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:36:28.541 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.376605721000004, 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.542 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.543 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:36:28.544 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:36:28.545 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.546 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.376547669999994, 2024-04-25T12:36:28.547 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.548 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.549 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.550 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:36:28.551 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 98.376487976999996, 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.553 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.554 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.557 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.339567895000002, 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.558 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.559 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.560 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.561 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.562 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.339478928999995, 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.563 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.564 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:36:28.565 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:36:28.566 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:28.567 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.339435088000002, 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.568 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.569 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.570 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.571 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.572 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.337921183999995, 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.573 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.574 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:36:28.575 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:28.576 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:36:28.577 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.337001967999996, 2024-04-25T12:36:28.578 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.579 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.580 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.581 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:36:28.582 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.583 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.336546615000003, 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:28.584 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:36:28.585 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.586 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.587 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 93.336517637, 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.588 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.589 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.590 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:28.591 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:28.593 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:28.593 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:28.593+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:28.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.685+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f78340c9590 con 0x7f78340c9b80 2024-04-25T12:36:28.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.687+0000 7f7830c00640 1 -- 172.21.15.123:0/1691834014 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v58) ==== 81+0+38078 (secure 0 0 0) 0x7f782005f5d0 con 0x7f78340c9b80 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "epoch": 58, 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "fsid": "2239057a-dc25-4ee3-8341-8c80e403f296", 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "created": "2024-04-25T12:33:54.601604+0000", 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "modified": "2024-04-25T12:36:25.882172+0000", 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "last_up_change": "2024-04-25T12:35:59.575545+0000", 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "last_in_change": "2024-04-25T12:33:58.290536+0000", 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_num": 5801984, 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_set": [ 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "noscrub", 2024-04-25T12:36:28.688 INFO:teuthology.orchestra.run.smithi123.stdout: "pglog_hardlimit", 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "purged_snapdirs", 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_deletes", 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "sortbitwise" 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_version": 7, 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_snapshot": "", 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_max": 4, 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd": 8, 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "require_min_compat_client": "luminous", 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "min_compat_client": "luminous", 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "require_osd_release": "squid", 2024-04-25T12:36:28.689 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_crimson": false, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "pools": [ 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 1, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": ".mgr", 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:06.827674+0000", 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:28.690 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 1, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 1, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 1, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 1, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 1, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:28.691 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "17", 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:28.692 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:28.693 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:28.694 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_max": 32, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 1 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": {} 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 8, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 8, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 0.25, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:36:28.695 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 2, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_metadata", 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:11.030660+0000", 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:28.696 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "20", 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:28.699 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:28.700 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_bias": 4, 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 16, 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_priority": 5 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": "cephfs" 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 2.5, 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 2.5, 2024-04-25T12:36:28.701 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 2.5, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 2.5, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 3, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data", 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:12.395253+0000", 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool", 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 1, 2024-04-25T12:36:28.702 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 2, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 1, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 0, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 64, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 64, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 64, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 64, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 64, 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:28.703 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "21", 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:28.704 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:28.705 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "", 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.05000000074505806 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "read_balance": { 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "score_type": "Fair distribution", 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "score_acting": 1.5, 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "score_stable": 1.5, 2024-04-25T12:36:28.707 INFO:teuthology.orchestra.run.smithi123.stdout: "optimal_score": 1, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_acting": 1.5, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "raw_score_stable": 1.5, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity_weighted": 1, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity": 1, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "average_primary_affinity_weighted": 1 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "pool": 4, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": "cephfs_data_ec", 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "create_time": "2024-04-25T12:34:15.975923+0000", 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 5, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "type": 3, 2024-04-25T12:36:28.708 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 4, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": 3, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_rule": 1, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_count": 0, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_target": 0, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "object_hash": 2, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscale_mode": "off", 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": 32, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num": 32, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_placement_num_target": 32, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_target": 32, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_pending": 32, 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "last_pg_merge_meta": { 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "source_pgid": "0.0", 2024-04-25T12:36:28.709 INFO:teuthology.orchestra.run.smithi123.stdout: "ready_epoch": 0, 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_started": 0, 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "last_epoch_clean": 0, 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "source_version": "0'0", 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "target_version": "0'0" 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "last_change": "25", 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend": "0", 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "auid": 0, 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_mode": "selfmanaged", 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_seq": 0, 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "snap_epoch": 0, 2024-04-25T12:36:28.710 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_snaps": [], 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps": "[]", 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_bytes": 0, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "quota_max_objects": 0, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "tiers": [], 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "tier_of": -1, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "read_tier": -1, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "write_tier": -1, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_mode": "none", 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_bytes": 0, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "target_max_objects": 0, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_flush_age": 0, 2024-04-25T12:36:28.711 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_min_evict_age": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_params": { 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "none" 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_period": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_count": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "use_gmt_hitset": true, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "min_read_recency_for_promote": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "min_write_recency_for_promote": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "hit_set_search_last_n": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "grade_table": [], 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "stripe_width": 8192, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "expected_num_objects": 0, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "fast_read": false, 2024-04-25T12:36:28.712 INFO:teuthology.orchestra.run.smithi123.stdout: "options": { 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num_min": 64, 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "application_metadata": { 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs": { 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "data": "cephfs" 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "osds": [ 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "2d9ee216-abfa-45db-a0ac-ef834981be6f", 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.713 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6816", 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6817", 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6818", 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6819", 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6822", 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.716 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6823", 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6820", 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6821", 2024-04-25T12:36:28.717 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1405705428 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6817/1405705428", 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6819/1405705428", 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6823/1405705428", 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6821/1405705428", 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "a60a3a6f-376a-4fb2-baf1-e13fc615f6e5", 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6800", 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6801", 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.720 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6802", 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6803", 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.721 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6806", 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6807", 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6804", 2024-04-25T12:36:28.722 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6805", 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1234958824 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6801/1234958824", 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6803/1234958824", 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6807/1234958824", 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6805/1234958824", 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.723 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "cfb7cbbd-82cd-46fe-b89b-15775df1fe93", 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:28.724 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6808", 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6809", 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.725 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6810", 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6811", 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.726 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6814", 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6815", 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6812", 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6813", 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2222173849 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6809/2222173849", 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6811/2222173849", 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6815/2222173849", 2024-04-25T12:36:28.728 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6813/2222173849", 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9e8ccd97-ab68-4f03-bbb3-5820eccf4568", 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:28.729 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 15, 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 53, 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6824", 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6825", 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.730 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6826", 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6827", 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.731 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6830", 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6831", 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6828", 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6829", 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1491071423 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.733 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.123:6825/1491071423", 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.123:6827/1491071423", 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.123:6831/1491071423", 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.123:6829/1491071423", 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "7d5bfd6e-173a-413d-abdb-df6d82c941d3", 2024-04-25T12:36:28.734 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6802", 2024-04-25T12:36:28.735 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6803", 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6804", 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6805", 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6808", 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6809", 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6806", 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6807", 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3135982807 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6803/3135982807", 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6805/3135982807", 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6809/3135982807", 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6807/3135982807", 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.739 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "9141fa91-c547-4711-910a-9ef123017fb5", 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 14, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 39, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 47, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 40, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.740 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6826", 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6827", 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.741 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6828", 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6829", 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6832", 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6833", 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6830", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6831", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4270711740 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6827/4270711740", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6829/4270711740", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6833/4270711740", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6831/4270711740", 2024-04-25T12:36:28.744 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "66fbc935-086e-4b03-aa03-96e0bc8b1d27", 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:28.745 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6810", 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6811", 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.746 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6812", 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6813", 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6816", 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6817", 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6814", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6815", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3626282289 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6811/3626282289", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6813/3626282289", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6817/3626282289", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6815/3626282289", 2024-04-25T12:36:28.749 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "uuid": "de59a619-b84a-4623-ac89-447178815d65", 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "up": 1, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "in": 1, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "weight": 1, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_affinity": 1, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_begin": 0, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "last_clean_end": 0, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "up_from": 14, 2024-04-25T12:36:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: "up_thru": 47, 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "down_at": 0, 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "lost_at": 0, 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addrs": { 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6818", 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6819", 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addrs": { 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6820", 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6821", 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addrs": { 2024-04-25T12:36:28.752 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6824", 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6825", 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addrs": { 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-04-25T12:36:28.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6822", 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.153:6823", 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3086604284 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "public_addr": "172.21.15.153:6819/3086604284", 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_addr": "172.21.15.153:6821/3086604284", 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_back_addr": "172.21.15.153:6825/3086604284", 2024-04-25T12:36:28.754 INFO:teuthology.orchestra.run.smithi123.stdout: "heartbeat_front_addr": "172.21.15.153:6823/3086604284", 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "state": [ 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "exists", 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "up" 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_xinfo": [ 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 0, 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.805055+0000", 2024-04-25T12:36:28.755 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 1, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.746266+0000", 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 2, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.756 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.706883+0000", 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 3, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.826277+0000", 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 4, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.402228+0000", 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 5, 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.536018+0000", 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 40 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 6, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.375016+0000", 2024-04-25T12:36:28.759 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": 7, 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "down_stamp": "0.000000", 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_probability": 0, 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "laggy_interval": 0, 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "features": 4540701547738038271, 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "old_weight": 0, 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "last_purged_snaps_scrub": "2024-04-25T12:34:03.454835+0000", 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "dead_epoch": 0 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap": [ 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.760 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "2.8", 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: "osds": [ 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: 3, 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: 1 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_items": [ 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "1.0", 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 5, 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 4 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.761 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "3.15", 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 0, 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 7 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: "pgid": "4.d", 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: "mappings": [ 2024-04-25T12:36:28.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "from": 3, 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "to": 0 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_upmap_primaries": [], 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_temp": [], 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "primary_temp": [], 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklist": {}, 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "range_blocklist": {}, 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "erasure_code_profiles": { 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "cephfs_data_ec": { 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-device-class": "", 2024-04-25T12:36:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-num-failure-domains": "0", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-osds-per-failure-domain": "0", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-root": "default", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "jerasure-per-chunk-alignment": "false", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "2", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "w": "8" 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "default": { 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-failure-domain": "osd", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "k": "2", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "m": "1", 2024-04-25T12:36:28.764 INFO:teuthology.orchestra.run.smithi123.stdout: "plugin": "jerasure", 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "technique": "reed_sol_van" 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "removed_snaps_queue": [], 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "new_removed_snaps": [], 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "new_purged_snaps": [], 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_node_flags": {}, 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "device_class_flags": {}, 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode": { 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_enabled": false, 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_bucket_count": 0, 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "degraded_stretch_mode": 0, 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "recovering_stretch_mode": 0, 2024-04-25T12:36:28.765 INFO:teuthology.orchestra.run.smithi123.stdout: "stretch_mode_bucket": 0 2024-04-25T12:36:28.766 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:28.766 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:28.767 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:28.768 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:28.768 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:28.768 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:28.768 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.689+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78140744b0 msgr2=0x7f78140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.689+0000 7f783b600640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78140744b0 0x7f78140769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7828035000 tx=0x7f7828008030 comp rx=0 tx=0).stop 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.689+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78340c9b80 msgr2=0x7f783414e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.689+0000 7f783b600640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78340c9b80 0x7f783414e3d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7834070ac0 tx=0x7f782000da50 comp rx=0 tx=0).stop 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 shutdown_connections 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f78140744b0 0x7f78140769c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7834159820 0x7f783414ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78340c9b80 0x7f783414e3d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 --2- 172.21.15.123:0/1691834014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78340c8250 0x7f78341fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 >> 172.21.15.123:0/1691834014 conn(0x7f783406c750 msgr2=0x7f7834073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:28.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 shutdown_connections 2024-04-25T12:36:28.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.690+0000 7f783b600640 1 -- 172.21.15.123:0/1691834014 wait complete. 2024-04-25T12:36:28.771 INFO:tasks.thrashosds.thrasher:j is {'epoch': 58, 'fsid': '2239057a-dc25-4ee3-8341-8c80e403f296', 'created': '2024-04-25T12:33:54.601604+0000', 'modified': '2024-04-25T12:36:25.882172+0000', 'last_up_change': '2024-04-25T12:35:59.575545+0000', 'last_in_change': '2024-04-25T12:33:58.290536+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': '2d9ee216-abfa-45db-a0ac-ef834981be6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 1405705428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 1405705428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 1405705428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 1405705428}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 1405705428}]}, 'public_addr': '172.21.15.123:6817/1405705428', 'cluster_addr': '172.21.15.123:6819/1405705428', 'heartbeat_back_addr': '172.21.15.123:6823/1405705428', 'heartbeat_front_addr': '172.21.15.123:6821/1405705428', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a60a3a6f-376a-4fb2-baf1-e13fc615f6e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 1234958824}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 1234958824}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 1234958824}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 1234958824}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 1234958824}]}, 'public_addr': '172.21.15.123:6801/1234958824', 'cluster_addr': '172.21.15.123:6803/1234958824', 'heartbeat_back_addr': '172.21.15.123:6807/1234958824', 'heartbeat_front_addr': '172.21.15.123:6805/1234958824', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'cfb7cbbd-82cd-46fe-b89b-15775df1fe93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 2222173849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 2222173849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 2222173849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 2222173849}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 2222173849}]}, 'public_addr': '172.21.15.123:6809/2222173849', 'cluster_addr': '172.21.15.123:6811/2222173849', 'heartbeat_back_addr': '172.21.15.123:6815/2222173849', 'heartbeat_front_addr': '172.21.15.123:6813/2222173849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9e8ccd97-ab68-4f03-bbb3-5820eccf4568', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 1491071423}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 1491071423}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 1491071423}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 1491071423}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 1491071423}]}, 'public_addr': '172.21.15.123:6825/1491071423', 'cluster_addr': '172.21.15.123:6827/1491071423', 'heartbeat_back_addr': '172.21.15.123:6831/1491071423', 'heartbeat_front_addr': '172.21.15.123:6829/1491071423', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d5bfd6e-173a-413d-abdb-df6d82c941d3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6802', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6803', 'nonce': 3135982807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6804', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6805', 'nonce': 3135982807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6808', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6809', 'nonce': 3135982807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6806', 'nonce': 3135982807}, {'type': 'v1', 'addr': '172.21.15.153:6807', 'nonce': 3135982807}]}, 'public_addr': '172.21.15.153:6803/3135982807', 'cluster_addr': '172.21.15.153:6805/3135982807', 'heartbeat_back_addr': '172.21.15.153:6809/3135982807', 'heartbeat_front_addr': '172.21.15.153:6807/3135982807', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9141fa91-c547-4711-910a-9ef123017fb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 39, 'up_from': 47, 'up_thru': 47, 'down_at': 40, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6826', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6827', 'nonce': 4270711740}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6828', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6829', 'nonce': 4270711740}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 4270711740}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6830', 'nonce': 4270711740}, {'type': 'v1', 'addr': '172.21.15.153:6831', 'nonce': 4270711740}]}, 'public_addr': '172.21.15.153:6827/4270711740', 'cluster_addr': '172.21.15.153:6829/4270711740', 'heartbeat_back_addr': '172.21.15.153:6833/4270711740', 'heartbeat_front_addr': '172.21.15.153:6831/4270711740', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '66fbc935-086e-4b03-aa03-96e0bc8b1d27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6810', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6811', 'nonce': 3626282289}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6812', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6813', 'nonce': 3626282289}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6816', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6817', 'nonce': 3626282289}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6814', 'nonce': 3626282289}, {'type': 'v1', 'addr': '172.21.15.153:6815', 'nonce': 3626282289}]}, 'public_addr': '172.21.15.153:6811/3626282289', 'cluster_addr': '172.21.15.153:6813/3626282289', 'heartbeat_back_addr': '172.21.15.153:6817/3626282289', 'heartbeat_front_addr': '172.21.15.153:6815/3626282289', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'de59a619-b84a-4623-ac89-447178815d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6818', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6819', 'nonce': 3086604284}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6820', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6821', 'nonce': 3086604284}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6824', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6825', 'nonce': 3086604284}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6822', 'nonce': 3086604284}, {'type': 'v1', 'addr': '172.21.15.153:6823', 'nonce': 3086604284}]}, 'public_addr': '172.21.15.153:6819/3086604284', 'cluster_addr': '172.21.15.153:6821/3086604284', 'heartbeat_back_addr': '172.21.15.153:6825/3086604284', 'heartbeat_front_addr': '172.21.15.153:6823/3086604284', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.805055+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.746266+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.706883+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.826277+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.402228+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.536018+0000', 'dead_epoch': 40}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.375016+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T12:34:03.454835+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.8', 'osds': [3, 1]}], 'pg_upmap_items': [{'pgid': '1.0', 'mappings': [{'from': 5, 'to': 4}]}, {'pgid': '3.15', 'mappings': [{'from': 0, 'to': 7}]}, {'pgid': '4.d', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-25T12:36:28.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:36:28.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.756+0000 7f0883200640 1 Processor -- start 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.756+0000 7f0883200640 1 -- start start 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.758+0000 7f0883200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088406e280 0x7f0884167da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.758+0000 7f0883200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0884072010 con 0x7f088415ce60 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.758+0000 7f0883200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08840cae10 con 0x7f0884156760 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.758+0000 7f0883200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f088406d6e0 con 0x7f088406e280 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0881e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0884156760 0x7f08840c4f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60252/0 (socket says 172.21.15.123:60252) 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0881e00640 1 -- 172.21.15.123:0/3793986031 learned_addr learned my addr 172.21.15.123:0/3793986031 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0882800640 1 --2- 172.21.15.123:0/3793986031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088406e280 0x7f0884167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0880a00640 1 -- 172.21.15.123:0/3793986031 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810472235 0 0) 0x7f08840cae10 con 0x7f0884156760 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0880a00640 1 -- 172.21.15.123:0/3793986031 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0868003520 con 0x7f0884156760 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0882800640 1 -- 172.21.15.123:0/3793986031 >> v1:172.21.15.153:6789/0 conn(0x7f0884156760 legacy=0x7f08840c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0882800640 1 -- 172.21.15.123:0/3793986031 >> v1:172.21.15.123:6789/0 conn(0x7f088415ce60 legacy=0x7f088406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0880a00640 1 -- 172.21.15.123:0/3793986031 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056616060 0 0) 0x7f0884072010 con 0x7f088415ce60 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.759+0000 7f0882800640 1 -- 172.21.15.123:0/3793986031 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088415d270 con 0x7f088406e280 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.761+0000 7f0882800640 1 --2- 172.21.15.123:0/3793986031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088406e280 0x7f0884167da0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f088406d860 tx=0x7f087c02a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de0d65d70b0924c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.761+0000 7f0880a00640 1 -- 172.21.15.123:0/3793986031 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c007740 con 0x7f088406e280 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.761+0000 7f0880a00640 1 -- 172.21.15.123:0/3793986031 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087c0078e0 con 0x7f088406e280 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.761+0000 7f0880a00640 1 -- 172.21.15.123:0/3793986031 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c007cb0 con 0x7f088406e280 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.762+0000 7f0883200640 1 -- 172.21.15.123:0/3793986031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088406e280 msgr2=0x7f0884167da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.762+0000 7f0883200640 1 --2- 172.21.15.123:0/3793986031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088406e280 0x7f0884167da0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f088406d860 tx=0x7f087c02a750 comp rx=0 tx=0).stop 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.762+0000 7f0883200640 1 -- 172.21.15.123:0/3793986031 shutdown_connections 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.762+0000 7f0883200640 1 --2- 172.21.15.123:0/3793986031 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088406e280 0x7f0884167da0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.762+0000 7f0883200640 1 -- 172.21.15.123:0/3793986031 >> 172.21.15.123:0/3793986031 conn(0x7f0884074520 msgr2=0x7f08840ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.763+0000 7f0883200640 1 -- 172.21.15.123:0/3793986031 shutdown_connections 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.763+0000 7f0883200640 1 -- 172.21.15.123:0/3793986031 wait complete. 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.766+0000 7f0883200640 1 Processor -- start 2024-04-25T12:36:28.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.766+0000 7f0883200640 1 -- start start 2024-04-25T12:36:28.776 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:28.767+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:28.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0883200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f088406e280 0x7f08841579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0883200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0884156760 0x7f0884158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0883200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088415ce60 0x7f0884150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0883200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0884071a50 con 0x7f088406e280 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0883200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0884156b70 con 0x7f0884156760 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0883200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0884071d30 con 0x7f088415ce60 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0881e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f088406e280 0x7f08841579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0882800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088415ce60 0x7f0884150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0882800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088415ce60 0x7f0884150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:34602/0 (socket says 172.21.15.123:34602) 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 learned_addr learned my addr 172.21.15.123:0/1768621135 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.769+0000 7f0881400640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0884156760 0x7f0884158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.770+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0884156760 msgr2=0x7f0884158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.770+0000 7f0882800640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0884156760 0x7f0884158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.770+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f088406e280 msgr2=0x7f08841579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.770+0000 7f0882800640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f088406e280 0x7f08841579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.770+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0884159d00 con 0x7f088415ce60 2024-04-25T12:36:28.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.770+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f088406e280 0x7f08841579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.770+0000 7f0882800640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088415ce60 0x7f0884150050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f087c009d90 tx=0x7f087c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.771+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c03f070 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.771+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087c031b40 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.771+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c009450 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.771+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088415b430 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.771+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f088415b9f0 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.772+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f0884076da0 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.774+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f087c03b410 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.775+0000 7f087a000640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f08640744b0 0x7f08640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.775+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 6648+0+0 (secure 0 0 0) 0x7f087c0b61c0 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.775+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f08640744b0 0x7f08640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.775+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v58) ==== 54+0+16 (secure 0 0 0) 0x7f087c04a070 con 0x7f088415ce60 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.776+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f08841572f0 0x7f08841e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.776+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08841e6f80 con 0x7f08841572f0 2024-04-25T12:36:28.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.776+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f08640744b0 0x7f08640769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0874005880 tx=0x7f0874009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.776+0000 7f0881400640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f08841572f0 0x7f08841e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.779+0000 7f0881400640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f08841572f0 0x7f08841e6a40 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.780+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08700012a0 con 0x7f08841572f0 2024-04-25T12:36:28.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.799+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f08840cabf0 con 0x7f08841572f0 2024-04-25T12:36:28.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.801+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f08840cabf0 con 0x7f08841572f0 2024-04-25T12:36:28.801 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:28.802 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:28.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.802+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0884158ca0 0x7f08841f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.802+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f0884153af0 con 0x7f0884158ca0 2024-04-25T12:36:28.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.802+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0884158ca0 0x7f08841f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.803+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0884158ca0 0x7f08841f5890 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.804+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0884153af0 con 0x7f0884158ca0 2024-04-25T12:36:28.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.825+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f08840cedc0 con 0x7f0884158ca0 2024-04-25T12:36:28.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.827+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f08840cedc0 con 0x7f0884158ca0 2024-04-25T12:36:28.828 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:28.828 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:28.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.828+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f08841f62e0 0x7f0884062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.828+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f08841f66f0 con 0x7f08841f62e0 2024-04-25T12:36:28.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.828+0000 7f0881400640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f08841f62e0 0x7f0884062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.829+0000 7f0881400640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f08841f62e0 0x7f0884062fe0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.830+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08841f66f0 con 0x7f08841f62e0 2024-04-25T12:36:28.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.846+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f08841595e0 con 0x7f08841f62e0 2024-04-25T12:36:28.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.848+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f08841595e0 con 0x7f08841f62e0 2024-04-25T12:36:28.849 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:28.849 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:28.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.849+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0884063a30 0x7f08840c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.849+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f08841595e0 con 0x7f0884063a30 2024-04-25T12:36:28.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.849+0000 7f0882800640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0884063a30 0x7f08840c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.850+0000 7f0882800640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0884063a30 0x7f08840c8d70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.853+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08841595e0 con 0x7f0884063a30 2024-04-25T12:36:28.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.867+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f08841595e0 con 0x7f0884063a30 2024-04-25T12:36:28.868 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:28.868+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:28.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.869+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f08841595e0 con 0x7f0884063a30 2024-04-25T12:36:28.869 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:28.870 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:28.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.870+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f08840c97c0 0x7f088402c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.870+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f08841595e0 con 0x7f08840c97c0 2024-04-25T12:36:28.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.870+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f08840c97c0 0x7f088402c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.871+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f08840c97c0 0x7f088402c9d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.872+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08841595e0 con 0x7f08840c97c0 2024-04-25T12:36:28.885 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:28.885 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:28.885 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:28.885 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:28.885 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:28.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.887+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f08840c9c30 con 0x7f08840c97c0 2024-04-25T12:36:28.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.889+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f08840c9c30 con 0x7f08840c97c0 2024-04-25T12:36:28.889 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:28.889 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:28.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.889+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f088402d420 0x7f088402d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.889+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f088402ddf0 con 0x7f088402d420 2024-04-25T12:36:28.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.889+0000 7f0881400640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f088402d420 0x7f088402d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.890+0000 7f0881400640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f088402d420 0x7f088402d830 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.891+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f088402ddf0 con 0x7f088402d420 2024-04-25T12:36:28.894 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:28.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.905+0000 7ff0a7400640 1 -- 172.21.15.123:0/296161366 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v59) ==== 97+0+0 (secure 0 0 0) 0x7ff09c05f6f0 con 0x7ff0b0159820 2024-04-25T12:36:28.905 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:36:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff0900744b0 msgr2=0x7ff0900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff0900744b0 0x7ff0900769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0034000 tx=0x7ff0a002e000 comp rx=0 tx=0).stop 2024-04-25T12:36:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0b0159820 msgr2=0x7ff0b0156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0b0159820 0x7ff0b0156130 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00d630 tx=0x7ff09c00daf0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 shutdown_connections 2024-04-25T12:36:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff0900744b0 0x7ff0900769c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0b0143ca0 0x7ff0b00cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0b0159820 0x7ff0b0156130 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 --2- 172.21.15.123:0/296161366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00c8250 0x7ff0b01420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.909+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 >> 172.21.15.123:0/296161366 conn(0x7ff0b006c750 msgr2=0x7ff0b00c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.910+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 shutdown_connections 2024-04-25T12:36:28.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.910+0000 7ff0b6600640 1 -- 172.21.15.123:0/296161366 wait complete. 2024-04-25T12:36:28.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.917+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f088406eb80 con 0x7f088402d420 2024-04-25T12:36:28.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.920+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f088406eb80 con 0x7f088402d420 2024-04-25T12:36:28.920 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:28.920 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:28.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.920+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0884205110 0x7f0884207530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.920+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f088420a460 con 0x7f0884205110 2024-04-25T12:36:28.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.921+0000 7f0882800640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0884205110 0x7f0884207530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.921+0000 7f0882800640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0884205110 0x7f0884207530 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.922+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f088420a460 con 0x7f0884205110 2024-04-25T12:36:28.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.950+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0884207a70 con 0x7f0884205110 2024-04-25T12:36:28.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.952+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0884207a70 con 0x7f0884205110 2024-04-25T12:36:28.953 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:28.953 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:28.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.953+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0884208190 0x7f08842085a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:28.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.953+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f088420d8b0 con 0x7f0884208190 2024-04-25T12:36:28.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.953+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0884208190 0x7f08842085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:28.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.954+0000 7f0881e00640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0884208190 0x7f08842085a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.955+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f088420d8b0 con 0x7f0884208190 2024-04-25T12:36:28.969 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:28.970+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:28.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.970+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f088420c730 con 0x7f0884208190 2024-04-25T12:36:28.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f087a000640 1 -- 172.21.15.123:0/1768621135 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f088420c730 con 0x7f0884208190 2024-04-25T12:36:28.972 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:28.972 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f08841572f0 msgr2=0x7f08841e6a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f08841572f0 0x7f08841e6a40 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0884158ca0 msgr2=0x7f08841f5890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f0884158ca0 0x7f08841f5890 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f08841f62e0 msgr2=0x7f0884062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f08841f62e0 0x7f0884062fe0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0884063a30 msgr2=0x7f08840c8d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f0884063a30 0x7f08840c8d70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f08840c97c0 msgr2=0x7f088402c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f08840c97c0 0x7f088402c9d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f088402d420 msgr2=0x7f088402d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f088402d420 0x7f088402d830 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0884205110 msgr2=0x7f0884207530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0884205110 0x7f0884207530 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0884208190 msgr2=0x7f08842085a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.972+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f0884208190 0x7f08842085a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 reap_dead start 2024-04-25T12:36:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 reap_dead start 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 reap_dead start 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 reap_dead start 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0882800640 1 -- 172.21.15.123:0/1768621135 reap_dead start 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f08640744b0 msgr2=0x7f08640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f08640744b0 0x7f08640769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0874005880 tx=0x7f0874009040 comp rx=0 tx=0).stop 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088415ce60 msgr2=0x7f0884150050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.973+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088415ce60 0x7f0884150050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f087c009d90 tx=0x7f087c004a40 comp rx=0 tx=0).stop 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.974+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 shutdown_connections 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.974+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f08640744b0 0x7f08640769c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.974+0000 7f0883200640 1 --2- 172.21.15.123:0/1768621135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f088415ce60 0x7f0884150050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.974+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 >> 172.21.15.123:0/1768621135 conn(0x7f0884074520 msgr2=0x7f0884150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:28.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.974+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 shutdown_connections 2024-04-25T12:36:28.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:28.974+0000 7f0883200640 1 -- 172.21.15.123:0/1768621135 wait complete. 2024-04-25T12:36:29.009 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:29.009 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:29.018 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.043+0000 7f5d67000640 1 Processor -- start 2024-04-25T12:36:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.043+0000 7f5d67000640 1 -- start start 2024-04-25T12:36:29.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.045+0000 7f5d67000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5d600c4d50 0x7f5d60154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.045+0000 7f5d67000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d6015afa0 0x7f5d600cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.045+0000 7f5d67000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6015b3b0 con 0x7f5d60074870 2024-04-25T12:36:29.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.045+0000 7f5d67000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5d600c61f0 con 0x7f5d6015afa0 2024-04-25T12:36:29.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.045+0000 7f5d67000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f5d600c87d0 con 0x7f5d600c4d50 2024-04-25T12:36:29.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5fe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5d600c4d50 0x7f5d60154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5f400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5d60074870 0x7f5d600c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:60262/0 (socket says 172.21.15.123:60262) 2024-04-25T12:36:29.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5f400640 1 -- 172.21.15.123:0/468380414 learned_addr learned my addr 172.21.15.123:0/468380414 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:29.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d65000640 1 --2- 172.21.15.123:0/468380414 >> v2:172.21.15.123:3300/0 conn(0x7f5d6015afa0 0x7f5d600cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5fe00640 1 -- 172.21.15.123:0/468380414 >> v2:172.21.15.123:3300/0 conn(0x7f5d6015afa0 msgr2=0x7f5d600cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:29.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5fe00640 1 --2- 172.21.15.123:0/468380414 >> v2:172.21.15.123:3300/0 conn(0x7f5d6015afa0 0x7f5d600cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5fe00640 1 -- 172.21.15.123:0/468380414 >> v1:172.21.15.153:6789/0 conn(0x7f5d60074870 legacy=0x7f5d600c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5fe00640 1 -- 172.21.15.123:0/468380414 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d60073ce0 con 0x7f5d600c4d50 2024-04-25T12:36:29.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d5ea00640 1 -- 172.21.15.123:0/468380414 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 570022507 0 0) 0x7f5d6015b3b0 con 0x7f5d60074870 2024-04-25T12:36:29.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.046+0000 7f5d65000640 1 --2- 172.21.15.123:0/468380414 >> v2:172.21.15.123:3300/0 conn(0x7f5d6015afa0 0x7f5d600cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:29.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.047+0000 7f5d5fe00640 1 --2- 172.21.15.123:0/468380414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d600c4d50 0x7f5d60154890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d600ce6d0 tx=0x7f5d5002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293818839ec30e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:29.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d5ea00640 1 -- 172.21.15.123:0/468380414 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50037070 con 0x7f5d600c4d50 2024-04-25T12:36:29.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d5ea00640 1 -- 172.21.15.123:0/468380414 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d50032440 con 0x7f5d600c4d50 2024-04-25T12:36:29.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d5ea00640 1 -- 172.21.15.123:0/468380414 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50032810 con 0x7f5d600c4d50 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d67000640 1 -- 172.21.15.123:0/468380414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d600c4d50 msgr2=0x7f5d60154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d67000640 1 --2- 172.21.15.123:0/468380414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d600c4d50 0x7f5d60154890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d600ce6d0 tx=0x7f5d5002a230 comp rx=0 tx=0).stop 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d67000640 1 -- 172.21.15.123:0/468380414 shutdown_connections 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d67000640 1 --2- 172.21.15.123:0/468380414 >> v2:172.21.15.123:3300/0 conn(0x7f5d6015afa0 0x7f5d600cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d67000640 1 --2- 172.21.15.123:0/468380414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d600c4d50 0x7f5d60154890 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.048+0000 7f5d67000640 1 -- 172.21.15.123:0/468380414 >> 172.21.15.123:0/468380414 conn(0x7f5d6006c750 msgr2=0x7f5d6006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.049+0000 7f5d67000640 1 -- 172.21.15.123:0/468380414 shutdown_connections 2024-04-25T12:36:29.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.049+0000 7f5d67000640 1 -- 172.21.15.123:0/468380414 wait complete. 2024-04-25T12:36:29.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.051+0000 7f5d67000640 1 Processor -- start 2024-04-25T12:36:29.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.051+0000 7f5d67000640 1 -- start start 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.052+0000 7f5d67000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d60074870 0x7f5d601f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.052+0000 7f5d67000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600c4d50 0x7f5d6014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.052+0000 7f5d67000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015afa0 0x7f5d601e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.052+0000 7f5d67000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d600c8460 con 0x7f5d60074870 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.052+0000 7f5d67000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d600742a0 con 0x7f5d6015afa0 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.052+0000 7f5d67000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5d600c9370 con 0x7f5d600c4d50 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d60074870 0x7f5d601f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d60074870 0x7f5d601f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60944/0 (socket says 172.21.15.123:60944) 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 -- 172.21.15.123:0/3352032840 learned_addr learned my addr 172.21.15.123:0/3352032840 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d65000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015afa0 0x7f5d601e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5f400640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600c4d50 0x7f5d6014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 -- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600c4d50 msgr2=0x7f5d6014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600c4d50 0x7f5d6014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 -- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015afa0 msgr2=0x7f5d601e68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015afa0 0x7f5d601e68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d5fe00640 1 -- 172.21.15.123:0/3352032840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d601f4650 con 0x7f5d60074870 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.053+0000 7f5d65000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015afa0 0x7f5d601e68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.054+0000 7f5d5fe00640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d60074870 0x7f5d601f4110 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d50035040 tx=0x7f5d500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.054+0000 7f5d57e00640 1 -- 172.21.15.123:0/3352032840 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50037050 con 0x7f5d60074870 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.054+0000 7f5d57e00640 1 -- 172.21.15.123:0/3352032840 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d50038040 con 0x7f5d60074870 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.054+0000 7f5d57e00640 1 -- 172.21.15.123:0/3352032840 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50004470 con 0x7f5d60074870 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.054+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6014ee50 con 0x7f5d60074870 2024-04-25T12:36:29.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.054+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6015dc50 con 0x7f5d60074870 2024-04-25T12:36:29.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.056+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d60076700 con 0x7f5d60074870 2024-04-25T12:36:29.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.057+0000 7f5d57e00640 1 -- 172.21.15.123:0/3352032840 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d50030980 con 0x7f5d60074870 2024-04-25T12:36:29.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.058+0000 7f5d57e00640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5d400788f0 0x7f5d4007ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.058+0000 7f5d57e00640 1 -- 172.21.15.123:0/3352032840 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6648+0+0 (secure 0 0 0) 0x7f5d500b6090 con 0x7f5d60074870 2024-04-25T12:36:29.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.058+0000 7f5d5f400640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5d400788f0 0x7f5d4007ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.060+0000 7f5d5f400640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5d400788f0 0x7f5d4007ae00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d600ca5d0 tx=0x7f5d4c007e40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:29.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.062+0000 7f5d57e00640 1 -- 172.21.15.123:0/3352032840 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d5007fe10 con 0x7f5d60074870 2024-04-25T12:36:29.102 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:29.103+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:29.169 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:29.170 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:29.170 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:29.171+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:29.179 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:29.271 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:29.272+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:29.331 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:29.331 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:29.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:29.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.370+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d6015d9e0 con 0x7f5d400788f0 2024-04-25T12:36:29.372 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:29.372+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:29.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.379+0000 7f5d57e00640 1 -- 172.21.15.123:0/3352032840 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332996 (secure 0 0 0) 0x7f5d6015d9e0 con 0x7f5d400788f0 2024-04-25T12:36:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:36:29.385 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:36:29.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5d400788f0 msgr2=0x7f5d4007ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5d400788f0 0x7f5d4007ae00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d600ca5d0 tx=0x7f5d4c007e40 comp rx=0 tx=0).stop 2024-04-25T12:36:29.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d60074870 msgr2=0x7f5d601f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d60074870 0x7f5d601f4110 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d50035040 tx=0x7f5d500046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 shutdown_connections 2024-04-25T12:36:29.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f5d400788f0 0x7f5d4007ae00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d6015afa0 0x7f5d601e68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d600c4d50 0x7f5d6014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 --2- 172.21.15.123:0/3352032840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d60074870 0x7f5d601f4110 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 >> 172.21.15.123:0/3352032840 conn(0x7f5d6006c750 msgr2=0x7f5d600c64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:29.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 shutdown_connections 2024-04-25T12:36:29.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.385+0000 7f5d67000640 1 -- 172.21.15.123:0/3352032840 wait complete. 2024-04-25T12:36:29.425 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-25T12:36:28.903522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601885,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":242,"num_write_kb":1682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222844,"kb_used_data":5464,"kb_used_omap":20,"kb_used_meta":217259,"kb_avail":754751876,"statfs":{"total":773094113280,"available":772865921024,"internally_reserved":0,"allocated":5595136,"data_stored":3856210,"data_compressed":66734,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":20525,"internal_metadata":222474195},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.192100"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895949+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:36:25.895949+0000","last_peered":"2024-04-25T12:36:25.895949+0000","last_clean":"2024-04-25T12:36:25.895949+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:36:25.895949+0000","last_undegraded":"2024-04-25T12:36:25.895949+0000","last_fullsized":"2024-04-25T12:36:25.895949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460654+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:36:26.460654+0000","last_peered":"2024-04-25T12:36:26.460654+0000","last_clean":"2024-04-25T12:36:26.460654+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:36:26.460654+0000","last_undegraded":"2024-04-25T12:36:26.460654+0000","last_fullsized":"2024-04-25T12:36:26.460654+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907030+0000","last_change":"2024-04-25T12:34:18.006972+0000","last_active":"2024-04-25T12:36:25.907030+0000","last_peered":"2024-04-25T12:36:25.907030+0000","last_clean":"2024-04-25T12:36:25.907030+0000","last_became_active":"2024-04-25T12:34:18.006555+0000","last_became_peered":"2024-04-25T12:34:18.006555+0000","last_unstale":"2024-04-25T12:36:25.907030+0000","last_undegraded":"2024-04-25T12:36:25.907030+0000","last_fullsized":"2024-04-25T12:36:25.907030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:41.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":56,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460480+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:36:26.460480+0000","last_peered":"2024-04-25T12:36:26.460480+0000","last_clean":"2024-04-25T12:36:26.460480+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:36:26.460480+0000","last_undegraded":"2024-04-25T12:36:26.460480+0000","last_fullsized":"2024-04-25T12:36:26.460480+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460689+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:36:26.460689+0000","last_peered":"2024-04-25T12:36:26.460689+0000","last_clean":"2024-04-25T12:36:26.460689+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:36:26.460689+0000","last_undegraded":"2024-04-25T12:36:26.460689+0000","last_fullsized":"2024-04-25T12:36:26.460689+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916651+0000","last_change":"2024-04-25T12:34:18.005815+0000","last_active":"2024-04-25T12:36:25.916651+0000","last_peered":"2024-04-25T12:36:25.916651+0000","last_clean":"2024-04-25T12:36:25.916651+0000","last_became_active":"2024-04-25T12:34:18.005291+0000","last_became_peered":"2024-04-25T12:34:18.005291+0000","last_unstale":"2024-04-25T12:36:25.916651+0000","last_undegraded":"2024-04-25T12:36:25.916651+0000","last_fullsized":"2024-04-25T12:36:25.916651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:42.826205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914894+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:36:25.914894+0000","last_peered":"2024-04-25T12:36:25.914894+0000","last_clean":"2024-04-25T12:36:25.914894+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:36:25.914894+0000","last_undegraded":"2024-04-25T12:36:25.914894+0000","last_fullsized":"2024-04-25T12:36:25.914894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.899730+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:36:25.899730+0000","last_peered":"2024-04-25T12:36:25.899730+0000","last_clean":"2024-04-25T12:36:25.899730+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:36:25.899730+0000","last_undegraded":"2024-04-25T12:36:25.899730+0000","last_fullsized":"2024-04-25T12:36:25.899730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916909+0000","last_change":"2024-04-25T12:36:00.627223+0000","last_active":"2024-04-25T12:36:25.916909+0000","last_peered":"2024-04-25T12:36:25.916909+0000","last_clean":"2024-04-25T12:36:25.916909+0000","last_became_active":"2024-04-25T12:36:00.626675+0000","last_became_peered":"2024-04-25T12:36:00.626675+0000","last_unstale":"2024-04-25T12:36:25.916909+0000","last_undegraded":"2024-04-25T12:36:25.916909+0000","last_fullsized":"2024-04-25T12:36:25.916909+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:11.204775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914910+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:36:25.914910+0000","last_peered":"2024-04-25T12:36:25.914910+0000","last_clean":"2024-04-25T12:36:25.914910+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:36:25.914910+0000","last_undegraded":"2024-04-25T12:36:25.914910+0000","last_fullsized":"2024-04-25T12:36:25.914910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.112191+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:36:26.112191+0000","last_peered":"2024-04-25T12:36:26.112191+0000","last_clean":"2024-04-25T12:36:26.112191+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:36:26.112191+0000","last_undegraded":"2024-04-25T12:36:26.112191+0000","last_fullsized":"2024-04-25T12:36:26.112191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.915780+0000","last_change":"2024-04-25T12:36:00.623046+0000","last_active":"2024-04-25T12:36:25.915780+0000","last_peered":"2024-04-25T12:36:25.915780+0000","last_clean":"2024-04-25T12:36:25.915780+0000","last_became_active":"2024-04-25T12:36:00.622702+0000","last_became_peered":"2024-04-25T12:36:00.622702+0000","last_unstale":"2024-04-25T12:36:25.915780+0000","last_undegraded":"2024-04-25T12:36:25.915780+0000","last_fullsized":"2024-04-25T12:36:25.915780+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111173+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:36:26.111173+0000","last_peered":"2024-04-25T12:36:26.111173+0000","last_clean":"2024-04-25T12:36:26.111173+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:36:26.111173+0000","last_undegraded":"2024-04-25T12:36:26.111173+0000","last_fullsized":"2024-04-25T12:36:26.111173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.113115+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:36:26.113115+0000","last_peered":"2024-04-25T12:36:26.113115+0000","last_clean":"2024-04-25T12:36:26.113115+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:36:26.113115+0000","last_undegraded":"2024-04-25T12:36:26.113115+0000","last_fullsized":"2024-04-25T12:36:26.113115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914404+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:36:25.914404+0000","last_peered":"2024-04-25T12:36:25.914404+0000","last_clean":"2024-04-25T12:36:25.914404+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:36:25.914404+0000","last_undegraded":"2024-04-25T12:36:25.914404+0000","last_fullsized":"2024-04-25T12:36:25.914404+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907226+0000","last_change":"2024-04-25T12:34:13.935138+0000","last_active":"2024-04-25T12:36:25.907226+0000","last_peered":"2024-04-25T12:36:25.907226+0000","last_clean":"2024-04-25T12:36:25.907226+0000","last_became_active":"2024-04-25T12:34:13.934787+0000","last_became_peered":"2024-04-25T12:34:13.934787+0000","last_unstale":"2024-04-25T12:36:25.907226+0000","last_undegraded":"2024-04-25T12:36:25.907226+0000","last_fullsized":"2024-04-25T12:36:25.907226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919290+0000","last_change":"2024-04-25T12:36:00.626426+0000","last_active":"2024-04-25T12:36:25.919290+0000","last_peered":"2024-04-25T12:36:25.919290+0000","last_clean":"2024-04-25T12:36:25.919290+0000","last_became_active":"2024-04-25T12:36:00.625818+0000","last_became_peered":"2024-04-25T12:36:00.625818+0000","last_unstale":"2024-04-25T12:36:25.919290+0000","last_undegraded":"2024-04-25T12:36:25.919290+0000","last_fullsized":"2024-04-25T12:36:25.919290+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:06.117031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914311+0000","last_change":"2024-04-25T12:34:18.008922+0000","last_active":"2024-04-25T12:36:25.914311+0000","last_peered":"2024-04-25T12:36:25.914311+0000","last_clean":"2024-04-25T12:36:25.914311+0000","last_became_active":"2024-04-25T12:34:18.008484+0000","last_became_peered":"2024-04-25T12:34:18.008484+0000","last_unstale":"2024-04-25T12:36:25.914311+0000","last_undegraded":"2024-04-25T12:36:25.914311+0000","last_fullsized":"2024-04-25T12:36:25.914311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895255+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:36:25.895255+0000","last_peered":"2024-04-25T12:36:25.895255+0000","last_clean":"2024-04-25T12:36:25.895255+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:36:25.895255+0000","last_undegraded":"2024-04-25T12:36:25.895255+0000","last_fullsized":"2024-04-25T12:36:25.895255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895773+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:36:25.895773+0000","last_peered":"2024-04-25T12:36:25.895773+0000","last_clean":"2024-04-25T12:36:25.895773+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:36:25.895773+0000","last_undegraded":"2024-04-25T12:36:25.895773+0000","last_fullsized":"2024-04-25T12:36:25.895773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907567+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:36:25.907567+0000","last_peered":"2024-04-25T12:36:25.907567+0000","last_clean":"2024-04-25T12:36:25.907567+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:36:25.907567+0000","last_undegraded":"2024-04-25T12:36:25.907567+0000","last_fullsized":"2024-04-25T12:36:25.907567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111701+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:36:26.111701+0000","last_peered":"2024-04-25T12:36:26.111701+0000","last_clean":"2024-04-25T12:36:26.111701+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:36:26.111701+0000","last_undegraded":"2024-04-25T12:36:26.111701+0000","last_fullsized":"2024-04-25T12:36:26.111701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":24,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.110883+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:36:26.110883+0000","last_peered":"2024-04-25T12:36:26.110883+0000","last_clean":"2024-04-25T12:36:26.110883+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:36:26.110883+0000","last_undegraded":"2024-04-25T12:36:26.110883+0000","last_fullsized":"2024-04-25T12:36:26.110883+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.913752+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:36:25.913752+0000","last_peered":"2024-04-25T12:36:25.913752+0000","last_clean":"2024-04-25T12:36:25.913752+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:36:25.913752+0000","last_undegraded":"2024-04-25T12:36:25.913752+0000","last_fullsized":"2024-04-25T12:36:25.913752+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895970+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:36:25.895970+0000","last_peered":"2024-04-25T12:36:25.895970+0000","last_clean":"2024-04-25T12:36:25.895970+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:36:25.895970+0000","last_undegraded":"2024-04-25T12:36:25.895970+0000","last_fullsized":"2024-04-25T12:36:25.895970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.114252+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:36:26.114252+0000","last_peered":"2024-04-25T12:36:26.114252+0000","last_clean":"2024-04-25T12:36:26.114252+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:36:26.114252+0000","last_undegraded":"2024-04-25T12:36:26.114252+0000","last_fullsized":"2024-04-25T12:36:26.114252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916950+0000","last_change":"2024-04-25T12:34:18.007476+0000","last_active":"2024-04-25T12:36:25.916950+0000","last_peered":"2024-04-25T12:36:25.916950+0000","last_clean":"2024-04-25T12:36:25.916950+0000","last_became_active":"2024-04-25T12:34:18.007000+0000","last_became_peered":"2024-04-25T12:34:18.007000+0000","last_unstale":"2024-04-25T12:36:25.916950+0000","last_undegraded":"2024-04-25T12:36:25.916950+0000","last_fullsized":"2024-04-25T12:36:25.916950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895985+0000","last_change":"2024-04-25T12:34:13.933293+0000","last_active":"2024-04-25T12:36:25.895985+0000","last_peered":"2024-04-25T12:36:25.895985+0000","last_clean":"2024-04-25T12:36:25.895985+0000","last_became_active":"2024-04-25T12:34:13.932897+0000","last_became_peered":"2024-04-25T12:34:13.932897+0000","last_unstale":"2024-04-25T12:36:25.895985+0000","last_undegraded":"2024-04-25T12:36:25.895985+0000","last_fullsized":"2024-04-25T12:36:25.895985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:38.751795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"43'6","reported_seq":94,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.896087+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:36:25.896087+0000","last_peered":"2024-04-25T12:36:25.896087+0000","last_clean":"2024-04-25T12:36:25.896087+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:36:25.896087+0000","last_undegraded":"2024-04-25T12:36:25.896087+0000","last_fullsized":"2024-04-25T12:36:25.896087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895141+0000","last_change":"2024-04-25T12:34:18.012355+0000","last_active":"2024-04-25T12:36:25.895141+0000","last_peered":"2024-04-25T12:36:25.895141+0000","last_clean":"2024-04-25T12:36:25.895141+0000","last_became_active":"2024-04-25T12:34:18.011697+0000","last_became_peered":"2024-04-25T12:34:18.011697+0000","last_unstale":"2024-04-25T12:36:25.895141+0000","last_undegraded":"2024-04-25T12:36:25.895141+0000","last_fullsized":"2024-04-25T12:36:25.895141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919555+0000","last_change":"2024-04-25T12:36:00.626931+0000","last_active":"2024-04-25T12:36:25.919555+0000","last_peered":"2024-04-25T12:36:25.919555+0000","last_clean":"2024-04-25T12:36:25.919555+0000","last_became_active":"2024-04-25T12:36:00.626533+0000","last_became_peered":"2024-04-25T12:36:00.626533+0000","last_unstale":"2024-04-25T12:36:25.919555+0000","last_undegraded":"2024-04-25T12:36:25.919555+0000","last_fullsized":"2024-04-25T12:36:25.919555+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.112480+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:36:26.112480+0000","last_peered":"2024-04-25T12:36:26.112480+0000","last_clean":"2024-04-25T12:36:26.112480+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:36:26.112480+0000","last_undegraded":"2024-04-25T12:36:26.112480+0000","last_fullsized":"2024-04-25T12:36:26.112480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919152+0000","last_change":"2024-04-25T12:34:18.008221+0000","last_active":"2024-04-25T12:36:25.919152+0000","last_peered":"2024-04-25T12:36:25.919152+0000","last_clean":"2024-04-25T12:36:25.919152+0000","last_became_active":"2024-04-25T12:34:18.007756+0000","last_became_peered":"2024-04-25T12:34:18.007756+0000","last_unstale":"2024-04-25T12:36:25.919152+0000","last_undegraded":"2024-04-25T12:36:25.919152+0000","last_fullsized":"2024-04-25T12:36:25.919152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:54.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.917488+0000","last_change":"2024-04-25T12:34:13.939791+0000","last_active":"2024-04-25T12:36:25.917488+0000","last_peered":"2024-04-25T12:36:25.917488+0000","last_clean":"2024-04-25T12:36:25.917488+0000","last_became_active":"2024-04-25T12:34:13.939359+0000","last_became_peered":"2024-04-25T12:34:13.939359+0000","last_unstale":"2024-04-25T12:36:25.917488+0000","last_undegraded":"2024-04-25T12:36:25.917488+0000","last_fullsized":"2024-04-25T12:36:25.917488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460317+0000","last_change":"2024-04-25T12:36:00.622303+0000","last_active":"2024-04-25T12:36:26.460317+0000","last_peered":"2024-04-25T12:36:26.460317+0000","last_clean":"2024-04-25T12:36:26.460317+0000","last_became_active":"2024-04-25T12:36:00.621920+0000","last_became_peered":"2024-04-25T12:36:00.621920+0000","last_unstale":"2024-04-25T12:36:26.460317+0000","last_undegraded":"2024-04-25T12:36:26.460317+0000","last_fullsized":"2024-04-25T12:36:26.460317+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:25.840527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.918982+0000","last_change":"2024-04-25T12:36:00.631090+0000","last_active":"2024-04-25T12:36:25.918982+0000","last_peered":"2024-04-25T12:36:25.918982+0000","last_clean":"2024-04-25T12:36:25.918982+0000","last_became_active":"2024-04-25T12:36:00.630484+0000","last_became_peered":"2024-04-25T12:36:00.630484+0000","last_unstale":"2024-04-25T12:36:25.918982+0000","last_undegraded":"2024-04-25T12:36:25.918982+0000","last_fullsized":"2024-04-25T12:36:25.918982+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.738025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914086+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:36:25.914086+0000","last_peered":"2024-04-25T12:36:25.914086+0000","last_clean":"2024-04-25T12:36:25.914086+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:36:25.914086+0000","last_undegraded":"2024-04-25T12:36:25.914086+0000","last_fullsized":"2024-04-25T12:36:25.914086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111560+0000","last_change":"2024-04-25T12:34:12.910252+0000","last_active":"2024-04-25T12:36:26.111560+0000","last_peered":"2024-04-25T12:36:26.111560+0000","last_clean":"2024-04-25T12:36:26.111560+0000","last_became_active":"2024-04-25T12:34:12.909704+0000","last_became_peered":"2024-04-25T12:34:12.909704+0000","last_unstale":"2024-04-25T12:36:26.111560+0000","last_undegraded":"2024-04-25T12:36:26.111560+0000","last_fullsized":"2024-04-25T12:36:26.111560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:23.991033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919359+0000","last_change":"2024-04-25T12:34:18.012419+0000","last_active":"2024-04-25T12:36:25.919359+0000","last_peered":"2024-04-25T12:36:25.919359+0000","last_clean":"2024-04-25T12:36:25.919359+0000","last_became_active":"2024-04-25T12:34:18.011974+0000","last_became_peered":"2024-04-25T12:34:18.011974+0000","last_unstale":"2024-04-25T12:36:25.919359+0000","last_undegraded":"2024-04-25T12:36:25.919359+0000","last_fullsized":"2024-04-25T12:36:25.919359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:07.849855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.459987+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:36:26.459987+0000","last_peered":"2024-04-25T12:36:26.459987+0000","last_clean":"2024-04-25T12:36:26.459987+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:36:26.459987+0000","last_undegraded":"2024-04-25T12:36:26.459987+0000","last_fullsized":"2024-04-25T12:36:26.459987+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":99,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907174+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:36:25.907174+0000","last_peered":"2024-04-25T12:36:25.907174+0000","last_clean":"2024-04-25T12:36:25.907174+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:36:25.907174+0000","last_undegraded":"2024-04-25T12:36:25.907174+0000","last_fullsized":"2024-04-25T12:36:25.907174+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919230+0000","last_change":"2024-04-25T12:34:18.009372+0000","last_active":"2024-04-25T12:36:25.919230+0000","last_peered":"2024-04-25T12:36:25.919230+0000","last_clean":"2024-04-25T12:36:25.919230+0000","last_became_active":"2024-04-25T12:34:18.008905+0000","last_became_peered":"2024-04-25T12:34:18.008905+0000","last_unstale":"2024-04-25T12:36:25.919230+0000","last_undegraded":"2024-04-25T12:36:25.919230+0000","last_fullsized":"2024-04-25T12:36:25.919230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.918932+0000","last_change":"2024-04-25T12:34:13.942476+0000","last_active":"2024-04-25T12:36:25.918932+0000","last_peered":"2024-04-25T12:36:25.918932+0000","last_clean":"2024-04-25T12:36:25.918932+0000","last_became_active":"2024-04-25T12:34:13.941963+0000","last_became_peered":"2024-04-25T12:34:13.941963+0000","last_unstale":"2024-04-25T12:36:25.918932+0000","last_undegraded":"2024-04-25T12:36:25.918932+0000","last_fullsized":"2024-04-25T12:36:25.918932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111343+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:36:26.111343+0000","last_peered":"2024-04-25T12:36:26.111343+0000","last_clean":"2024-04-25T12:36:26.111343+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:36:26.111343+0000","last_undegraded":"2024-04-25T12:36:26.111343+0000","last_fullsized":"2024-04-25T12:36:26.111343+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460846+0000","last_change":"2024-04-25T12:36:00.633695+0000","last_active":"2024-04-25T12:36:26.460846+0000","last_peered":"2024-04-25T12:36:26.460846+0000","last_clean":"2024-04-25T12:36:26.460846+0000","last_became_active":"2024-04-25T12:36:00.633193+0000","last_became_peered":"2024-04-25T12:36:00.633193+0000","last_unstale":"2024-04-25T12:36:26.460846+0000","last_undegraded":"2024-04-25T12:36:26.460846+0000","last_fullsized":"2024-04-25T12:36:26.460846+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:10.743500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895365+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:36:25.895365+0000","last_peered":"2024-04-25T12:36:25.895365+0000","last_clean":"2024-04-25T12:36:25.895365+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:36:25.895365+0000","last_undegraded":"2024-04-25T12:36:25.895365+0000","last_fullsized":"2024-04-25T12:36:25.895365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.113168+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:36:26.113168+0000","last_peered":"2024-04-25T12:36:26.113168+0000","last_clean":"2024-04-25T12:36:26.113168+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:36:26.113168+0000","last_undegraded":"2024-04-25T12:36:26.113168+0000","last_fullsized":"2024-04-25T12:36:26.113168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":195,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.898891+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:36:25.898891+0000","last_peered":"2024-04-25T12:36:25.898891+0000","last_clean":"2024-04-25T12:36:25.898891+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:36:25.898891+0000","last_undegraded":"2024-04-25T12:36:25.898891+0000","last_fullsized":"2024-04-25T12:36:25.898891+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.110349+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:36:26.110349+0000","last_peered":"2024-04-25T12:36:26.110349+0000","last_clean":"2024-04-25T12:36:26.110349+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:36:26.110349+0000","last_undegraded":"2024-04-25T12:36:26.110349+0000","last_fullsized":"2024-04-25T12:36:26.110349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.112278+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:36:26.112278+0000","last_peered":"2024-04-25T12:36:26.112278+0000","last_clean":"2024-04-25T12:36:26.112278+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:36:26.112278+0000","last_undegraded":"2024-04-25T12:36:26.112278+0000","last_fullsized":"2024-04-25T12:36:26.112278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919561+0000","last_change":"2024-04-25T12:34:12.916994+0000","last_active":"2024-04-25T12:36:25.919561+0000","last_peered":"2024-04-25T12:36:25.919561+0000","last_clean":"2024-04-25T12:36:25.919561+0000","last_became_active":"2024-04-25T12:34:12.916653+0000","last_became_peered":"2024-04-25T12:34:12.916653+0000","last_unstale":"2024-04-25T12:36:25.919561+0000","last_undegraded":"2024-04-25T12:36:25.919561+0000","last_fullsized":"2024-04-25T12:36:25.919561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:37.046182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.915900+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:36:25.915900+0000","last_peered":"2024-04-25T12:36:25.915900+0000","last_clean":"2024-04-25T12:36:25.915900+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:36:25.915900+0000","last_undegraded":"2024-04-25T12:36:25.915900+0000","last_fullsized":"2024-04-25T12:36:25.915900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914745+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:36:25.914745+0000","last_peered":"2024-04-25T12:36:25.914745+0000","last_clean":"2024-04-25T12:36:25.914745+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:36:25.914745+0000","last_undegraded":"2024-04-25T12:36:25.914745+0000","last_fullsized":"2024-04-25T12:36:25.914745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":53,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460676+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:36:26.460676+0000","last_peered":"2024-04-25T12:36:26.460676+0000","last_clean":"2024-04-25T12:36:26.460676+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:36:26.460676+0000","last_undegraded":"2024-04-25T12:36:26.460676+0000","last_fullsized":"2024-04-25T12:36:26.460676+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460205+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:36:26.460205+0000","last_peered":"2024-04-25T12:36:26.460205+0000","last_clean":"2024-04-25T12:36:26.460205+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:36:26.460205+0000","last_undegraded":"2024-04-25T12:36:26.460205+0000","last_fullsized":"2024-04-25T12:36:26.460205+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916347+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:36:25.916347+0000","last_peered":"2024-04-25T12:36:25.916347+0000","last_clean":"2024-04-25T12:36:25.916347+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:36:25.916347+0000","last_undegraded":"2024-04-25T12:36:25.916347+0000","last_fullsized":"2024-04-25T12:36:25.916347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895993+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:36:25.895993+0000","last_peered":"2024-04-25T12:36:25.895993+0000","last_clean":"2024-04-25T12:36:25.895993+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:36:25.895993+0000","last_undegraded":"2024-04-25T12:36:25.895993+0000","last_fullsized":"2024-04-25T12:36:25.895993+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.109906+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:36:26.109906+0000","last_peered":"2024-04-25T12:36:26.109906+0000","last_clean":"2024-04-25T12:36:26.109906+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:36:26.109906+0000","last_undegraded":"2024-04-25T12:36:26.109906+0000","last_fullsized":"2024-04-25T12:36:26.109906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914042+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:36:25.914042+0000","last_peered":"2024-04-25T12:36:25.914042+0000","last_clean":"2024-04-25T12:36:25.914042+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:36:25.914042+0000","last_undegraded":"2024-04-25T12:36:25.914042+0000","last_fullsized":"2024-04-25T12:36:25.914042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916644+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:36:25.916644+0000","last_peered":"2024-04-25T12:36:25.916644+0000","last_clean":"2024-04-25T12:36:25.916644+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:36:25.916644+0000","last_undegraded":"2024-04-25T12:36:25.916644+0000","last_fullsized":"2024-04-25T12:36:25.916644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.898908+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:36:25.898908+0000","last_peered":"2024-04-25T12:36:25.898908+0000","last_clean":"2024-04-25T12:36:25.898908+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:36:25.898908+0000","last_undegraded":"2024-04-25T12:36:25.898908+0000","last_fullsized":"2024-04-25T12:36:25.898908+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111566+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:36:26.111566+0000","last_peered":"2024-04-25T12:36:26.111566+0000","last_clean":"2024-04-25T12:36:26.111566+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:36:26.111566+0000","last_undegraded":"2024-04-25T12:36:26.111566+0000","last_fullsized":"2024-04-25T12:36:26.111566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":102,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895313+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:36:25.895313+0000","last_peered":"2024-04-25T12:36:25.895313+0000","last_clean":"2024-04-25T12:36:25.895313+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:36:25.895313+0000","last_undegraded":"2024-04-25T12:36:25.895313+0000","last_fullsized":"2024-04-25T12:36:25.895313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460730+0000","last_change":"2024-04-25T12:36:00.633538+0000","last_active":"2024-04-25T12:36:26.460730+0000","last_peered":"2024-04-25T12:36:26.460730+0000","last_clean":"2024-04-25T12:36:26.460730+0000","last_became_active":"2024-04-25T12:36:00.632957+0000","last_became_peered":"2024-04-25T12:36:00.632957+0000","last_unstale":"2024-04-25T12:36:26.460730+0000","last_undegraded":"2024-04-25T12:36:26.460730+0000","last_fullsized":"2024-04-25T12:36:26.460730+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:41.919677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.898372+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:36:25.898372+0000","last_peered":"2024-04-25T12:36:25.898372+0000","last_clean":"2024-04-25T12:36:25.898372+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:36:25.898372+0000","last_undegraded":"2024-04-25T12:36:25.898372+0000","last_fullsized":"2024-04-25T12:36:25.898372+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.913167+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:36:25.913167+0000","last_peered":"2024-04-25T12:36:25.913167+0000","last_clean":"2024-04-25T12:36:25.913167+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:36:25.913167+0000","last_undegraded":"2024-04-25T12:36:25.913167+0000","last_fullsized":"2024-04-25T12:36:25.913167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460829+0000","last_change":"2024-04-25T12:36:00.631519+0000","last_active":"2024-04-25T12:36:26.460829+0000","last_peered":"2024-04-25T12:36:26.460829+0000","last_clean":"2024-04-25T12:36:26.460829+0000","last_became_active":"2024-04-25T12:36:00.631077+0000","last_became_peered":"2024-04-25T12:36:00.631077+0000","last_unstale":"2024-04-25T12:36:26.460829+0000","last_undegraded":"2024-04-25T12:36:26.460829+0000","last_fullsized":"2024-04-25T12:36:26.460829+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:45.009426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.112879+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:36:26.112879+0000","last_peered":"2024-04-25T12:36:26.112879+0000","last_clean":"2024-04-25T12:36:26.112879+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:36:26.112879+0000","last_undegraded":"2024-04-25T12:36:26.112879+0000","last_fullsized":"2024-04-25T12:36:26.112879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111431+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:36:26.111431+0000","last_peered":"2024-04-25T12:36:26.111431+0000","last_clean":"2024-04-25T12:36:26.111431+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:36:26.111431+0000","last_undegraded":"2024-04-25T12:36:26.111431+0000","last_fullsized":"2024-04-25T12:36:26.111431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919157+0000","last_change":"2024-04-25T12:34:18.002892+0000","last_active":"2024-04-25T12:36:25.919157+0000","last_peered":"2024-04-25T12:36:25.919157+0000","last_clean":"2024-04-25T12:36:25.919157+0000","last_became_active":"2024-04-25T12:34:18.002414+0000","last_became_peered":"2024-04-25T12:34:18.002414+0000","last_unstale":"2024-04-25T12:36:25.919157+0000","last_undegraded":"2024-04-25T12:36:25.919157+0000","last_fullsized":"2024-04-25T12:36:25.919157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919083+0000","last_change":"2024-04-25T12:34:13.961116+0000","last_active":"2024-04-25T12:36:25.919083+0000","last_peered":"2024-04-25T12:36:25.919083+0000","last_clean":"2024-04-25T12:36:25.919083+0000","last_became_active":"2024-04-25T12:34:13.960867+0000","last_became_peered":"2024-04-25T12:34:13.960867+0000","last_unstale":"2024-04-25T12:36:25.919083+0000","last_undegraded":"2024-04-25T12:36:25.919083+0000","last_fullsized":"2024-04-25T12:36:25.919083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907380+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:36:25.907380+0000","last_peered":"2024-04-25T12:36:25.907380+0000","last_clean":"2024-04-25T12:36:25.907380+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:36:25.907380+0000","last_undegraded":"2024-04-25T12:36:25.907380+0000","last_fullsized":"2024-04-25T12:36:25.907380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.112095+0000","last_change":"2024-04-25T12:36:00.636248+0000","last_active":"2024-04-25T12:36:26.112095+0000","last_peered":"2024-04-25T12:36:26.112095+0000","last_clean":"2024-04-25T12:36:26.112095+0000","last_became_active":"2024-04-25T12:36:00.635874+0000","last_became_peered":"2024-04-25T12:36:00.635874+0000","last_unstale":"2024-04-25T12:36:26.112095+0000","last_undegraded":"2024-04-25T12:36:26.112095+0000","last_fullsized":"2024-04-25T12:36:26.112095+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.244233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":72,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907368+0000","last_change":"2024-04-25T12:35:00.239744+0000","last_active":"2024-04-25T12:36:25.907368+0000","last_peered":"2024-04-25T12:36:25.907368+0000","last_clean":"2024-04-25T12:36:25.907368+0000","last_became_active":"2024-04-25T12:35:00.239414+0000","last_became_peered":"2024-04-25T12:35:00.239414+0000","last_unstale":"2024-04-25T12:36:25.907368+0000","last_undegraded":"2024-04-25T12:36:25.907368+0000","last_fullsized":"2024-04-25T12:36:25.907368+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:17.015606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"42'17","reported_seq":103,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.918098+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:36:25.918098+0000","last_peered":"2024-04-25T12:36:25.918098+0000","last_clean":"2024-04-25T12:36:25.918098+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:36:25.918098+0000","last_undegraded":"2024-04-25T12:36:25.918098+0000","last_fullsized":"2024-04-25T12:36:25.918098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7907,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895255+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:36:25.895255+0000","last_peered":"2024-04-25T12:36:25.895255+0000","last_clean":"2024-04-25T12:36:25.895255+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:36:25.895255+0000","last_undegraded":"2024-04-25T12:36:25.895255+0000","last_fullsized":"2024-04-25T12:36:25.895255+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111923+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:36:26.111923+0000","last_peered":"2024-04-25T12:36:26.111923+0000","last_clean":"2024-04-25T12:36:26.111923+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:36:26.111923+0000","last_undegraded":"2024-04-25T12:36:26.111923+0000","last_fullsized":"2024-04-25T12:36:26.111923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907560+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:36:25.907560+0000","last_peered":"2024-04-25T12:36:25.907560+0000","last_clean":"2024-04-25T12:36:25.907560+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:36:25.907560+0000","last_undegraded":"2024-04-25T12:36:25.907560+0000","last_fullsized":"2024-04-25T12:36:25.907560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.917020+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:36:25.917020+0000","last_peered":"2024-04-25T12:36:25.917020+0000","last_clean":"2024-04-25T12:36:25.917020+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:36:25.917020+0000","last_undegraded":"2024-04-25T12:36:25.917020+0000","last_fullsized":"2024-04-25T12:36:25.917020+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460552+0000","last_change":"2024-04-25T12:36:00.636634+0000","last_active":"2024-04-25T12:36:26.460552+0000","last_peered":"2024-04-25T12:36:26.460552+0000","last_clean":"2024-04-25T12:36:26.460552+0000","last_became_active":"2024-04-25T12:36:00.636371+0000","last_became_peered":"2024-04-25T12:36:00.636371+0000","last_unstale":"2024-04-25T12:36:26.460552+0000","last_undegraded":"2024-04-25T12:36:26.460552+0000","last_fullsized":"2024-04-25T12:36:26.460552+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:36.185819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.915631+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:36:25.915631+0000","last_peered":"2024-04-25T12:36:25.915631+0000","last_clean":"2024-04-25T12:36:25.915631+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:36:25.915631+0000","last_undegraded":"2024-04-25T12:36:25.915631+0000","last_fullsized":"2024-04-25T12:36:25.915631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.906938+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:36:25.906938+0000","last_peered":"2024-04-25T12:36:25.906938+0000","last_clean":"2024-04-25T12:36:25.906938+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:36:25.906938+0000","last_undegraded":"2024-04-25T12:36:25.906938+0000","last_fullsized":"2024-04-25T12:36:25.906938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.913945+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:36:25.913945+0000","last_peered":"2024-04-25T12:36:25.913945+0000","last_clean":"2024-04-25T12:36:25.913945+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:36:25.913945+0000","last_undegraded":"2024-04-25T12:36:25.913945+0000","last_fullsized":"2024-04-25T12:36:25.913945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.899908+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:36:25.899908+0000","last_peered":"2024-04-25T12:36:25.899908+0000","last_clean":"2024-04-25T12:36:25.899908+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:36:25.899908+0000","last_undegraded":"2024-04-25T12:36:25.899908+0000","last_fullsized":"2024-04-25T12:36:25.899908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895225+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:36:25.895225+0000","last_peered":"2024-04-25T12:36:25.895225+0000","last_clean":"2024-04-25T12:36:25.895225+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:36:25.895225+0000","last_undegraded":"2024-04-25T12:36:25.895225+0000","last_fullsized":"2024-04-25T12:36:25.895225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895775+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:36:25.895775+0000","last_peered":"2024-04-25T12:36:25.895775+0000","last_clean":"2024-04-25T12:36:25.895775+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:36:25.895775+0000","last_undegraded":"2024-04-25T12:36:25.895775+0000","last_fullsized":"2024-04-25T12:36:25.895775+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895321+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:36:25.895321+0000","last_peered":"2024-04-25T12:36:25.895321+0000","last_clean":"2024-04-25T12:36:25.895321+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:36:25.895321+0000","last_undegraded":"2024-04-25T12:36:25.895321+0000","last_fullsized":"2024-04-25T12:36:25.895321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111240+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:36:26.111240+0000","last_peered":"2024-04-25T12:36:26.111240+0000","last_clean":"2024-04-25T12:36:26.111240+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:36:26.111240+0000","last_undegraded":"2024-04-25T12:36:26.111240+0000","last_fullsized":"2024-04-25T12:36:26.111240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.113949+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:36:26.113949+0000","last_peered":"2024-04-25T12:36:26.113949+0000","last_clean":"2024-04-25T12:36:26.113949+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:36:26.113949+0000","last_undegraded":"2024-04-25T12:36:26.113949+0000","last_fullsized":"2024-04-25T12:36:26.113949+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460078+0000","last_change":"2024-04-25T12:36:00.623718+0000","last_active":"2024-04-25T12:36:26.460078+0000","last_peered":"2024-04-25T12:36:26.460078+0000","last_clean":"2024-04-25T12:36:26.460078+0000","last_became_active":"2024-04-25T12:36:00.622664+0000","last_became_peered":"2024-04-25T12:36:00.622664+0000","last_unstale":"2024-04-25T12:36:26.460078+0000","last_undegraded":"2024-04-25T12:36:26.460078+0000","last_fullsized":"2024-04-25T12:36:26.460078+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.061061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.918973+0000","last_change":"2024-04-25T12:34:12.923135+0000","last_active":"2024-04-25T12:36:25.918973+0000","last_peered":"2024-04-25T12:36:25.918973+0000","last_clean":"2024-04-25T12:36:25.918973+0000","last_became_active":"2024-04-25T12:34:12.922750+0000","last_became_peered":"2024-04-25T12:34:12.922750+0000","last_unstale":"2024-04-25T12:36:25.918973+0000","last_undegraded":"2024-04-25T12:36:25.918973+0000","last_fullsized":"2024-04-25T12:36:25.918973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111306+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:36:26.111306+0000","last_peered":"2024-04-25T12:36:26.111306+0000","last_clean":"2024-04-25T12:36:26.111306+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:36:26.111306+0000","last_undegraded":"2024-04-25T12:36:26.111306+0000","last_fullsized":"2024-04-25T12:36:26.111306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895569+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:36:25.895569+0000","last_peered":"2024-04-25T12:36:25.895569+0000","last_clean":"2024-04-25T12:36:25.895569+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:36:25.895569+0000","last_undegraded":"2024-04-25T12:36:25.895569+0000","last_fullsized":"2024-04-25T12:36:25.895569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895512+0000","last_change":"2024-04-25T12:36:00.624696+0000","last_active":"2024-04-25T12:36:25.895512+0000","last_peered":"2024-04-25T12:36:25.895512+0000","last_clean":"2024-04-25T12:36:25.895512+0000","last_became_active":"2024-04-25T12:36:00.624209+0000","last_became_peered":"2024-04-25T12:36:00.624209+0000","last_unstale":"2024-04-25T12:36:25.895512+0000","last_undegraded":"2024-04-25T12:36:25.895512+0000","last_fullsized":"2024-04-25T12:36:25.895512+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:14.608733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":59,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460143+0000","last_change":"2024-04-25T12:36:00.623752+0000","last_active":"2024-04-25T12:36:26.460143+0000","last_peered":"2024-04-25T12:36:26.460143+0000","last_clean":"2024-04-25T12:36:26.460143+0000","last_became_active":"2024-04-25T12:36:00.623008+0000","last_became_peered":"2024-04-25T12:36:00.623008+0000","last_unstale":"2024-04-25T12:36:26.460143+0000","last_undegraded":"2024-04-25T12:36:26.460143+0000","last_fullsized":"2024-04-25T12:36:26.460143+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:12.341618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.900020+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:36:25.900020+0000","last_peered":"2024-04-25T12:36:25.900020+0000","last_clean":"2024-04-25T12:36:25.900020+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:36:25.900020+0000","last_undegraded":"2024-04-25T12:36:25.900020+0000","last_fullsized":"2024-04-25T12:36:25.900020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907412+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:36:25.907412+0000","last_peered":"2024-04-25T12:36:25.907412+0000","last_clean":"2024-04-25T12:36:25.907412+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:36:25.907412+0000","last_undegraded":"2024-04-25T12:36:25.907412+0000","last_fullsized":"2024-04-25T12:36:25.907412+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916761+0000","last_change":"2024-04-25T12:34:13.940073+0000","last_active":"2024-04-25T12:36:25.916761+0000","last_peered":"2024-04-25T12:36:25.916761+0000","last_clean":"2024-04-25T12:36:25.916761+0000","last_became_active":"2024-04-25T12:34:13.939352+0000","last_became_peered":"2024-04-25T12:34:13.939352+0000","last_unstale":"2024-04-25T12:36:25.916761+0000","last_undegraded":"2024-04-25T12:36:25.916761+0000","last_fullsized":"2024-04-25T12:36:25.916761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907084+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:36:25.907084+0000","last_peered":"2024-04-25T12:36:25.907084+0000","last_clean":"2024-04-25T12:36:25.907084+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:36:25.907084+0000","last_undegraded":"2024-04-25T12:36:25.907084+0000","last_fullsized":"2024-04-25T12:36:25.907084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111012+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:36:26.111012+0000","last_peered":"2024-04-25T12:36:26.111012+0000","last_clean":"2024-04-25T12:36:26.111012+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:36:26.111012+0000","last_undegraded":"2024-04-25T12:36:26.111012+0000","last_fullsized":"2024-04-25T12:36:26.111012+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460350+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:36:26.460350+0000","last_peered":"2024-04-25T12:36:26.460350+0000","last_clean":"2024-04-25T12:36:26.460350+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:36:26.460350+0000","last_undegraded":"2024-04-25T12:36:26.460350+0000","last_fullsized":"2024-04-25T12:36:26.460350+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919492+0000","last_change":"2024-04-25T12:34:13.967630+0000","last_active":"2024-04-25T12:36:25.919492+0000","last_peered":"2024-04-25T12:36:25.919492+0000","last_clean":"2024-04-25T12:36:25.919492+0000","last_became_active":"2024-04-25T12:34:13.967339+0000","last_became_peered":"2024-04-25T12:34:13.967339+0000","last_unstale":"2024-04-25T12:36:25.919492+0000","last_undegraded":"2024-04-25T12:36:25.919492+0000","last_fullsized":"2024-04-25T12:36:25.919492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:46.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919498+0000","last_change":"2024-04-25T12:34:13.935435+0000","last_active":"2024-04-25T12:36:25.919498+0000","last_peered":"2024-04-25T12:36:25.919498+0000","last_clean":"2024-04-25T12:36:25.919498+0000","last_became_active":"2024-04-25T12:34:13.935081+0000","last_became_peered":"2024-04-25T12:34:13.935081+0000","last_unstale":"2024-04-25T12:36:25.919498+0000","last_undegraded":"2024-04-25T12:36:25.919498+0000","last_fullsized":"2024-04-25T12:36:25.919498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.913899+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:36:25.913899+0000","last_peered":"2024-04-25T12:36:25.913899+0000","last_clean":"2024-04-25T12:36:25.913899+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:36:25.913899+0000","last_undegraded":"2024-04-25T12:36:25.913899+0000","last_fullsized":"2024-04-25T12:36:25.913899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460685+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:36:26.460685+0000","last_peered":"2024-04-25T12:36:26.460685+0000","last_clean":"2024-04-25T12:36:26.460685+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:36:26.460685+0000","last_undegraded":"2024-04-25T12:36:26.460685+0000","last_fullsized":"2024-04-25T12:36:26.460685+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111190+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:36:26.111190+0000","last_peered":"2024-04-25T12:36:26.111190+0000","last_clean":"2024-04-25T12:36:26.111190+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:36:26.111190+0000","last_undegraded":"2024-04-25T12:36:26.111190+0000","last_fullsized":"2024-04-25T12:36:26.111190+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.459984+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:36:26.459984+0000","last_peered":"2024-04-25T12:36:26.459984+0000","last_clean":"2024-04-25T12:36:26.459984+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:36:26.459984+0000","last_undegraded":"2024-04-25T12:36:26.459984+0000","last_fullsized":"2024-04-25T12:36:26.459984+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.899908+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:36:25.899908+0000","last_peered":"2024-04-25T12:36:25.899908+0000","last_clean":"2024-04-25T12:36:25.899908+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:36:25.899908+0000","last_undegraded":"2024-04-25T12:36:25.899908+0000","last_fullsized":"2024-04-25T12:36:25.899908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.895396+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:36:25.895396+0000","last_peered":"2024-04-25T12:36:25.895396+0000","last_clean":"2024-04-25T12:36:25.895396+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:36:25.895396+0000","last_undegraded":"2024-04-25T12:36:25.895396+0000","last_fullsized":"2024-04-25T12:36:25.895396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914648+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:36:25.914648+0000","last_peered":"2024-04-25T12:36:25.914648+0000","last_clean":"2024-04-25T12:36:25.914648+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:36:25.914648+0000","last_undegraded":"2024-04-25T12:36:25.914648+0000","last_fullsized":"2024-04-25T12:36:25.914648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.112379+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:36:26.112379+0000","last_peered":"2024-04-25T12:36:26.112379+0000","last_clean":"2024-04-25T12:36:26.112379+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:36:26.112379+0000","last_undegraded":"2024-04-25T12:36:26.112379+0000","last_fullsized":"2024-04-25T12:36:26.112379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.907647+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:36:25.907647+0000","last_peered":"2024-04-25T12:36:25.907647+0000","last_clean":"2024-04-25T12:36:25.907647+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:36:25.907647+0000","last_undegraded":"2024-04-25T12:36:25.907647+0000","last_fullsized":"2024-04-25T12:36:25.907647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.114160+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:36:26.114160+0000","last_peered":"2024-04-25T12:36:26.114160+0000","last_clean":"2024-04-25T12:36:26.114160+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:36:26.114160+0000","last_undegraded":"2024-04-25T12:36:26.114160+0000","last_fullsized":"2024-04-25T12:36:26.114160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.111052+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:36:26.111052+0000","last_peered":"2024-04-25T12:36:26.111052+0000","last_clean":"2024-04-25T12:36:26.111052+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:36:26.111052+0000","last_undegraded":"2024-04-25T12:36:26.111052+0000","last_fullsized":"2024-04-25T12:36:26.111052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460071+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:36:26.460071+0000","last_peered":"2024-04-25T12:36:26.460071+0000","last_clean":"2024-04-25T12:36:26.460071+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:36:26.460071+0000","last_undegraded":"2024-04-25T12:36:26.460071+0000","last_fullsized":"2024-04-25T12:36:26.460071+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.918992+0000","last_change":"2024-04-25T12:34:13.935924+0000","last_active":"2024-04-25T12:36:25.918992+0000","last_peered":"2024-04-25T12:36:25.918992+0000","last_clean":"2024-04-25T12:36:25.918992+0000","last_became_active":"2024-04-25T12:34:13.935139+0000","last_became_peered":"2024-04-25T12:34:13.935139+0000","last_unstale":"2024-04-25T12:36:25.918992+0000","last_undegraded":"2024-04-25T12:36:25.918992+0000","last_fullsized":"2024-04-25T12:36:25.918992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.919290+0000","last_change":"2024-04-25T12:34:13.976605+0000","last_active":"2024-04-25T12:36:25.919290+0000","last_peered":"2024-04-25T12:36:25.919290+0000","last_clean":"2024-04-25T12:36:25.919290+0000","last_became_active":"2024-04-25T12:34:13.976350+0000","last_became_peered":"2024-04-25T12:34:13.976350+0000","last_unstale":"2024-04-25T12:36:25.919290+0000","last_undegraded":"2024-04-25T12:36:25.919290+0000","last_fullsized":"2024-04-25T12:36:25.919290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914551+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:36:25.914551+0000","last_peered":"2024-04-25T12:36:25.914551+0000","last_clean":"2024-04-25T12:36:25.914551+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:36:25.914551+0000","last_undegraded":"2024-04-25T12:36:25.914551+0000","last_fullsized":"2024-04-25T12:36:25.914551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914721+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:36:25.914721+0000","last_peered":"2024-04-25T12:36:25.914721+0000","last_clean":"2024-04-25T12:36:25.914721+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:36:25.914721+0000","last_undegraded":"2024-04-25T12:36:25.914721+0000","last_fullsized":"2024-04-25T12:36:25.914721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.915463+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:36:25.915463+0000","last_peered":"2024-04-25T12:36:25.915463+0000","last_clean":"2024-04-25T12:36:25.915463+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:36:25.915463+0000","last_undegraded":"2024-04-25T12:36:25.915463+0000","last_fullsized":"2024-04-25T12:36:25.915463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.899808+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:36:25.899808+0000","last_peered":"2024-04-25T12:36:25.899808+0000","last_clean":"2024-04-25T12:36:25.899808+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:36:25.899808+0000","last_undegraded":"2024-04-25T12:36:25.899808+0000","last_fullsized":"2024-04-25T12:36:25.899808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460250+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:36:26.460250+0000","last_peered":"2024-04-25T12:36:26.460250+0000","last_clean":"2024-04-25T12:36:26.460250+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:36:26.460250+0000","last_undegraded":"2024-04-25T12:36:26.460250+0000","last_fullsized":"2024-04-25T12:36:26.460250+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.912822+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:36:25.912822+0000","last_peered":"2024-04-25T12:36:25.912822+0000","last_clean":"2024-04-25T12:36:25.912822+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:36:25.912822+0000","last_undegraded":"2024-04-25T12:36:25.912822+0000","last_fullsized":"2024-04-25T12:36:25.912822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916058+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:36:25.916058+0000","last_peered":"2024-04-25T12:36:25.916058+0000","last_clean":"2024-04-25T12:36:25.916058+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:36:25.916058+0000","last_undegraded":"2024-04-25T12:36:25.916058+0000","last_fullsized":"2024-04-25T12:36:25.916058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.914189+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:36:25.914189+0000","last_peered":"2024-04-25T12:36:25.914189+0000","last_clean":"2024-04-25T12:36:25.914189+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:36:25.914189+0000","last_undegraded":"2024-04-25T12:36:25.914189+0000","last_fullsized":"2024-04-25T12:36:25.914189+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.917998+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:36:25.917998+0000","last_peered":"2024-04-25T12:36:25.917998+0000","last_clean":"2024-04-25T12:36:25.917998+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:36:25.917998+0000","last_undegraded":"2024-04-25T12:36:25.917998+0000","last_fullsized":"2024-04-25T12:36:25.917998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460504+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:36:26.460504+0000","last_peered":"2024-04-25T12:36:26.460504+0000","last_clean":"2024-04-25T12:36:26.460504+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:36:26.460504+0000","last_undegraded":"2024-04-25T12:36:26.460504+0000","last_fullsized":"2024-04-25T12:36:26.460504+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":40,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:26.460605+0000","last_change":"2024-04-25T12:36:00.630044+0000","last_active":"2024-04-25T12:36:26.460605+0000","last_peered":"2024-04-25T12:36:26.460605+0000","last_clean":"2024-04-25T12:36:26.460605+0000","last_became_active":"2024-04-25T12:36:00.629698+0000","last_became_peered":"2024-04-25T12:36:00.629698+0000","last_unstale":"2024-04-25T12:36:26.460605+0000","last_undegraded":"2024-04-25T12:36:26.460605+0000","last_fullsized":"2024-04-25T12:36:26.460605+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:20.463389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T12:36:25.916251+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:36:25.916251+0000","last_peered":"2024-04-25T12:36:25.916251+0000","last_clean":"2024-04-25T12:36:25.916251+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:36:25.916251+0000","last_undegraded":"2024-04-25T12:36:25.916251+0000","last_fullsized":"2024-04-25T12:36:25.916251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11517,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":23034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":332781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":332871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":332463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542175,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":332759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462919,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28904,"kb_used_data":668,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342936,"statfs":{"total":96636764160,"available":96607166464,"internally_reserved":0,"allocated":684032,"data_stored":332359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":931398,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":339496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":737280,"data_stored":922083,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:36:29.430 INFO:tasks.thrashosds.thrasher:Setting 4.15 to [1, 0, 7, 3] 2024-04-25T12:36:29.431 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.15', '1', '0', '7', '3'] 2024-04-25T12:36:29.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.15 1 0 7 3 2024-04-25T12:36:29.472 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:29.473+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:29.497 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:29.497 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:29.506 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:29.590 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:29.591+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:29.659 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:29.659 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:29.668 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:29.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.704+0000 7fdfa3e00640 1 Processor -- start 2024-04-25T12:36:29.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.704+0000 7fdfa3e00640 1 -- start start 2024-04-25T12:36:29.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf9c070250 0x7fdf9c1578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa3e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdf9c06f7c0 0x7fdf9c15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa3e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c074600 con 0x7fdf9c0c4cc0 2024-04-25T12:36:29.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa3e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c0c7270 con 0x7fdf9c070250 2024-04-25T12:36:29.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa3e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c0c9fb0 con 0x7fdf9c06f7c0 2024-04-25T12:36:29.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf9c070250 0x7fdf9c1578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf9c070250 0x7fdf9c1578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60946/0 (socket says 172.21.15.123:60946) 2024-04-25T12:36:29.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa1400640 1 -- 172.21.15.123:0/7825575 learned_addr learned my addr 172.21.15.123:0/7825575 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:29.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.706+0000 7fdfa1e00640 1 --2- 172.21.15.123:0/7825575 >> v2:172.21.15.153:3300/0 conn(0x7fdf9c06f7c0 0x7fdf9c15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.707+0000 7fdfa1400640 1 -- 172.21.15.123:0/7825575 >> v2:172.21.15.153:3300/0 conn(0x7fdf9c06f7c0 msgr2=0x7fdf9c15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:29.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.707+0000 7fdfa1400640 1 --2- 172.21.15.123:0/7825575 >> v2:172.21.15.153:3300/0 conn(0x7fdf9c06f7c0 0x7fdf9c15dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.707+0000 7fdf9be00640 1 -- 172.21.15.123:0/7825575 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292533934 0 0) 0x7fdf9c074600 con 0x7fdf9c0c4cc0 2024-04-25T12:36:29.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.707+0000 7fdfa1400640 1 -- 172.21.15.123:0/7825575 >> v1:172.21.15.123:6789/0 conn(0x7fdf9c0c4cc0 legacy=0x7fdf9c0c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.707+0000 7fdfa1400640 1 -- 172.21.15.123:0/7825575 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c157e00 con 0x7fdf9c070250 2024-04-25T12:36:29.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.707+0000 7fdfa1e00640 1 --2- 172.21.15.123:0/7825575 >> v2:172.21.15.153:3300/0 conn(0x7fdf9c06f7c0 0x7fdf9c15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:29.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.709+0000 7fdfa1400640 1 --2- 172.21.15.123:0/7825575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c070250 0x7fdf9c1578c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0c65b0 tx=0x7fdf9002a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69df11e1c0ddbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:29.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.709+0000 7fdf9be00640 1 -- 172.21.15.123:0/7825575 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90037070 con 0x7fdf9c070250 2024-04-25T12:36:29.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.709+0000 7fdf9be00640 1 -- 172.21.15.123:0/7825575 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf90032640 con 0x7fdf9c070250 2024-04-25T12:36:29.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.709+0000 7fdf9be00640 1 -- 172.21.15.123:0/7825575 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9002ae60 con 0x7fdf9c070250 2024-04-25T12:36:29.710 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:29.710+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:29.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.710+0000 7fdfa3e00640 1 -- 172.21.15.123:0/7825575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c070250 msgr2=0x7fdf9c1578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.710+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/7825575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c070250 0x7fdf9c1578c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0c65b0 tx=0x7fdf9002a310 comp rx=0 tx=0).stop 2024-04-25T12:36:29.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.710+0000 7fdfa3e00640 1 -- 172.21.15.123:0/7825575 shutdown_connections 2024-04-25T12:36:29.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.710+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/7825575 >> v2:172.21.15.153:3300/0 conn(0x7fdf9c06f7c0 0x7fdf9c15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.710+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/7825575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c070250 0x7fdf9c1578c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.710+0000 7fdfa3e00640 1 -- 172.21.15.123:0/7825575 >> 172.21.15.123:0/7825575 conn(0x7fdf9c06c570 msgr2=0x7fdf9c06c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:29.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.711+0000 7fdfa3e00640 1 -- 172.21.15.123:0/7825575 shutdown_connections 2024-04-25T12:36:29.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.711+0000 7fdfa3e00640 1 -- 172.21.15.123:0/7825575 wait complete. 2024-04-25T12:36:29.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.713+0000 7fdfa3e00640 1 Processor -- start 2024-04-25T12:36:29.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.713+0000 7fdfa3e00640 1 -- start start 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c06f7c0 0x7fdf9c158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf9c070250 0x7fdf9c0c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf9c0c4cc0 0x7fdf9c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c0c9960 con 0x7fdf9c06f7c0 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c074b50 con 0x7fdf9c070250 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa3e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c0c9c40 con 0x7fdf9c0c4cc0 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa1e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf9c0c4cc0 0x7fdf9c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa0a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf9c070250 0x7fdf9c0c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa1e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf9c0c4cc0 0x7fdf9c0cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:34614/0 (socket says 172.21.15.123:34614) 2024-04-25T12:36:29.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.715+0000 7fdfa1e00640 1 -- 172.21.15.123:0/1305725364 learned_addr learned my addr 172.21.15.123:0/1305725364 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:29.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa0a00640 1 -- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf9c0c4cc0 msgr2=0x7fdf9c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa0a00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf9c0c4cc0 0x7fdf9c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa0a00640 1 -- 172.21.15.123:0/1305725364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c06f7c0 msgr2=0x7fdf9c158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:29.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa0a00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c06f7c0 0x7fdf9c158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa0a00640 1 -- 172.21.15.123:0/1305725364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c149390 con 0x7fdf9c070250 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa0a00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf9c070250 0x7fdf9c0c3eb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c3e0 tx=0x7fdf8800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdf99600640 1 -- 172.21.15.123:0/1305725364 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88017070 con 0x7fdf9c070250 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdf99600640 1 -- 172.21.15.123:0/1305725364 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf880035f0 con 0x7fdf9c070250 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdf99600640 1 -- 172.21.15.123:0/1305725364 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880039e0 con 0x7fdf9c070250 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c0c43f0 con 0x7fdf9c070250 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.716+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c1498e0 con 0x7fdf9c070250 2024-04-25T12:36:29.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.718+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c0ca3e0 con 0x7fdf9c070250 2024-04-25T12:36:29.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.720+0000 7fdf99600640 1 -- 172.21.15.123:0/1305725364 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf8801a070 con 0x7fdf9c070250 2024-04-25T12:36:29.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.721+0000 7fdf99600640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf7c0744b0 0x7fdf7c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:29.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.721+0000 7fdf99600640 1 -- 172.21.15.123:0/1305725364 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6648+0+0 (secure 0 0 0) 0x7fdf880948a0 con 0x7fdf9c070250 2024-04-25T12:36:29.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.721+0000 7fdfa1400640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf7c0744b0 0x7fdf7c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:29.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.722+0000 7fdfa1400640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf7c0744b0 0x7fdf7c0769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf90034000 tx=0x7fdf9002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:29.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.726+0000 7fdf99600640 1 -- 172.21.15.123:0/1305725364 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf9c0ca3e0 con 0x7fdf9c070250 2024-04-25T12:36:29.776 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:29.776+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:29.821 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:29.821 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:29.830 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:29.877 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:29.877+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:29.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:29.937+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "4.15", "id": [1, 0, 7, 3]} v 0) -- 0x7fdf9c076700 con 0x7fdf9c070250 2024-04-25T12:36:29.942 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:29.943 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:29.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:29.977 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:29.978+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:30.078 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:30.078+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:30.179 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:30.179+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:30.193 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:30.193 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:30.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:30.301 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:30.301+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:30.380 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:30.381+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:30.481 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:30.481+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:30.503 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:30.503 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:30.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:30.582 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:30.582+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:30.682 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:30.683+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:30.783 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:30.783+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:30.810 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:30.810 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:30.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:30.884 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:30.885+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:30.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.913+0000 7fdf99600640 1 -- 172.21.15.123:0/1305725364 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "4.15", "id": [1, 0, 7, 3]}]=0 set 4.15 pg_upmap mapping to [1,0,7,3] v60) ==== 134+0+0 (secure 0 0 0) 0x7fdf8805e590 con 0x7fdf9c070250 2024-04-25T12:36:30.916 INFO:teuthology.orchestra.run.smithi123.stderr:set 4.15 pg_upmap mapping to [1,0,7,3] 2024-04-25T12:36:30.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf7c0744b0 msgr2=0x7fdf7c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:30.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf7c0744b0 0x7fdf7c0769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf90034000 tx=0x7fdf9002c000 comp rx=0 tx=0).stop 2024-04-25T12:36:30.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf9c070250 msgr2=0x7fdf9c0c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:30.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf9c070250 0x7fdf9c0c3eb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c3e0 tx=0x7fdf8800c8a0 comp rx=0 tx=0).stop 2024-04-25T12:36:30.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 shutdown_connections 2024-04-25T12:36:30.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf7c0744b0 0x7fdf7c0769c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:30.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf9c0c4cc0 0x7fdf9c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:30.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf9c070250 0x7fdf9c0c3eb0 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c3e0 tx=0x7fdf8800c8a0 comp rx=0 tx=0).stop 2024-04-25T12:36:30.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 --2- 172.21.15.123:0/1305725364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf9c06f7c0 0x7fdf9c158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:30.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.917+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 >> 172.21.15.123:0/1305725364 conn(0x7fdf9c06c570 msgr2=0x7fdf9c15a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:30.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.918+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 shutdown_connections 2024-04-25T12:36:30.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:30.918+0000 7fdfa3e00640 1 -- 172.21.15.123:0/1305725364 wait complete. 2024-04-25T12:36:30.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:36:31.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:31.004 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:31.004+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:31.085 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:31.086+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:31.186 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:31.187+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:31.281 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:31.281 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:31.287 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:31.288+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:31.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:31.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.352+0000 7f1b78200640 1 Processor -- start 2024-04-25T12:36:31.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.352+0000 7f1b78200640 1 -- start start 2024-04-25T12:36:31.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.355+0000 7f1b78200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1b700c4790 0x7f1b70154350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.355+0000 7f1b78200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b700cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.355+0000 7f1b78200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7015a7c0 con 0x7f1b700cbc50 2024-04-25T12:36:31.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.355+0000 7f1b78200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1b70154890 con 0x7f1b700c4790 2024-04-25T12:36:31.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.355+0000 7f1b78200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7015aa80 con 0x7f1b700caef0 2024-04-25T12:36:31.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.356+0000 7f1b75800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1b700cbc50 0x7f1b700cc8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35852/0 (socket says 172.21.15.123:35852) 2024-04-25T12:36:31.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.356+0000 7f1b75800640 1 -- 172.21.15.123:0/2868118473 learned_addr learned my addr 172.21.15.123:0/2868118473 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:31.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.356+0000 7f1b76200640 1 --2- 172.21.15.123:0/2868118473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b700cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.357+0000 7f1b74e00640 1 --2- 172.21.15.123:0/2868118473 >> v2:172.21.15.123:3300/0 conn(0x7f1b700c4790 0x7f1b70154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.357+0000 7f1b6fe00640 1 -- 172.21.15.123:0/2868118473 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328125512 0 0) 0x7f1b7015a7c0 con 0x7f1b700cbc50 2024-04-25T12:36:31.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.357+0000 7f1b6fe00640 1 -- 172.21.15.123:0/2868118473 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b58003520 con 0x7f1b700cbc50 2024-04-25T12:36:31.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.357+0000 7f1b76200640 1 -- 172.21.15.123:0/2868118473 >> v2:172.21.15.123:3300/0 conn(0x7f1b700c4790 msgr2=0x7f1b70154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:31.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.357+0000 7f1b76200640 1 --2- 172.21.15.123:0/2868118473 >> v2:172.21.15.123:3300/0 conn(0x7f1b700c4790 0x7f1b70154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.357+0000 7f1b76200640 1 -- 172.21.15.123:0/2868118473 >> v1:172.21.15.123:6789/0 conn(0x7f1b700cbc50 legacy=0x7f1b700cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.358+0000 7f1b76200640 1 -- 172.21.15.123:0/2868118473 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7015ad60 con 0x7f1b700caef0 2024-04-25T12:36:31.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.358+0000 7f1b74e00640 1 --2- 172.21.15.123:0/2868118473 >> v2:172.21.15.123:3300/0 conn(0x7f1b700c4790 0x7f1b70154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:31.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.359+0000 7f1b76200640 1 --2- 172.21.15.123:0/2868118473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b700cb300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b70154a10 tx=0x7f1b6802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa1d9eff7503811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.360+0000 7f1b6fe00640 1 -- 172.21.15.123:0/2868118473 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007740 con 0x7f1b700caef0 2024-04-25T12:36:31.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.361+0000 7f1b6fe00640 1 -- 172.21.15.123:0/2868118473 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b680078e0 con 0x7f1b700caef0 2024-04-25T12:36:31.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.361+0000 7f1b6fe00640 1 -- 172.21.15.123:0/2868118473 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007c90 con 0x7f1b700caef0 2024-04-25T12:36:31.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 -- 172.21.15.123:0/2868118473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 msgr2=0x7f1b700cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 --2- 172.21.15.123:0/2868118473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b700cb300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b70154a10 tx=0x7f1b6802a5f0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 -- 172.21.15.123:0/2868118473 shutdown_connections 2024-04-25T12:36:31.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 --2- 172.21.15.123:0/2868118473 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b700cb300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 --2- 172.21.15.123:0/2868118473 >> v2:172.21.15.123:3300/0 conn(0x7f1b700c4790 0x7f1b70154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 -- 172.21.15.123:0/2868118473 >> 172.21.15.123:0/2868118473 conn(0x7f1b70068600 msgr2=0x7f1b70068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:31.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 -- 172.21.15.123:0/2868118473 shutdown_connections 2024-04-25T12:36:31.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.363+0000 7f1b78200640 1 -- 172.21.15.123:0/2868118473 wait complete. 2024-04-25T12:36:31.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.367+0000 7f1b78200640 1 Processor -- start 2024-04-25T12:36:31.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.367+0000 7f1b78200640 1 -- start start 2024-04-25T12:36:31.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.369+0000 7f1b78200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b700c4790 0x7f1b70155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b78200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b7015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b78200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b700cbc50 0x7f1b7015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b78200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7015b320 con 0x7f1b700c4790 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b78200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b700cc380 con 0x7f1b700cbc50 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b78200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7015a410 con 0x7f1b700caef0 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b75800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b700c4790 0x7f1b70155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b75800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b700c4790 0x7f1b70155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60982/0 (socket says 172.21.15.123:60982) 2024-04-25T12:36:31.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b75800640 1 -- 172.21.15.123:0/503404173 learned_addr learned my addr 172.21.15.123:0/503404173 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:31.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.370+0000 7f1b74e00640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b7015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.371+0000 7f1b74e00640 1 -- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b700cbc50 msgr2=0x7f1b7015dd50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T12:36:31.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.371+0000 7f1b74e00640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b700cbc50 0x7f1b7015dd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.371+0000 7f1b74e00640 1 -- 172.21.15.123:0/503404173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b700c4790 msgr2=0x7f1b70155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.371+0000 7f1b74e00640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b700c4790 0x7f1b70155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.371+0000 7f1b74e00640 1 -- 172.21.15.123:0/503404173 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b70156060 con 0x7f1b700caef0 2024-04-25T12:36:31.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.371+0000 7f1b74e00640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b7015d810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00e870 tx=0x7f1b5c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.372+0000 7f1b6d600640 1 -- 172.21.15.123:0/503404173 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c017070 con 0x7f1b700caef0 2024-04-25T12:36:31.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.372+0000 7f1b6d600640 1 -- 172.21.15.123:0/503404173 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c00c8e0 con 0x7f1b700caef0 2024-04-25T12:36:31.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.372+0000 7f1b6d600640 1 -- 172.21.15.123:0/503404173 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00ccb0 con 0x7f1b700caef0 2024-04-25T12:36:31.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.372+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b70149790 con 0x7f1b700caef0 2024-04-25T12:36:31.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.372+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b70155320 con 0x7f1b700caef0 2024-04-25T12:36:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.374+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b701494f0 con 0x7f1b700caef0 2024-04-25T12:36:31.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.376+0000 7f1b6d600640 1 -- 172.21.15.123:0/503404173 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1b5c01a070 con 0x7f1b700caef0 2024-04-25T12:36:31.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.378+0000 7f1b6d600640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1b500744e0 0x7f1b500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.378+0000 7f1b6d600640 1 -- 172.21.15.123:0/503404173 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(60..60 src has 1..60) ==== 6685+0+0 (secure 0 0 0) 0x7f1b5c095570 con 0x7f1b700caef0 2024-04-25T12:36:31.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.378+0000 7f1b75800640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1b500744e0 0x7f1b500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.379+0000 7f1b75800640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1b500744e0 0x7f1b500769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1b640076b0 tx=0x7f1b64007e60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.380+0000 7f1b6d600640 1 -- 172.21.15.123:0/503404173 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b701494f0 con 0x7f1b700caef0 2024-04-25T12:36:31.388 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:31.389+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:31.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.396+0000 7fc4d9600640 1 Processor -- start 2024-04-25T12:36:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.397+0000 7fc4d9600640 1 -- start start 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.400+0000 7fc4d9600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc4d41568c0 0x7fc4d40ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.400+0000 7fc4d9600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c6800 0x7fc4d415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.400+0000 7fc4d9600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d40cedf0 con 0x7fc4d40cdca0 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.400+0000 7fc4d9600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d40cebd0 con 0x7fc4d41568c0 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.400+0000 7fc4d9600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40cf0d0 con 0x7fc4d40c6800 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.401+0000 7fc4d3e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc4d41568c0 0x7fc4d40ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.401+0000 7fc4d3400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc4d40cdca0 0x7fc4d40ccf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35868/0 (socket says 172.21.15.123:35868) 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.401+0000 7fc4d3400640 1 -- 172.21.15.123:0/1545564574 learned_addr learned my addr 172.21.15.123:0/1545564574 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.401+0000 7fc4d8c00640 1 --2- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c6800 0x7fc4d415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.401+0000 7fc4d3e00640 1 -- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c6800 msgr2=0x7fc4d415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:31.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.401+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c6800 0x7fc4d415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.401+0000 7fc4d3e00640 1 -- 172.21.15.123:0/1545564574 >> v1:172.21.15.123:6789/0 conn(0x7fc4d40cdca0 legacy=0x7fc4d40ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.402+0000 7fc4d3e00640 1 -- 172.21.15.123:0/1545564574 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d40c7df0 con 0x7fc4d41568c0 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.403+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d41568c0 0x7fc4d40ce690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009d90 tx=0x7fc4cc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940e35f211b8d15d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.404+0000 7fc4d2a00640 1 -- 172.21.15.123:0/1545564574 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc007740 con 0x7fc4d41568c0 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.404+0000 7fc4d2a00640 1 -- 172.21.15.123:0/1545564574 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc0078e0 con 0x7fc4d41568c0 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.404+0000 7fc4d2a00640 1 -- 172.21.15.123:0/1545564574 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc007cb0 con 0x7fc4d41568c0 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 -- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d41568c0 msgr2=0x7fc4d40ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 --2- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d41568c0 0x7fc4d40ce690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009d90 tx=0x7fc4cc02a660 comp rx=0 tx=0).stop 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 -- 172.21.15.123:0/1545564574 shutdown_connections 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 --2- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40c6800 0x7fc4d415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 --2- 172.21.15.123:0/1545564574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d41568c0 0x7fc4d40ce690 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 -- 172.21.15.123:0/1545564574 >> 172.21.15.123:0/1545564574 conn(0x7fc4d406dcd0 msgr2=0x7fc4d406e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 -- 172.21.15.123:0/1545564574 shutdown_connections 2024-04-25T12:36:31.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.405+0000 7fc4d9600640 1 -- 172.21.15.123:0/1545564574 wait complete. 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.409+0000 7fc4d9600640 1 Processor -- start 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.410+0000 7fc4d9600640 1 -- start start 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.412+0000 7fc4d9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d40c6800 0x7fc4d4158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.412+0000 7fc4d9600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40cdca0 0x7fc4d41f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.412+0000 7fc4d9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4156ec0 0x7fc4d4148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4156ec0 0x7fc4d4148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d3400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40cdca0 0x7fc4d41f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d3400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40cdca0 0x7fc4d41f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:34650/0 (socket says 172.21.15.123:34650) 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d9600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40c83b0 con 0x7fc4d4156ec0 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d9600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40c8720 con 0x7fc4d40c6800 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d9600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40cd630 con 0x7fc4d40cdca0 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4156ec0 0x7fc4d4148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60996/0 (socket says 172.21.15.123:60996) 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d8c00640 1 -- 172.21.15.123:0/2956800111 learned_addr learned my addr 172.21.15.123:0/2956800111 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.413+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d40c6800 0x7fc4d4158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.414+0000 7fc4d3e00640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40cdca0 msgr2=0x7fc4d41f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.414+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc4d40cdca0 0x7fc4d41f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.414+0000 7fc4d3e00640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4156ec0 msgr2=0x7fc4d4148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.414+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d4156ec0 0x7fc4d4148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.414+0000 7fc4d3e00640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4149a60 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.414+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d40c6800 0x7fc4d4158c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc031000 tx=0x7fc4cc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.415+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc03f070 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.415+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc031be0 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.415+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc043410 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.415+0000 7fc4d9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d4154bc0 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.415+0000 7fc4d9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d4155180 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.417+0000 7fc4d9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc4d4076da0 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.419+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc4cc065a80 con 0x7fc4d40c6800 2024-04-25T12:36:31.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.421+0000 7fc4cbe00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b40744b0 0x7fc4b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.421+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6685+0+0 (secure 0 0 0) 0x7fc4cc0b69f0 con 0x7fc4d40c6800 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.421+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v60) ==== 54+0+16 (secure 0 0 0) 0x7fc4cc0b70a0 con 0x7fc4d40c6800 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.422+0000 7fc4d3400640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b40744b0 0x7fc4b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.423+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc4a8001650 0x7fc4a8003b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.423+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4a8006c00 con 0x7fc4a8001650 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.424+0000 7fc4d3400640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b40744b0 0x7fc4b40769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4d415d330 tx=0x7fc4c4007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.424+0000 7fc4d8c00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc4a8001650 0x7fc4a8003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.426+0000 7fc4d8c00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc4a8001650 0x7fc4a8003b60 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.429+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a8006c00 con 0x7fc4a8001650 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.460+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a8005cb0 con 0x7fc4a8001650 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.463+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a8005cb0 con 0x7fc4a8001650 2024-04-25T12:36:31.540 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:31.541 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:31.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.464+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc4a80041e0 0x7fc4a80045f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.464+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc4a8004c70 con 0x7fc4a80041e0 2024-04-25T12:36:31.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.464+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc4a80041e0 0x7fc4a80045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.465+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc4a80041e0 0x7fc4a80045f0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.466+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a8004c70 con 0x7fc4a80041e0 2024-04-25T12:36:31.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.498+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a80051e0 con 0x7fc4a80041e0 2024-04-25T12:36:31.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.501+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a80051e0 con 0x7fc4a80041e0 2024-04-25T12:36:31.542 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:31.543 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:31.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.501+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc4a80093c0 0x7fc4a800b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.502+0000 7fc4d3400640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc4a80093c0 0x7fc4a800b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.503+0000 7fc4d3400640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc4a80093c0 0x7fc4a800b7e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.503+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc4a800e760 con 0x7fc4a80093c0 2024-04-25T12:36:31.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.505+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a800e760 con 0x7fc4a80093c0 2024-04-25T12:36:31.543 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:31.533+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.534+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a800c0a0 con 0x7fc4a80093c0 2024-04-25T12:36:31.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.537+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a800c0a0 con 0x7fc4a80093c0 2024-04-25T12:36:31.544 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.538+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc4a800c2b0 0x7fc4a800c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.538+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc4a800cc00 con 0x7fc4a800c2b0 2024-04-25T12:36:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.538+0000 7fc4d8c00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc4a800c2b0 0x7fc4a800c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.539+0000 7fc4d8c00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc4a800c2b0 0x7fc4a800c6c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.541+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a800cc00 con 0x7fc4a800c2b0 2024-04-25T12:36:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.570+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a800d110 con 0x7fc4a800c2b0 2024-04-25T12:36:31.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.573+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a800d110 con 0x7fc4a800c2b0 2024-04-25T12:36:31.574 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:31.574 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:31.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.574+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc4a800d4e0 0x7fc4a8012ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.574+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc4a8015e50 con 0x7fc4a800d4e0 2024-04-25T12:36:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.574+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc4a800d4e0 0x7fc4a8012ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.575+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc4a800d4e0 0x7fc4a8012ed0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.576+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a8015e50 con 0x7fc4a800d4e0 2024-04-25T12:36:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.605+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a8015e20 con 0x7fc4a800d4e0 2024-04-25T12:36:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.608+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a8015e20 con 0x7fc4a800d4e0 2024-04-25T12:36:31.608 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.608+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fc4a80139c0 0x7fc4a8013e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.608+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc4a8019220 con 0x7fc4a80139c0 2024-04-25T12:36:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.609+0000 7fc4d3400640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fc4a80139c0 0x7fc4a8013e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.609+0000 7fc4d3400640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fc4a80139c0 0x7fc4a8013e30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.610+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a8019220 con 0x7fc4a80139c0 2024-04-25T12:36:31.633 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:31.634+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:31.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.639+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a80180a0 con 0x7fc4a80139c0 2024-04-25T12:36:31.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.641+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a80180a0 con 0x7fc4a80139c0 2024-04-25T12:36:31.642 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:31.643 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:31.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.642+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc4a80187c0 0x7fc4a8018bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.642+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc4a801c5c0 con 0x7fc4a80187c0 2024-04-25T12:36:31.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.643+0000 7fc4d8c00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc4a80187c0 0x7fc4a8018bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.643+0000 7fc4d8c00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc4a80187c0 0x7fc4a8018bd0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.644+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a801c5c0 con 0x7fc4a80187c0 2024-04-25T12:36:31.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.673+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a8014480 con 0x7fc4a80187c0 2024-04-25T12:36:31.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.676+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a8014480 con 0x7fc4a80187c0 2024-04-25T12:36:31.676 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.676+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc4a8014b00 0x7fc4a8014f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.676+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc4a80156c0 con 0x7fc4a8014b00 2024-04-25T12:36:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.677+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc4a8014b00 0x7fc4a8014f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.677+0000 7fc4d3e00640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc4a8014b00 0x7fc4a8014f10 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.678+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a80156c0 con 0x7fc4a8014b00 2024-04-25T12:36:31.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.707+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4a80156c0 con 0x7fc4a8014b00 2024-04-25T12:36:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4cbe00640 1 -- 172.21.15.123:0/2956800111 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4a80156c0 con 0x7fc4a8014b00 2024-04-25T12:36:31.710 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc4a8001650 msgr2=0x7fc4a8003b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fc4a8001650 0x7fc4a8003b60 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc4a80041e0 msgr2=0x7fc4a80045f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fc4a80041e0 0x7fc4a80045f0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc4a80093c0 msgr2=0x7fc4a800b7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fc4a80093c0 0x7fc4a800b7e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc4a800c2b0 msgr2=0x7fc4a800c6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fc4a800c2b0 0x7fc4a800c6c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc4a800d4e0 msgr2=0x7fc4a8012ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.710+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fc4a800d4e0 0x7fc4a8012ed0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fc4a80139c0 msgr2=0x7fc4a8013e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fc4a80139c0 0x7fc4a8013e30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc4a80187c0 msgr2=0x7fc4a8018bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fc4a80187c0 0x7fc4a8018bd0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc4a8014b00 msgr2=0x7fc4a8014f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fc4a8014b00 0x7fc4a8014f10 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4d8c00640 1 -- 172.21.15.123:0/2956800111 reap_dead start 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4d8c00640 1 -- 172.21.15.123:0/2956800111 reap_dead start 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4d8c00640 1 -- 172.21.15.123:0/2956800111 reap_dead start 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4d8c00640 1 -- 172.21.15.123:0/2956800111 reap_dead start 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.711+0000 7fc4d8c00640 1 -- 172.21.15.123:0/2956800111 reap_dead start 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b40744b0 msgr2=0x7fc4b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b40744b0 0x7fc4b40769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4d415d330 tx=0x7fc4c4007e80 comp rx=0 tx=0).stop 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d40c6800 msgr2=0x7fc4d4158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d40c6800 0x7fc4d4158c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc031000 tx=0x7fc4cc004a40 comp rx=0 tx=0).stop 2024-04-25T12:36:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 shutdown_connections 2024-04-25T12:36:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fc4b40744b0 0x7fc4b40769c0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4d415d330 tx=0x7fc4c4007e80 comp rx=0 tx=0).stop 2024-04-25T12:36:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 --2- 172.21.15.123:0/2956800111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4d40c6800 0x7fc4d4158c90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 >> 172.21.15.123:0/2956800111 conn(0x7fc4d406dcd0 msgr2=0x7fc4d4150b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 shutdown_connections 2024-04-25T12:36:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.712+0000 7fc4c9600640 1 -- 172.21.15.123:0/2956800111 wait complete. 2024-04-25T12:36:31.716 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:31.717 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:31.734 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:31.734+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:31.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:31.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.788+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f1b70076700 con 0x7f1b700caef0 2024-04-25T12:36:31.836 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:31.836+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:31.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.914+0000 7f1b6d600640 1 -- 172.21.15.123:0/503404173 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v61) ==== 91+0+0 (secure 0 0 0) 0x7f1b5c05f350 con 0x7f1b700caef0 2024-04-25T12:36:31.915 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:36:31.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1b500744e0 msgr2=0x7f1b500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1b500744e0 0x7f1b500769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1b640076b0 tx=0x7f1b64007e60 comp rx=0 tx=0).stop 2024-04-25T12:36:31.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 msgr2=0x7f1b7015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:31.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b7015d810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00e870 tx=0x7f1b5c00ed30 comp rx=0 tx=0).stop 2024-04-25T12:36:31.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 shutdown_connections 2024-04-25T12:36:31.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1b500744e0 0x7f1b500769f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b700cbc50 0x7f1b7015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b700caef0 0x7f1b7015d810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 --2- 172.21.15.123:0/503404173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b700c4790 0x7f1b70155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:31.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.920+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 >> 172.21.15.123:0/503404173 conn(0x7f1b70068600 msgr2=0x7f1b70153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:31.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.921+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 shutdown_connections 2024-04-25T12:36:31.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:31.921+0000 7f1b78200640 1 -- 172.21.15.123:0/503404173 wait complete. 2024-04-25T12:36:31.937 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:31.937+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:32.037 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:32.038+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:32.138 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:32.138+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:32.166 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:32.166 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:32.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:32.166 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:32.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 144.13979585999999, 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53759151699999996, 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.167 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024334+0000", 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7179999999999999e-06 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024345+0000", 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0608000000000001e-05 2024-04-25T12:36:32.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024373+0000", 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7966e-05 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024392+0000", 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024490+0000", 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7823000000000002e-05 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for peered", 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024520+0000", 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0056000000000002e-05 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.559770+0000", 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53525036000000004 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560001+0000", 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023139100000000001 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560330+0000", 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032824500000000001 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561700+0000", 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013704730000000001 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561742+0000", 2024-04-25T12:36:32.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1826000000000002e-05 2024-04-25T12:36:32.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561855+0000", 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011287599999999999 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561924+0000", 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9484000000000001e-05 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 143.57687264099999, 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095393000000000006, 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587257+0000", 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9100000000000003e-07 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587490+0000", 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000233186 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587502+0000", 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2499e-05 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587512+0000", 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7699999999999996e-06 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.591891+0000", 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043783770000000001 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592036+0000", 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014543999999999999 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592306+0000", 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026964399999999998 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595185+0000", 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028791730000000001 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596722+0000", 2024-04-25T12:36:32.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001537131 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596762+0000", 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0491000000000002e-05 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596795+0000", 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2798e-05 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 143.576599722, 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012300247, 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 24 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587530+0000", 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3799999999999996e-07 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587688+0000", 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015835999999999999 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587699+0000", 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0727000000000001e-05 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587710+0000", 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0987e-05 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593399+0000", 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005688935 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593685+0000", 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028662199999999999 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593893+0000", 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020758500000000001 2024-04-25T12:36:32.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595599+0000", 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017058209999999999 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599668+0000", 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040696439999999999 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599769+0000", 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100752 2024-04-25T12:36:32.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599829+0000", 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0075999999999999e-05 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 143.57637766400001, 2024-04-25T12:36:32.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012351434, 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 25 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587752+0000", 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587758+0000", 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2199999999999997e-06 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587770+0000", 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2129e-05 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587778+0000", 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594340+0000", 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065618270000000001 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594415+0000", 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4535000000000004e-05 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594510+0000", 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5100999999999996e-05 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595607+0000", 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010968239999999999 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599926+0000", 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043194469999999997 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600027+0000", 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100961 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600102+0000", 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5339000000000005e-05 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.192 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 143.371428474, 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091591999999999993, 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-04-25T12:36:32.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792701+0000", 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.17e-06 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792911+0000", 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000210163 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792932+0000", 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0472e-05 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792952+0000", 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0553000000000001e-05 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799157+0000", 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062046030000000004 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799246+0000", 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8505999999999996e-05 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799389+0000", 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014319100000000001 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801232+0000", 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018427599999999999 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801753+0000", 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052153699999999998 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801814+0000", 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1014000000000002e-05 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801859+0000", 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5231000000000001e-05 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.198 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 143.37099012600001, 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092910930000000003, 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 52 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793140+0000", 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-06 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793151+0000", 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0966e-05 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793168+0000", 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793184+0000", 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6200000000000001e-05 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801271+0000", 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080863189999999998 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801360+0000", 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9248999999999999e-05 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801515+0000", 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000154883 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802197+0000", 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068243299999999998 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802339+0000", 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014159 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802393+0000", 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4215000000000002e-05 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802430+0000", 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6434999999999999e-05 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.204 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.912986856, 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012798129, 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 70 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251143+0000", 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2729999999999999e-06 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251153+0000", 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8439999999999993e-06 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251172+0000", 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8906000000000002e-05 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251183+0000", 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1653e-05 2024-04-25T12:36:32.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257332+0000", 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061486459999999998 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257490+0000", 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000157947 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257724+0000", 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023408200000000001 2024-04-25T12:36:32.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259695+0000", 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001971169 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263731+0000", 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035683 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263842+0000", 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110985 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263940+0000", 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7941000000000003e-05 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:32.210 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.91292994700001, 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013164844, 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 71 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251200+0000", 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2999999999999999e-07 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251206+0000", 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9399999999999999e-06 2024-04-25T12:36:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251215+0000", 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251223+0000", 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2260000000000005e-06 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259955+0000", 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087315629999999995 2024-04-25T12:36:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260077+0000", 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122263 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260310+0000", 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232702 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263699+0000", 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033888080000000001 2024-04-25T12:36:32.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264218+0000", 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051973199999999996 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264304+0000", 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5433999999999996e-05 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264363+0000", 2024-04-25T12:36:32.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9743e-05 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.91288833799999, 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017184050999999999, 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.216 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 72 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251241+0000", 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.23e-07 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251477+0000", 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023584499999999999 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251504+0000", 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7633000000000001e-05 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251516+0000", 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.164e-05 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260993+0000", 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094771119999999993 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261122+0000", 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012831400000000001 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261319+0000", 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019727199999999999 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264078+0000", 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027589120000000001 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268258+0000", 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041806569999999999 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268357+0000", 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8721000000000006e-05 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268424+0000", 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7222000000000001e-05 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.91259339499999, 2024-04-25T12:36:32.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017105597, 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 73 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:32.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251536+0000", 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4300000000000001e-07 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251543+0000", 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3840000000000002e-06 2024-04-25T12:36:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251552+0000", 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6860000000000001e-06 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251560+0000", 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0499999999999992e-06 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262615+0000", 2024-04-25T12:36:32.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011054494 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262757+0000", 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142521 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262999+0000", 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024147400000000001 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264190+0000", 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011911840000000001 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268472+0000", 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042823940000000001 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268571+0000", 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8218000000000001e-05 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268641+0000", 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0248999999999998e-05 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.91255640899999, 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017309201999999999, 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.227 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 74 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:32.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251573+0000", 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9999999999999997e-07 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251579+0000", 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6330000000000001e-06 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251592+0000", 2024-04-25T12:36:32.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3869999999999999e-05 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251621+0000", 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8966e-05 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264505+0000", 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012883307 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264627+0000", 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012204 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264832+0000", 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020571899999999999 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268227+0000", 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003394308 2024-04-25T12:36:32.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268663+0000", 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043630699999999998 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268761+0000", 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7774000000000003e-05 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268881+0000", 2024-04-25T12:36:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120578 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.91248630000001, 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01751983, 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.233 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 75 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251643+0000", 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5300000000000003e-07 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251849+0000", 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000206345 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251863+0000", 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3077e-05 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251873+0000", 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.08e-05 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265525+0000", 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013651876 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265667+0000", 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014156699999999999 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265939+0000", 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027219700000000001 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268931+0000", 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029921990000000001 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269008+0000", 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6619000000000006e-05 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269082+0000", 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4401999999999997e-05 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269162+0000", 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9895000000000007e-05 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.912238632, 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018357248, 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 76 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251891+0000", 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7300000000000005e-07 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251905+0000", 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4279e-05 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251915+0000", 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251935+0000", 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.961e-05 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267152+0000", 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015217341000000001 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267276+0000", 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012396 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267509+0000", 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023329500000000001 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268988+0000", 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014781709999999999 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270041+0000", 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001053806 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270161+0000", 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119494 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270247+0000", 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6382000000000001e-05 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.893014123, 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099439890000000003, 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 80 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271115+0000", 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271221+0000", 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105713 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271236+0000", 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5415999999999999e-05 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271240+0000", 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3449999999999999e-06 2024-04-25T12:36:32.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275441+0000", 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042009090000000001 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275579+0000", 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013740700000000001 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275795+0000", 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021634900000000001 2024-04-25T12:36:32.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.277926+0000", 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021311220000000001 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.280958+0000", 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030320490000000002 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281016+0000", 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8232000000000001e-05 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281058+0000", 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1993999999999998e-05 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.53799775799999, 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094693499999999996, 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.249 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 92 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626132+0000", 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4399999999999999e-07 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626137+0000", 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4929999999999998e-06 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626147+0000", 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5300000000000002e-06 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626155+0000", 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4130000000000006e-06 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629532+0000", 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033774830000000001 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629626+0000", 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3394999999999996e-05 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629792+0000", 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016611100000000001 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633343+0000", 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00355055 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635469+0000", 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021262009999999999 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635548+0000", 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9167999999999998e-05 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635600+0000", 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2262000000000002e-05 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.537961422, 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096152330000000008, 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 93 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.255 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626168+0000", 2024-04-25T12:36:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9699999999999995e-07 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626290+0000", 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121941 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626304+0000", 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3729e-05 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626321+0000", 2024-04-25T12:36:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7062999999999999e-05 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630147+0000", 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003826 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630229+0000", 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1953000000000006e-05 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630372+0000", 2024-04-25T12:36:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014345199999999999 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633348+0000", 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029755789999999999 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635672+0000", 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023242829999999999 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635733+0000", 2024-04-25T12:36:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0899000000000001e-05 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635782+0000", 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9636999999999999e-05 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 142.53778750199999, 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097157200000000006, 2024-04-25T12:36:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 94 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626342+0000", 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4300000000000002e-07 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626348+0000", 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0059999999999998e-06 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626358+0000", 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626367+0000", 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8939999999999999e-06 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631146+0000", 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047795429999999998 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631307+0000", 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016041699999999999 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631539+0000", 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232258 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635089+0000", 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035503980000000002 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635880+0000", 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079070200000000001 2024-04-25T12:36:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635984+0000", 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104221 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.636057+0000", 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2597999999999998e-05 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.04607795, 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010749583, 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118054+0000", 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118075+0000", 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118119+0000", 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4372000000000001e-05 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118163+0000", 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3766000000000001e-05 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119484+0000", 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001320494 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119803+0000", 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031975599999999998 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120121+0000", 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031758999999999997 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128491+0000", 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083702940000000003 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128605+0000", 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113158 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128733+0000", 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012861099999999999 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128800+0000", 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7086000000000001e-05 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.045878964, 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011301215999999999, 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118251+0000", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6160000000000001e-06 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118263+0000", 2024-04-25T12:36:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2243e-05 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118305+0000", 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1838999999999997e-05 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118327+0000", 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2082999999999998e-05 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120981+0000", 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002653201 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121221+0000", 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000240058 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121655+0000", 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043405400000000002 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128501+0000", 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068465519999999997 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128984+0000", 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048240400000000003 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129487+0000", 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050289499999999995 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129551+0000", 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4270999999999995e-05 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.04577153300001, 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012520414000000001, 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118358+0000", 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3230000000000001e-06 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118368+0000", 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3899999999999999e-06 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2481999999999999e-05 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118410+0000", 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9304e-05 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126108+0000", 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076979409999999998 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126379+0000", 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027136700000000001 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126653+0000", 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274238 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129707+0000", 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030541800000000001 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130686+0000", 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097908900000000004 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130791+0000", 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104824 2024-04-25T12:36:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130877+0000", 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6275999999999999e-05 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:32.281 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:32.283 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:32.283 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:32.283+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:32.383 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:32.383+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:32.483 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:32.484+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:32.567 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:32.568 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:32.568 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:32.568 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:32.584 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:32.585+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:32.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:32.685 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:32.686+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:32.786 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:32.786+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:32.825 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:32.826 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:32.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:32.887 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:32.887+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:32.988 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:32.988+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:33.089 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:33.089+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:33.189 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:33.189+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:33.281 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:33.281 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 134.32888291399999, 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083045749999999998, 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950865+0000", 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950868+0000", 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9210000000000001e-06 2024-04-25T12:36:33.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950882+0000", 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381e-05 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950903+0000", 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1384000000000001e-05 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950921+0000", 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7238e-05 2024-04-25T12:36:33.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951166+0000", 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024490599999999998 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951391+0000", 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000225045 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951612+0000", 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022131599999999999 2024-04-25T12:36:33.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955733+0000", 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041211190000000003 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032725409999999999 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959099+0000", 2024-04-25T12:36:33.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3420999999999999e-05 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959170+0000", 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0873999999999999e-05 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 134.32877097100001, 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012502821000000001, 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.287 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950977+0000", 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950978+0000", 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0890000000000001e-06 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950986+0000", 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.554e-06 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950998+0000", 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1788e-05 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951009+0000", 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952232+0000", 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012233019999999999 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952425+0000", 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019327400000000001 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952812+0000", 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038624999999999998 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959232+0000", 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064198110000000001 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961995+0000", 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027638889999999998 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962182+0000", 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018673100000000001 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963480+0000", 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012980050000000001 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 134.32872217100001, 2024-04-25T12:36:33.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012666844999999999, 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951026+0000", 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951027+0000", 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8599999999999998e-07 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951032+0000", 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0490000000000003e-06 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-25T12:36:33.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951052+0000", 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2259999999999994e-06 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953198+0000", 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021463979999999999 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953345+0000", 2024-04-25T12:36:33.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014680499999999999 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953519+0000", 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017397300000000001 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960234+0000", 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006715408 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963501+0000", 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032672170000000002 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963609+0000", 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001081 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963693+0000", 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3523999999999995e-05 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.297 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 134.32865101100001, 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012790558, 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951097+0000", 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951098+0000", 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951107+0000", 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7569999999999993e-06 2024-04-25T12:36:33.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951118+0000", 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1582e-05 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951128+0000", 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9240000000000002e-06 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954212+0000", 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030832429999999998 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954420+0000", 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000208376 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017360100000000001 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961619+0000", 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070253479999999998 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963708+0000", 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020888999999999999 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963814+0000", 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106136 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963888+0000", 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3862000000000005e-05 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.302 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 134.32860259899999, 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012965035, 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951146+0000", 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8599999999999997e-07 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951151+0000", 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1670000000000001e-06 2024-04-25T12:36:33.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951160+0000", 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.268e-06 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951169+0000", 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6139999999999992e-06 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955155+0000", 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039863429999999998 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955328+0000", 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017313300000000001 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964111+0000", 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087827240000000004 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.306 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:28 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 134.320406671, 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002757158, 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959342+0000", 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959344+0000", 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959364+0000", 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9675999999999999e-05 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959395+0000", 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1801999999999998e-05 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959432+0000", 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6225e-05 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959692+0000", 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000260659 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959753+0000", 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0498999999999998e-05 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961984+0000", 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022308300000000001 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962099+0000", 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011518699999999999 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:42 2.14 2:2e2fa760:::200.00000001:head [write 3029~141 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.214162673, 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002822162, 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065586+0000", 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065589+0000", 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065610+0000", 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0840999999999999e-05 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065650+0000", 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9875e-05 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065675+0000", 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4745000000000001e-05 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065798+0000", 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012348200000000001 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065919+0000", 2024-04-25T12:36:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120318 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066173+0000", 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025423299999999998 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067842+0000", 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016688129999999999 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068247+0000", 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040533099999999999 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068333+0000", 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5687999999999997e-05 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.068408+0000", 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5146000000000002e-05 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:43 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.21398067200001, 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041608499999999998, 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065768+0000", 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065769+0000", 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2079999999999999e-06 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065814+0000", 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975000000000002e-05 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065828+0000", 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4599e-05 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.065868+0000", 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0024999999999997e-05 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.066893+0000", 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010249219999999999 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067011+0000", 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117871 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.067186+0000", 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017516400000000001 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069129+0000", 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019425550000000001 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069785+0000", 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065641899999999999 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069866+0000", 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1052999999999998e-05 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:36.069928+0000", 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2058999999999997e-05 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:15 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.161946685, 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011824773, 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:33.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117802+0000", 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117804+0000", 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9750000000000001e-06 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117813+0000", 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3759999999999997e-06 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117834+0000", 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1124999999999998e-05 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117847+0000", 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2883e-05 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118025+0000", 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177839 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118395+0000", 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037031800000000001 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034116300000000001 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129488+0000", 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010751993 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129506+0000", 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7448e-05 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129566+0000", 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0309999999999997e-05 2024-04-25T12:36:33.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129626+0000", 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0343000000000001e-05 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:16 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.161870729, 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011946005000000001, 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.326 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117878+0000", 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9999999999999997e-07 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117883+0000", 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6399999999999996e-06 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117891+0000", 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5329999999999994e-06 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117898+0000", 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9500000000000004e-06 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122995+0000", 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050967080000000001 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123229+0000", 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023451099999999999 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123664+0000", 2024-04-25T12:36:33.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043457400000000002 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129649+0000", 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059846480000000004 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129705+0000", 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6121e-05 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129779+0000", 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4249e-05 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129824+0000", 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4471000000000002e-05 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:7 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.15744198900001, 2024-04-25T12:36:33.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00720596, 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122306+0000", 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122308+0000", 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1900000000000002e-06 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122326+0000", 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7683e-05 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122365+0000", 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9269e-05 2024-04-25T12:36:33.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122389+0000", 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3946000000000001e-05 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122662+0000", 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000272544 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122756+0000", 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4418000000000003e-05 2024-04-25T12:36:33.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129416+0000", 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066597100000000001 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129512+0000", 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6199999999999994e-05 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:8 2.b e25/18 2:d70a01c6:::60c.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.15725992599999, 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070621019999999998, 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122488+0000", 2024-04-25T12:36:33.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122489+0000", 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-06 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122501+0000", 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1533e-05 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122533+0000", 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2396e-05 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122552+0000", 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649000000000001e-05 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123339+0000", 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078653600000000001 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123391+0000", 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2423000000000003e-05 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129516+0000", 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061249310000000001 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129550+0000", 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4520999999999997e-05 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:24 2.3 e25/18 2:c112adae:::101.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.157155562, 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069897529999999996, 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122593+0000", 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7000000000000004e-07 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122680+0000", 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6503999999999997e-05 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122701+0000", 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0590000000000001e-05 2024-04-25T12:36:33.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0796e-05 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124644+0000", 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001902118 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124691+0000", 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7499e-05 2024-04-25T12:36:33.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048613789999999999 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129582+0000", 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0097e-05 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:26 2.3 e25/18 2:c27f454c:::mds1_sessionmap:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:33.343 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.155880005, 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068121010000000001, 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123868+0000", 2024-04-25T12:36:33.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123869+0000", 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123880+0000", 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0485999999999999e-05 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123895+0000", 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5132e-05 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123904+0000", 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4210000000000001e-06 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125541+0000", 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016370320000000001 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125866+0000", 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032462000000000002 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130515+0000", 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046495659999999999 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130680+0000", 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016496800000000001 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:21 2.8 e25/18 2:1470b2f8:::613.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 103.154825781, 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048991699999999996, 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124923+0000", 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124924+0000", 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9599999999999999e-06 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125033+0000", 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010833800000000001 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125060+0000", 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7589e-05 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125083+0000", 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2390000000000001e-05 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125507+0000", 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000424268 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125527+0000", 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129725+0000", 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004197302 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129822+0000", 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7153999999999998e-05 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.351 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:24 2.3 e25/18 2:c216c0d1:::mds2_inotable:head v 25'3, mlcod=25'1)", 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 98.125960140999993, 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041162450000000001, 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:33.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153788+0000", 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153791+0000", 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.39e-06 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:33.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0996e-05 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153859+0000", 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7159e-05 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153885+0000", 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6588999999999999e-05 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154838+0000", 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00095239299999999995 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154899+0000", 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1297999999999995e-05 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157726+0000", 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028268339999999999 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000178586 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:45 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.404448646000006, 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00316285, 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875300+0000", 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875302+0000", 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999999e-06 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875314+0000", 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2114e-05 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875335+0000", 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0951e-05 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.875355+0000", 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0339999999999998e-05 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876423+0000", 2024-04-25T12:36:33.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010681950000000001 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876599+0000", 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017600999999999999 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.876851+0000", 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025160999999999998 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878088+0000", 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012375839999999999 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878331+0000", 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000242227 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878383+0000", 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2510999999999999e-05 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:08.878462+0000", 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9274e-05 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.362 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:46 2.14 2:2e2fa760:::200.00000001:head [write 3428~2077 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e27)", 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.361888093999994, 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003124501, 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917860+0000", 2024-04-25T12:36:33.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917863+0000", 2024-04-25T12:36:33.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2660000000000002e-06 2024-04-25T12:36:33.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917884+0000", 2024-04-25T12:36:33.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0953000000000001e-05 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917916+0000", 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1412999999999999e-05 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.917936+0000", 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0083000000000001e-05 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918046+0000", 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110029 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918170+0000", 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012447399999999999 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.918392+0000", 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000221654 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920042+0000", 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001649958 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920847+0000", 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080491999999999998 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920919+0000", 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1791000000000004e-05 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:12.920985+0000", 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5959999999999999e-05 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:48 2.14 2:2e2fa760:::200.00000001:head [write 5505~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.489496162999998, 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010227916, 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:33.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790252+0000", 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790256+0000", 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7570000000000002e-06 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790280+0000", 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4456000000000001e-05 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790319+0000", 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9103000000000002e-05 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790344+0000", 2024-04-25T12:36:33.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4998999999999999e-05 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790557+0000", 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021251199999999999 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790572+0000", 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5003e-05 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.796879+0000", 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063072359999999999 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797129+0000", 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025002800000000001 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.797609+0000", 2024-04-25T12:36:33.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047965200000000001 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800188+0000", 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002578922 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800281+0000", 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3561999999999996e-05 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800387+0000", 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010545800000000001 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800480+0000", 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3227999999999996e-05 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:49 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-04-25T12:36:33.376 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.489226582000001, 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011186097000000001, 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790522+0000", 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790524+0000", 2024-04-25T12:36:33.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999998e-06 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790541+0000", 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7567e-05 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790566+0000", 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4338999999999999e-05 2024-04-25T12:36:33.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790581+0000", 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5123000000000001e-05 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790708+0000", 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012676699999999999 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.790718+0000", 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.798785+0000", 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080674500000000003 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799024+0000", 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000239044 2024-04-25T12:36:33.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.799376+0000", 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035158299999999998 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.800980+0000", 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016040939999999999 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801537+0000", 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055677400000000001 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801639+0000", 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102388 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:35:46.801708+0000", 2024-04-25T12:36:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8504999999999995e-05 2024-04-25T12:36:33.384 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.384 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.384 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.384 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:33.384 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:33.384 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:33.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:33.386 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:33.385+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:33.486 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:33.486+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:33.587 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:33.587+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:33.676 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:33.677 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:33.677 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:33.677 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:33.688 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:33.689+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:33.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:33.755 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:33.828 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:33.828+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:33.890 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:33.890+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:33.964 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:36:34.015 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:34.015+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:34.016 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:34.016 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:34.016 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:34.016 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:34.016 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:34.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:34.092 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:34.093+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:34.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.146+0000 7f0947c00640 1 Processor -- start 2024-04-25T12:36:34.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.146+0000 7f0947c00640 1 -- start start 2024-04-25T12:36:34.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.149+0000 7f0947c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0940161470 0x7f09400ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.149+0000 7f0947c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09400ca9b0 con 0x7f09400ca5a0 2024-04-25T12:36:34.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.149+0000 7f0947c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f094015b0b0 con 0x7f094015aca0 2024-04-25T12:36:34.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.149+0000 7f0947c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0940074170 con 0x7f0940161470 2024-04-25T12:36:34.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.149+0000 7f093fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0940161470 0x7f09400ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.149+0000 7f093fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0940161470 0x7f09400ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:45966/0 (socket says 172.21.15.123:45966) 2024-04-25T12:36:34.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.149+0000 7f093fe00640 1 -- 172.21.15.123:0/201748155 learned_addr learned my addr 172.21.15.123:0/201748155 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:34.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.150+0000 7f093fe00640 1 -- 172.21.15.123:0/201748155 >> v1:172.21.15.153:6789/0 conn(0x7f094015aca0 legacy=0x7f09400c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.150+0000 7f093fe00640 1 -- 172.21.15.123:0/201748155 >> v1:172.21.15.123:6789/0 conn(0x7f09400ca5a0 legacy=0x7f094016b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:34.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.150+0000 7f093fe00640 1 -- 172.21.15.123:0/201748155 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0940073480 con 0x7f0940161470 2024-04-25T12:36:34.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.151+0000 7f093fe00640 1 --2- 172.21.15.123:0/201748155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0940161470 0x7f09400ca060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f092c0098a0 tx=0x7f092c02a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc59e726f3499b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.152+0000 7f093f400640 1 -- 172.21.15.123:0/201748155 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c037070 con 0x7f0940161470 2024-04-25T12:36:34.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.152+0000 7f093f400640 1 -- 172.21.15.123:0/201748155 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092c032640 con 0x7f0940161470 2024-04-25T12:36:34.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.152+0000 7f093f400640 1 -- 172.21.15.123:0/201748155 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c032a10 con 0x7f0940161470 2024-04-25T12:36:34.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.153+0000 7f0947c00640 1 -- 172.21.15.123:0/201748155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0940161470 msgr2=0x7f09400ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.153+0000 7f0947c00640 1 --2- 172.21.15.123:0/201748155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0940161470 0x7f09400ca060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f092c0098a0 tx=0x7f092c02a330 comp rx=0 tx=0).stop 2024-04-25T12:36:34.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.153+0000 7f0947c00640 1 -- 172.21.15.123:0/201748155 shutdown_connections 2024-04-25T12:36:34.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.153+0000 7f0947c00640 1 --2- 172.21.15.123:0/201748155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0940161470 0x7f09400ca060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.154+0000 7f0947c00640 1 -- 172.21.15.123:0/201748155 >> 172.21.15.123:0/201748155 conn(0x7f094006f440 msgr2=0x7f094006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:34.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.154+0000 7f0947c00640 1 -- 172.21.15.123:0/201748155 shutdown_connections 2024-04-25T12:36:34.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.154+0000 7f0947c00640 1 -- 172.21.15.123:0/201748155 wait complete. 2024-04-25T12:36:34.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.157+0000 7f0947c00640 1 Processor -- start 2024-04-25T12:36:34.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.158+0000 7f0947c00640 1 -- start start 2024-04-25T12:36:34.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f0947c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09400ca5a0 0x7f0940152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f0947c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f094015aca0 0x7f09401fa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f0947c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940161470 0x7f094014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f0947c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0940071220 con 0x7f09400ca5a0 2024-04-25T12:36:34.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f0947c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0940073230 con 0x7f0940161470 2024-04-25T12:36:34.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f0947c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f09400708f0 con 0x7f094015aca0 2024-04-25T12:36:34.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f093fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f094015aca0 0x7f09401fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.160+0000 7f093fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f094015aca0 0x7f09401fa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:45980/0 (socket says 172.21.15.123:45980) 2024-04-25T12:36:34.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.161+0000 7f093fe00640 1 -- 172.21.15.123:0/1944381479 learned_addr learned my addr 172.21.15.123:0/1944381479 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:34.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.161+0000 7f093fe00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940161470 msgr2=0x7f094014daf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:36:34.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.161+0000 7f093fe00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940161470 0x7f094014daf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.161+0000 7f093fe00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09400ca5a0 msgr2=0x7f0940152b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:36:34.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.162+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09400ca5a0 0x7f0940152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.162+0000 7f093fe00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09400ca5a0 0x7f0940152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.162+0000 7f093fe00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09400c9350 con 0x7f094015aca0 2024-04-25T12:36:34.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.162+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09400ca5a0 0x7f0940152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:34.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.162+0000 7f093fe00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f094015aca0 0x7f09401fa960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f092c000c00 tx=0x7f092c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.163+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c041020 con 0x7f094015aca0 2024-04-25T12:36:34.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.163+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092c030a70 con 0x7f094015aca0 2024-04-25T12:36:34.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.163+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09401e9aa0 con 0x7f094015aca0 2024-04-25T12:36:34.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.163+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0940154040 con 0x7f094015aca0 2024-04-25T12:36:34.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.163+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c037040 con 0x7f094015aca0 2024-04-25T12:36:34.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.165+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f091c005270 con 0x7f094015aca0 2024-04-25T12:36:34.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.167+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f092c044020 con 0x7f094015aca0 2024-04-25T12:36:34.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.168+0000 7f093cc00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09200744e0 0x7f09200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.169+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 6685+0+0 (secure 0 0 0) 0x7f092c0b6380 con 0x7f094015aca0 2024-04-25T12:36:34.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.169+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v61) ==== 54+0+16 (secure 0 0 0) 0x7f092c0b8920 con 0x7f094015aca0 2024-04-25T12:36:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.169+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09200744e0 0x7f09200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.169+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f091c0045e0 0x7f091c0049f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.170+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f091c004f30 con 0x7f091c0045e0 2024-04-25T12:36:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.170+0000 7f0945c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f091c0045e0 0x7f091c0049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.170+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09200744e0 0x7f09200769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f09340015e0 tx=0x7f0934009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.171+0000 7f0945c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f091c0045e0 0x7f091c0049f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.173+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c004f30 con 0x7f091c0045e0 2024-04-25T12:36:34.196 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:34.196+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:34.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.204+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c0013d0 con 0x7f091c0045e0 2024-04-25T12:36:34.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.207+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c0013d0 con 0x7f091c0045e0 2024-04-25T12:36:34.207 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:34.208 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:34.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.208+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f091c001e90 0x7f091c002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.208+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f091c00b390 con 0x7f091c001e90 2024-04-25T12:36:34.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.209+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f091c001e90 0x7f091c002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.210+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f091c001e90 0x7f091c002990 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.211+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c00b390 con 0x7f091c001e90 2024-04-25T12:36:34.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.242+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c003120 con 0x7f091c001e90 2024-04-25T12:36:34.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.245+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c003120 con 0x7f091c001e90 2024-04-25T12:36:34.245 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:34.245 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:34.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.245+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f091c009c80 0x7f091c00a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.246+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f091c00a5d0 con 0x7f091c009c80 2024-04-25T12:36:34.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.247+0000 7f093fe00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f091c009c80 0x7f091c00a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.247+0000 7f093fe00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f091c009c80 0x7f091c00a090 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.249+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c00a5d0 con 0x7f091c009c80 2024-04-25T12:36:34.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.278+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c00aae0 con 0x7f091c009c80 2024-04-25T12:36:34.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.281+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c00aae0 con 0x7f091c009c80 2024-04-25T12:36:34.282 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:34.282 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:34.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.282+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f091c00aee0 0x7f091c00fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.282+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f091c012a70 con 0x7f091c00aee0 2024-04-25T12:36:34.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.284+0000 7f0945c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f091c00aee0 0x7f091c00fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.285+0000 7f0945c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f091c00aee0 0x7f091c00fa70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.286+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c012a70 con 0x7f091c00aee0 2024-04-25T12:36:34.294 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:34.294+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:34.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.304+0000 7f6bdd000640 1 Processor -- start 2024-04-25T12:36:34.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.304+0000 7f6bdd000640 1 -- start start 2024-04-25T12:36:34.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.307+0000 7f6bdd000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6bd80c8250 0x7f6bd815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.307+0000 7f6bdd000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6bd80cda90 0x7f6bd80cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.307+0000 7f6bdd000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd80c2b80 con 0x7f6bd8074690 2024-04-25T12:36:34.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.307+0000 7f6bdd000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd80c4400 con 0x7f6bd80cda90 2024-04-25T12:36:34.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.307+0000 7f6bdd000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd80c46e0 con 0x7f6bd80c8250 2024-04-25T12:36:34.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.308+0000 7f6bd5400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6bd80c8250 0x7f6bd815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.308+0000 7f6bd5400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6bd80c8250 0x7f6bd815ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:58870/0 (socket says 172.21.15.123:58870) 2024-04-25T12:36:34.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.308+0000 7f6bd5400640 1 -- 172.21.15.123:0/59602639 learned_addr learned my addr 172.21.15.123:0/59602639 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:34.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.308+0000 7f6bd5400640 1 -- 172.21.15.123:0/59602639 >> v2:172.21.15.123:3300/0 conn(0x7f6bd80cda90 msgr2=0x7f6bd80cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:34.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.308+0000 7f6bd5400640 1 --2- 172.21.15.123:0/59602639 >> v2:172.21.15.123:3300/0 conn(0x7f6bd80cda90 0x7f6bd80cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.308+0000 7f6bd5400640 1 -- 172.21.15.123:0/59602639 >> v1:172.21.15.153:6789/0 conn(0x7f6bd8074690 legacy=0x7f6bd81566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.308+0000 7f6bd5400640 1 -- 172.21.15.123:0/59602639 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd815c610 con 0x7f6bd80c8250 2024-04-25T12:36:34.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.310+0000 7f6bd5400640 1 --2- 172.21.15.123:0/59602639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd815ce10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80ce540 tx=0x7f6bc402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92bfcc069104bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.310+0000 7f6bd4a00640 1 -- 172.21.15.123:0/59602639 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4037070 con 0x7f6bd80c8250 2024-04-25T12:36:34.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.311+0000 7f6bd4a00640 1 -- 172.21.15.123:0/59602639 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc402ae10 con 0x7f6bd80c8250 2024-04-25T12:36:34.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.311+0000 7f6bd4a00640 1 -- 172.21.15.123:0/59602639 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4032660 con 0x7f6bd80c8250 2024-04-25T12:36:34.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 -- 172.21.15.123:0/59602639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 msgr2=0x7f6bd815ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 --2- 172.21.15.123:0/59602639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd815ce10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80ce540 tx=0x7f6bc402a180 comp rx=0 tx=0).stop 2024-04-25T12:36:34.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 -- 172.21.15.123:0/59602639 shutdown_connections 2024-04-25T12:36:34.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 --2- 172.21.15.123:0/59602639 >> v2:172.21.15.123:3300/0 conn(0x7f6bd80cda90 0x7f6bd80cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 --2- 172.21.15.123:0/59602639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd815ce10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 -- 172.21.15.123:0/59602639 >> 172.21.15.123:0/59602639 conn(0x7f6bd806c750 msgr2=0x7f6bd806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:34.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 -- 172.21.15.123:0/59602639 shutdown_connections 2024-04-25T12:36:34.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.312+0000 7f6bdd000640 1 -- 172.21.15.123:0/59602639 wait complete. 2024-04-25T12:36:34.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.315+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c0119e0 con 0x7f091c00aee0 2024-04-25T12:36:34.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.318+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c0119e0 con 0x7f091c00aee0 2024-04-25T12:36:34.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.319+0000 7f6bdd000640 1 Processor -- start 2024-04-25T12:36:34.319 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:34.319 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:34.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.319+0000 7f6bdd000640 1 -- start start 2024-04-25T12:36:34.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.319+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f091c012100 0x7f091c010050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.319+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f091c0126c0 con 0x7f091c012100 2024-04-25T12:36:34.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.319+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f091c012100 0x7f091c010050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.320+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f091c012100 0x7f091c010050 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.321+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c0126c0 con 0x7f091c012100 2024-04-25T12:36:34.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.321+0000 7f6bdd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd8074690 0x7f6bd8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.321+0000 7f6bdd000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.321+0000 7f6bdd000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bd80cda90 0x7f6bd814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.321+0000 7f6bdd000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd8074aa0 con 0x7f6bd8074690 2024-04-25T12:36:34.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.321+0000 7f6bdd000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80c49c0 con 0x7f6bd80c8250 2024-04-25T12:36:34.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.321+0000 7f6bdd000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd8074c20 con 0x7f6bd80cda90 2024-04-25T12:36:34.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd8150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:58882/0 (socket says 172.21.15.123:58882) 2024-04-25T12:36:34.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd5400640 1 -- 172.21.15.123:0/4002401634 learned_addr learned my addr 172.21.15.123:0/4002401634 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:34.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd5e00640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd8074690 0x7f6bd8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd6800640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bd80cda90 0x7f6bd814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd5400640 1 -- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bd80cda90 msgr2=0x7f6bd814f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd5400640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bd80cda90 0x7f6bd814f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.322+0000 7f6bd5400640 1 -- 172.21.15.123:0/4002401634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd8074690 msgr2=0x7f6bd8154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.323+0000 7f6bd5400640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd8074690 0x7f6bd8154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.323+0000 7f6bd5400640 1 -- 172.21.15.123:0/4002401634 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd81e9780 con 0x7f6bd80c8250 2024-04-25T12:36:34.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.323+0000 7f6bd5e00640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd8074690 0x7f6bd8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:34.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.323+0000 7f6bd5400640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd8150d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4000c00 tx=0x7f6bc40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.324+0000 7f6bce000640 1 -- 172.21.15.123:0/4002401634 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4042070 con 0x7f6bd80c8250 2024-04-25T12:36:34.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.324+0000 7f6bce000640 1 -- 172.21.15.123:0/4002401634 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc403e670 con 0x7f6bd80c8250 2024-04-25T12:36:34.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.324+0000 7f6bce000640 1 -- 172.21.15.123:0/4002401634 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4037040 con 0x7f6bd80c8250 2024-04-25T12:36:34.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.324+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd81543e0 con 0x7f6bd80c8250 2024-04-25T12:36:34.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.324+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd81e8f80 con 0x7f6bd80c8250 2024-04-25T12:36:34.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.326+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd80b3bf0 con 0x7f6bd80c8250 2024-04-25T12:36:34.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.329+0000 7f6bce000640 1 -- 172.21.15.123:0/4002401634 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6bc4045070 con 0x7f6bd80c8250 2024-04-25T12:36:34.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.330+0000 7f6bce000640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6bb80744b0 0x7f6bb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.330+0000 7f6bce000640 1 -- 172.21.15.123:0/4002401634 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6685+0+0 (secure 0 0 0) 0x7f6bc40b6680 con 0x7f6bd80c8250 2024-04-25T12:36:34.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.331+0000 7f6bd5e00640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6bb80744b0 0x7f6bb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.332+0000 7f6bd5e00640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6bb80744b0 0x7f6bb80769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001cb0 tx=0x7f6bc8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.332+0000 7f6bce000640 1 -- 172.21.15.123:0/4002401634 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6bd80b3bf0 con 0x7f6bd80c8250 2024-04-25T12:36:34.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.350+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c0126c0 con 0x7f091c012100 2024-04-25T12:36:34.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.352+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c0126c0 con 0x7f091c012100 2024-04-25T12:36:34.352 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:34.353 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:34.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.353+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f091c010aa0 0x7f091c010ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.353+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f091c0195b0 con 0x7f091c010aa0 2024-04-25T12:36:34.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.353+0000 7f093fe00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f091c010aa0 0x7f091c010ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.354+0000 7f093fe00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f091c010aa0 0x7f091c010ed0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.355+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c0195b0 con 0x7f091c010aa0 2024-04-25T12:36:34.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.384+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c014cc0 con 0x7f091c010aa0 2024-04-25T12:36:34.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.386+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c014cc0 con 0x7f091c010aa0 2024-04-25T12:36:34.386 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:34.386 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:34.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.386+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f091c0151d0 0x7f091c0155e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.386+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f091c015ba0 con 0x7f091c0151d0 2024-04-25T12:36:34.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.387+0000 7f0945c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f091c0151d0 0x7f091c0155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.387+0000 7f0945c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f091c0151d0 0x7f091c0155e0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.388+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c015ba0 con 0x7f091c0151d0 2024-04-25T12:36:34.394 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:34.395+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.419+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c0008d0 con 0x7f091c0151d0 2024-04-25T12:36:34.421 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.421+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c0008d0 con 0x7f091c0151d0 2024-04-25T12:36:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.422+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f091c01a950 0x7f091c01cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.422+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f091c01fbc0 con 0x7f091c01a950 2024-04-25T12:36:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.422+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f091c01a950 0x7f091c01cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.423+0000 7f0945200640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f091c01a950 0x7f091c01cd70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:34.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.424+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f091c01fbc0 con 0x7f091c01a950 2024-04-25T12:36:34.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.440+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f091c01d330 con 0x7f091c01a950 2024-04-25T12:36:34.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f093cc00640 1 -- 172.21.15.123:0/1944381479 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f091c01d330 con 0x7f091c01a950 2024-04-25T12:36:34.443 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f091c0045e0 msgr2=0x7f091c0049f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f091c0045e0 0x7f091c0049f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f091c001e90 msgr2=0x7f091c002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f091c001e90 0x7f091c002990 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f091c009c80 msgr2=0x7f091c00a090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f091c009c80 0x7f091c00a090 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f091c00aee0 msgr2=0x7f091c00fa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f091c00aee0 0x7f091c00fa70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f091c012100 msgr2=0x7f091c010050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f091c012100 0x7f091c010050 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f091c010aa0 msgr2=0x7f091c010ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f091c010aa0 0x7f091c010ed0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f091c0151d0 msgr2=0x7f091c0155e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f091c0151d0 0x7f091c0155e0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f091c01a950 msgr2=0x7f091c01cd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f091c01a950 0x7f091c01cd70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0945c00640 1 -- 172.21.15.123:0/1944381479 reap_dead start 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.443+0000 7f0945c00640 1 -- 172.21.15.123:0/1944381479 reap_dead start 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0945c00640 1 -- 172.21.15.123:0/1944381479 reap_dead start 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0945c00640 1 -- 172.21.15.123:0/1944381479 reap_dead start 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0945c00640 1 -- 172.21.15.123:0/1944381479 reap_dead start 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09200744e0 msgr2=0x7f09200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09200744e0 0x7f09200769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f09340015e0 tx=0x7f0934009040 comp rx=0 tx=0).stop 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f094015aca0 msgr2=0x7f09401fa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f094015aca0 0x7f09401fa960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f092c000c00 tx=0x7f092c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 shutdown_connections 2024-04-25T12:36:34.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09200744e0 0x7f09200769f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.444+0000 7f0947c00640 1 --2- 172.21.15.123:0/1944381479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f094015aca0 0x7f09401fa960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.445+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 >> 172.21.15.123:0/1944381479 conn(0x7f094006f440 msgr2=0x7f0940160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:34.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.445+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 shutdown_connections 2024-04-25T12:36:34.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.445+0000 7f0947c00640 1 -- 172.21.15.123:0/1944381479 wait complete. 2024-04-25T12:36:34.457 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:34.458 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:34.494 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:34.496 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:34.496+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:34.597 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:34.598+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:34.619 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:34.619 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:34.629 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:34.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.643+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f6bd806a260 con 0x7f6bd80c8250 2024-04-25T12:36:34.697 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:34.698+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:34.741 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:34.741 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:34.750 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:34.837 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:34.838+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:34.900 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:34.901+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:34.904 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:34.904 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:34.913 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:34.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.946+0000 7f6bce000640 1 -- 172.21.15.123:0/4002401634 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v62) ==== 101+0+0 (secure 0 0 0) 0x7f6bd806a260 con 0x7f6bd80c8250 2024-04-25T12:36:34.945 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:36:34.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.949+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6bb80744b0 msgr2=0x7f6bb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.949+0000 7f6bdd000640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6bb80744b0 0x7f6bb80769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001cb0 tx=0x7f6bc8009040 comp rx=0 tx=0).stop 2024-04-25T12:36:34.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.949+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 msgr2=0x7f6bd8150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:34.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.949+0000 7f6bdd000640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd8150d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4000c00 tx=0x7f6bc40046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 shutdown_connections 2024-04-25T12:36:34.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6bb80744b0 0x7f6bb80769c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6bd80cda90 0x7f6bd814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bd80c8250 0x7f6bd8150d40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 --2- 172.21.15.123:0/4002401634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd8074690 0x7f6bd8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:34.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 >> 172.21.15.123:0/4002401634 conn(0x7f6bd806c750 msgr2=0x7f6bd80c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:34.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 shutdown_connections 2024-04-25T12:36:34.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:34.950+0000 7f6bdd000640 1 -- 172.21.15.123:0/4002401634 wait complete. 2024-04-25T12:36:35.000 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:35.001+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:35.067 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:35.067 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:35.076 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:35.117 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:35.118+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:35.202 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:35.203+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:35.228 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:35.229 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:35.238 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:35.323 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:35.324+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:35.390 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:35.390 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:35.399 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:35.440 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:35.441+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:35.504 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:35.505+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:35.552 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:35.552 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:35.562 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:35.606 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:35.607+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:35.674 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:35.674 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:35.684 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:35.706 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:35.707+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:35.807 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:35.808+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:35.835 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:35.835 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:35.844 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:35.929 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:35.930+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:35.964 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:35.964 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:35.964 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-25T12:36:35.964 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-25T12:36:35.997 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:35.997 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:36.006 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:36.047 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:36.048+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:36.109 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:36.110+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:36.158 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:36.158 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:36.167 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:36.210 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:36.211+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:36.310 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:36.311+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:36.319 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:36.320 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:36.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:36.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.344+0000 7f4b9b200640 1 Processor -- start 2024-04-25T12:36:36.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.344+0000 7f4b9b200640 1 -- start start 2024-04-25T12:36:36.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b9b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c1568c0 0x7f4b9c0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b9b200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4b9c0cdca0 0x7f4b9c0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b9b200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9c0cedf0 con 0x7f4b9c0c6800 2024-04-25T12:36:36.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b9b200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4b9c0cebd0 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b9b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9c0cf0d0 con 0x7f4b9c1568c0 2024-04-25T12:36:36.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b99e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c1568c0 0x7f4b9c0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b99e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c1568c0 0x7f4b9c0ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:45998/0 (socket says 172.21.15.123:45998) 2024-04-25T12:36:36.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b99e00640 1 -- 172.21.15.123:0/1868529506 learned_addr learned my addr 172.21.15.123:0/1868529506 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:36.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b99400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4b9c0cdca0 0x7f4b9c0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.347+0000 7f4b9a800640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4b9c0c6800 0x7f4b9c15c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:36136/0 (socket says 172.21.15.123:36136) 2024-04-25T12:36:36.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.348+0000 7f4b99e00640 1 -- 172.21.15.123:0/1868529506 >> v2:172.21.15.153:3300/0 conn(0x7f4b9c0cdca0 msgr2=0x7f4b9c0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:36.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.348+0000 7f4b99e00640 1 --2- 172.21.15.123:0/1868529506 >> v2:172.21.15.153:3300/0 conn(0x7f4b9c0cdca0 0x7f4b9c0ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.348+0000 7f4b99e00640 1 -- 172.21.15.123:0/1868529506 >> v1:172.21.15.153:6789/0 conn(0x7f4b9c0c6800 legacy=0x7f4b9c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.348+0000 7f4b99e00640 1 -- 172.21.15.123:0/1868529506 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c0c7df0 con 0x7f4b9c1568c0 2024-04-25T12:36:36.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.348+0000 7f4b99400640 1 --2- 172.21.15.123:0/1868529506 >> v2:172.21.15.153:3300/0 conn(0x7f4b9c0cdca0 0x7f4b9c0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:36.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.349+0000 7f4b99e00640 1 --2- 172.21.15.123:0/1868529506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c1568c0 0x7f4b9c0ce690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009870 tx=0x7f4b8c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30fe2c5aba2bd736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.350+0000 7f4b98a00640 1 -- 172.21.15.123:0/1868529506 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c037070 con 0x7f4b9c1568c0 2024-04-25T12:36:36.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.350+0000 7f4b98a00640 1 -- 172.21.15.123:0/1868529506 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8c032440 con 0x7f4b9c1568c0 2024-04-25T12:36:36.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.350+0000 7f4b98a00640 1 -- 172.21.15.123:0/1868529506 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c032810 con 0x7f4b9c1568c0 2024-04-25T12:36:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 -- 172.21.15.123:0/1868529506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c1568c0 msgr2=0x7f4b9c0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 --2- 172.21.15.123:0/1868529506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c1568c0 0x7f4b9c0ce690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009870 tx=0x7f4b8c02a250 comp rx=0 tx=0).stop 2024-04-25T12:36:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 -- 172.21.15.123:0/1868529506 shutdown_connections 2024-04-25T12:36:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 --2- 172.21.15.123:0/1868529506 >> v2:172.21.15.153:3300/0 conn(0x7f4b9c0cdca0 0x7f4b9c0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 --2- 172.21.15.123:0/1868529506 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c1568c0 0x7f4b9c0ce690 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 -- 172.21.15.123:0/1868529506 >> 172.21.15.123:0/1868529506 conn(0x7f4b9c06dcd0 msgr2=0x7f4b9c06e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 -- 172.21.15.123:0/1868529506 shutdown_connections 2024-04-25T12:36:36.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.351+0000 7f4b9b200640 1 -- 172.21.15.123:0/1868529506 wait complete. 2024-04-25T12:36:36.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.355+0000 7f4b9b200640 1 Processor -- start 2024-04-25T12:36:36.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.355+0000 7f4b9b200640 1 -- start start 2024-04-25T12:36:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.357+0000 7f4b9b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9c0c6800 0x7f4b9c157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.357+0000 7f4b9b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c0cdca0 0x7f4b9c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.357+0000 7f4b9b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b9c14ec40 0x7f4b9c14f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.357+0000 7f4b9b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c0c83b0 con 0x7f4b9c0c6800 2024-04-25T12:36:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.357+0000 7f4b9b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c0c8720 con 0x7f4b9c14ec40 2024-04-25T12:36:36.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.357+0000 7f4b9b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9c15cb00 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.358+0000 7f4b99400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c0cdca0 0x7f4b9c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.358+0000 7f4b99400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c0cdca0 0x7f4b9c0d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:46012/0 (socket says 172.21.15.123:46012) 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.358+0000 7f4b99400640 1 -- 172.21.15.123:0/1886401908 learned_addr learned my addr 172.21.15.123:0/1886401908 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.358+0000 7f4b99e00640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9c0c6800 0x7f4b9c157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.358+0000 7f4b9a800640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b9c14ec40 0x7f4b9c14f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.358+0000 7f4b99400640 1 -- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b9c14ec40 msgr2=0x7f4b9c14f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b99400640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b9c14ec40 0x7f4b9c14f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b99400640 1 -- 172.21.15.123:0/1886401908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9c0c6800 msgr2=0x7f4b9c157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b99400640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9c0c6800 0x7f4b9c157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b99400640 1 -- 172.21.15.123:0/1886401908 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c1f2bb0 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b9a800640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b9c14ec40 0x7f4b9c14f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b99400640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4b9c0cdca0 0x7f4b9c0d00f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800e900 tx=0x7f4b8800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88017070 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8800ca00 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8800cdf0 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.359+0000 7f4b9b200640 1 -- 172.21.15.123:0/1886401908 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c1582f0 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.360+0000 7f4b9b200640 1 -- 172.21.15.123:0/1886401908 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c15c470 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.361+0000 7f4b9b200640 1 -- 172.21.15.123:0/1886401908 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b9c076da0 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.363+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b8801a070 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.365+0000 7f4b92000640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b7c0744b0 0x7f4b7c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.365+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 6685+0+0 (secure 0 0 0) 0x7f4b880955b0 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.365+0000 7f4b92000640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4b7c07de10 0x7f4b7c080300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.365+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b8801ee50 con 0x7f4b7c07de10 2024-04-25T12:36:36.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.365+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f4b8809c050 con 0x7f4b9c0cdca0 2024-04-25T12:36:36.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.365+0000 7f4b99e00640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b7c0744b0 0x7f4b7c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.366+0000 7f4b9a800640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4b7c07de10 0x7f4b7c080300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.367+0000 7f4b9a800640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f4b7c07de10 0x7f4b7c080300 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.367+0000 7f4b99e00640 1 --2- 172.21.15.123:0/1886401908 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4b7c0744b0 0x7f4b7c0769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c034000 tx=0x7f4b8c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.368+0000 7f4b92000640 1 -- 172.21.15.123:0/1886401908 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8801ee50 con 0x7f4b7c07de10 2024-04-25T12:36:36.411 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:36.412+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:36.421 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:36.421 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:36.421 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:36.421 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:36.421 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:36.421 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:36.422 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:36.422 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:36.422 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:36.422 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:36.422 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:36.422 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:36.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:36.444 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:36.445 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-25T12:36:36.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:36.538 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:36.538+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:36.613 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:36.614+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:36.715 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:36.716+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:36.728 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:36.728 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:36.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:36.816 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:36.817+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:36.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.848+0000 7f2681400640 1 Processor -- start 2024-04-25T12:36:36.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.848+0000 7f2681400640 1 -- start start 2024-04-25T12:36:36.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.851+0000 7f2681400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c0c2720 0x7f267c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.851+0000 7f2681400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f267c0c8cf0 0x7f267c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.851+0000 7f2681400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267c0c40f0 con 0x7f267c156990 2024-04-25T12:36:36.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.851+0000 7f2681400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0c85f0 con 0x7f267c0c8cf0 2024-04-25T12:36:36.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.851+0000 7f2681400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f267c074610 con 0x7f267c0c2720 2024-04-25T12:36:36.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.851+0000 7f267be00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f267c156990 0x7f267c06eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:36150/0 (socket says 172.21.15.123:36150) 2024-04-25T12:36:36.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.851+0000 7f267be00640 1 -- 172.21.15.123:0/3986388399 learned_addr learned my addr 172.21.15.123:0/3986388399 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:36.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f2680a00640 1 --2- 172.21.15.123:0/3986388399 >> v2:172.21.15.153:3300/0 conn(0x7f267c0c8cf0 0x7f267c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f267b400640 1 --2- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c0c2720 0x7f267c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f267aa00640 1 -- 172.21.15.123:0/3986388399 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156941726 0 0) 0x7f267c0c40f0 con 0x7f267c156990 2024-04-25T12:36:36.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f267aa00640 1 -- 172.21.15.123:0/3986388399 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2660003520 con 0x7f267c156990 2024-04-25T12:36:36.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f2680a00640 1 -- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c0c2720 msgr2=0x7f267c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:36.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f2680a00640 1 --2- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c0c2720 0x7f267c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f2680a00640 1 -- 172.21.15.123:0/3986388399 >> v1:172.21.15.153:6789/0 conn(0x7f267c156990 legacy=0x7f267c06eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.852+0000 7f2680a00640 1 -- 172.21.15.123:0/3986388399 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0c4360 con 0x7f267c0c8cf0 2024-04-25T12:36:36.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.853+0000 7f267b400640 1 --2- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c0c2720 0x7f267c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:36.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.854+0000 7f2680a00640 1 --2- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 0x7f267c0c9100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2674009d90 tx=0x7f267402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9734ed4bcabce29d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.854+0000 7f267aa00640 1 -- 172.21.15.123:0/3986388399 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674007740 con 0x7f267c0c8cf0 2024-04-25T12:36:36.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.855+0000 7f267aa00640 1 -- 172.21.15.123:0/3986388399 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26740078e0 con 0x7f267c0c8cf0 2024-04-25T12:36:36.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.855+0000 7f267aa00640 1 -- 172.21.15.123:0/3986388399 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674007c90 con 0x7f267c0c8cf0 2024-04-25T12:36:36.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.856+0000 7f2681400640 1 -- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 msgr2=0x7f267c0c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.856+0000 7f2681400640 1 --2- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 0x7f267c0c9100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2674009d90 tx=0x7f267402a6f0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.857+0000 7f2681400640 1 -- 172.21.15.123:0/3986388399 shutdown_connections 2024-04-25T12:36:36.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.857+0000 7f2681400640 1 --2- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 0x7f267c0c9100 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.857+0000 7f2681400640 1 --2- 172.21.15.123:0/3986388399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c0c2720 0x7f267c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.857+0000 7f2681400640 1 -- 172.21.15.123:0/3986388399 >> 172.21.15.123:0/3986388399 conn(0x7f267c0ca170 msgr2=0x7f267c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:36.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.857+0000 7f2681400640 1 -- 172.21.15.123:0/3986388399 shutdown_connections 2024-04-25T12:36:36.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.857+0000 7f2681400640 1 -- 172.21.15.123:0/3986388399 wait complete. 2024-04-25T12:36:36.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.858+0000 7f8f2da00640 1 Processor -- start 2024-04-25T12:36:36.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.858+0000 7f8f2da00640 1 -- start start 2024-04-25T12:36:36.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.861+0000 7f2681400640 1 Processor -- start 2024-04-25T12:36:36.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.861+0000 7f8f2da00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8f280c93f0 0x7f8f280c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.861+0000 7f8f2da00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f280c3be0 con 0x7f8f280c9800 2024-04-25T12:36:36.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.861+0000 7f8f2da00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f280c3ec0 con 0x7f8f281537a0 2024-04-25T12:36:36.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.861+0000 7f8f2da00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8f280cb3a0 con 0x7f8f280c93f0 2024-04-25T12:36:36.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.861+0000 7f2681400640 1 -- start start 2024-04-25T12:36:36.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.862+0000 7f8f26a00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8f281537a0 0x7f8f280cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:36160/0 (socket says 172.21.15.123:36160) 2024-04-25T12:36:36.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.862+0000 7f8f26a00640 1 -- 172.21.15.123:0/2879331648 learned_addr learned my addr 172.21.15.123:0/2879331648 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:36.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.862+0000 7f8f26000640 1 --2- 172.21.15.123:0/2879331648 >> v2:172.21.15.153:3300/0 conn(0x7f8f280c93f0 0x7f8f280c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.863+0000 7f8f25600640 1 -- 172.21.15.123:0/2879331648 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789207426 0 0) 0x7f8f280c3ec0 con 0x7f8f281537a0 2024-04-25T12:36:36.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.863+0000 7f8f25600640 1 -- 172.21.15.123:0/2879331648 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f0c003520 con 0x7f8f281537a0 2024-04-25T12:36:36.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.863+0000 7f8f26000640 1 -- 172.21.15.123:0/2879331648 >> v1:172.21.15.153:6789/0 conn(0x7f8f281537a0 legacy=0x7f8f280cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.863+0000 7f8f26000640 1 -- 172.21.15.123:0/2879331648 >> v1:172.21.15.123:6789/0 conn(0x7f8f280c9800 legacy=0x7f8f280c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:36.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.863+0000 7f8f26000640 1 -- 172.21.15.123:0/2879331648 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2815ca40 con 0x7f8f280c93f0 2024-04-25T12:36:36.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.864+0000 7f2681400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f267c0c2720 0x7f267c1f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.864+0000 7f2681400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 0x7f267c14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.864+0000 7f2681400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c156990 0x7f267c14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.864+0000 7f2681400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0c9640 con 0x7f267c0c2720 2024-04-25T12:36:36.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.864+0000 7f2681400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0c8280 con 0x7f267c0c8cf0 2024-04-25T12:36:36.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.864+0000 7f2681400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f267c06f260 con 0x7f267c156990 2024-04-25T12:36:36.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.864+0000 7f8f26000640 1 --2- 172.21.15.123:0/2879331648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 0x7f8f280c41d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800fa10 tx=0x7f8f180329b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e724b2d49826489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f2680a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c156990 0x7f267c14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f2680a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c156990 0x7f267c14be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:46024/0 (socket says 172.21.15.123:46024) 2024-04-25T12:36:36.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 learned_addr learned my addr 172.21.15.123:0/712321231 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:36.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f8f25600640 1 -- 172.21.15.123:0/2879331648 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800e4e0 con 0x7f8f280c93f0 2024-04-25T12:36:36.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f267b400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 0x7f267c14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f8f25600640 1 -- 172.21.15.123:0/2879331648 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1803d070 con 0x7f8f280c93f0 2024-04-25T12:36:36.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f8f25600640 1 -- 172.21.15.123:0/2879331648 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800e8e0 con 0x7f8f280c93f0 2024-04-25T12:36:36.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 msgr2=0x7f267c14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f2680a00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f267c0c8cf0 0x7f267c14eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f267c0c2720 msgr2=0x7f267c1f25c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:36:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.865+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f267c0c2720 0x7f267c1f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f2680a00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f267c0c2720 0x7f267c1f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c15c720 con 0x7f267c156990 2024-04-25T12:36:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f8f2da00640 1 -- 172.21.15.123:0/2879331648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 msgr2=0x7f8f280c41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f8f2da00640 1 --2- 172.21.15.123:0/2879331648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 0x7f8f280c41d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800fa10 tx=0x7f8f180329b0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f2680a00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c156990 0x7f267c14be90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2674009eb0 tx=0x7f2674004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f8f2da00640 1 -- 172.21.15.123:0/2879331648 shutdown_connections 2024-04-25T12:36:36.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f8f2da00640 1 --2- 172.21.15.123:0/2879331648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 0x7f8f280c41d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.866+0000 7f8f2da00640 1 -- 172.21.15.123:0/2879331648 >> 172.21.15.123:0/2879331648 conn(0x7f8f28071b90 msgr2=0x7f8f28071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:36.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.867+0000 7f8f2da00640 1 -- 172.21.15.123:0/2879331648 shutdown_connections 2024-04-25T12:36:36.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.867+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674009450 con 0x7f267c156990 2024-04-25T12:36:36.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.867+0000 7f8f2da00640 1 -- 172.21.15.123:0/2879331648 wait complete. 2024-04-25T12:36:36.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.867+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674004040 con 0x7f267c156990 2024-04-25T12:36:36.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.867+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674004430 con 0x7f267c156990 2024-04-25T12:36:36.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.867+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c0c5770 con 0x7f267c156990 2024-04-25T12:36:36.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.867+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c0c5d30 con 0x7f267c156990 2024-04-25T12:36:36.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.870+0000 7f8f2da00640 1 Processor -- start 2024-04-25T12:36:36.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.869+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f267c068eb0 con 0x7f267c156990 2024-04-25T12:36:36.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.870+0000 7f8f2da00640 1 -- start start 2024-04-25T12:36:36.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.870+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2674045020 con 0x7f267c156990 2024-04-25T12:36:36.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f2da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 0x7f8f281eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f2da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f280c9800 0x7f8f28157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f2da00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f281537a0 0x7f8f280c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f2da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2815d2e0 con 0x7f8f280c9800 2024-04-25T12:36:36.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f2da00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2815d000 con 0x7f8f280c93f0 2024-04-25T12:36:36.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f2da00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8f280c3880 con 0x7f8f281537a0 2024-04-25T12:36:36.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f26a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 0x7f8f281eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f26a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 0x7f8f281eeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:58936/0 (socket says 172.21.15.123:58936) 2024-04-25T12:36:36.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f8f26a00640 1 -- 172.21.15.123:0/1226158857 learned_addr learned my addr 172.21.15.123:0/1226158857 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:36.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f8f27400640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f281537a0 0x7f8f280c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f2673e00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f265c0744b0 0x7f265c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f8f26000640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f280c9800 0x7f8f28157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.872+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 6685+0+0 (secure 0 0 0) 0x7f26740b5e50 con 0x7f267c156990 2024-04-25T12:36:36.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f8f26a00640 1 -- 172.21.15.123:0/1226158857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f281537a0 msgr2=0x7f8f280c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f8f26a00640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f281537a0 0x7f8f280c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f8f26a00640 1 -- 172.21.15.123:0/1226158857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f280c9800 msgr2=0x7f8f28157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:36.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f8f26a00640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f280c9800 0x7f8f28157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:36.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f8f26a00640 1 -- 172.21.15.123:0/1226158857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f281ef2e0 con 0x7f8f280c93f0 2024-04-25T12:36:36.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.873+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f265c0744b0 0x7f265c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f8f26a00640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f280c93f0 0x7f8f281eeda0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c440 tx=0x7f8f1400c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v62) ==== 54+0+16 (secure 0 0 0) 0x7f267c068eb0 con 0x7f267c156990 2024-04-25T12:36:36.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14017070 con 0x7f8f280c93f0 2024-04-25T12:36:36.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14005ce0 con 0x7f8f280c93f0 2024-04-25T12:36:36.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f140072b0 con 0x7f8f280c93f0 2024-04-25T12:36:36.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f265c0744b0 0x7f265c0769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f266c000f60 tx=0x7f266c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f8f2da00640 1 -- 172.21.15.123:0/1226158857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f281f1d60 con 0x7f8f280c93f0 2024-04-25T12:36:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.874+0000 7f8f2da00640 1 -- 172.21.15.123:0/1226158857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281f25e0 con 0x7f8f280c93f0 2024-04-25T12:36:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.875+0000 7f2681400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f267c0c5f30 0x7f267c14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.875+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f267c076da0 con 0x7f267c0c5f30 2024-04-25T12:36:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.876+0000 7f8f2da00640 1 -- 172.21.15.123:0/1226158857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f28076700 con 0x7f8f280c93f0 2024-04-25T12:36:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.876+0000 7f267b400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f267c0c5f30 0x7f267c14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.877+0000 7f267b400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f267c0c5f30 0x7f267c14c3d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.877+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f1401a070 con 0x7f8f280c93f0 2024-04-25T12:36:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.879+0000 7f8f1ea00640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f08074530 0x7f8f08076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.879+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f267c076da0 con 0x7f267c0c5f30 2024-04-25T12:36:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.879+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6685+0+0 (secure 0 0 0) 0x7f8f14094910 con 0x7f8f280c93f0 2024-04-25T12:36:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.879+0000 7f8f1ea00640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8f0807df10 0x7f8f08080400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.879+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f08080ad0 con 0x7f8f0807df10 2024-04-25T12:36:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.879+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8f140992a0 con 0x7f8f280c93f0 2024-04-25T12:36:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.879+0000 7f8f26000640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f08074530 0x7f8f08076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.880+0000 7f8f27400640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8f0807df10 0x7f8f08080400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.881+0000 7f8f26000640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f08074530 0x7f8f08076a40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f1803c000 tx=0x7f8f18008070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.882+0000 7f8f27400640 1 --2- 172.21.15.123:0/1226158857 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8f0807df10 0x7f8f08080400 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.883+0000 7f8f1ea00640 1 -- 172.21.15.123:0/1226158857 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f08080ad0 con 0x7f8f0807df10 2024-04-25T12:36:36.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.910+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f267c074870 con 0x7f267c0c5f30 2024-04-25T12:36:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.913+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f267c074870 con 0x7f267c0c5f30 2024-04-25T12:36:36.913 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:36.914 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:36.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.914+0000 7f2681400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f267c0c52b0 0x7f267c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.914+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f267c0c9230 con 0x7f267c0c52b0 2024-04-25T12:36:36.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.915+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f267c0c52b0 0x7f267c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.916+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f267c0c52b0 0x7f267c0c66e0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.916 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:36.917+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:36.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.917+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f267c0c9230 con 0x7f267c0c52b0 2024-04-25T12:36:36.936 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:36.936 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:36.937 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:36.937 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:36.937 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:36.937 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:36.938 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:36.938 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:36.938 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:36.938 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:36.939 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:36.939 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:36.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.951+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f267c0c49f0 con 0x7f267c0c52b0 2024-04-25T12:36:36.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.955+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f267c0c49f0 con 0x7f267c0c52b0 2024-04-25T12:36:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:36.956 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:36.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.956+0000 7f2681400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f267c0c7130 0x7f267c062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.956+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f267c0c7540 con 0x7f267c0c7130 2024-04-25T12:36:36.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.956+0000 7f267b400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f267c0c7130 0x7f267c062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.957+0000 7f267b400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f267c0c7130 0x7f267c062fe0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.959+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f267c0c7540 con 0x7f267c0c7130 2024-04-25T12:36:36.962 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:36.963 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:36.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-25T12:36:36.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.986+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f267c0aed30 con 0x7f267c0c7130 2024-04-25T12:36:36.990 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:36.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.989+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f267c0aed30 con 0x7f267c0c7130 2024-04-25T12:36:36.991 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:36.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.990+0000 7f2681400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f267c14cc80 0x7f267c14d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:36.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.990+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f267c0aef40 con 0x7f267c14cc80 2024-04-25T12:36:36.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.990+0000 7f2680a00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f267c14cc80 0x7f267c14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:36.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.991+0000 7f2680a00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f267c14cc80 0x7f267c14d090 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:36.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:36.992+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f267c0aef40 con 0x7f267c14cc80 2024-04-25T12:36:36.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:36:37.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.025+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f267c14d5d0 con 0x7f267c14cc80 2024-04-25T12:36:37.027 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:37.028+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:37.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.028+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f267c14d5d0 con 0x7f267c14cc80 2024-04-25T12:36:37.028 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:37.028 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:37.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.029+0000 7f2681400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f267c02c9d0 0x7f267c02cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.029+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f267c02d320 con 0x7f267c02c9d0 2024-04-25T12:36:37.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.029+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f267c02c9d0 0x7f267c02cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.030+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f267c02c9d0 0x7f267c02cde0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.033+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f267c02d320 con 0x7f267c02c9d0 2024-04-25T12:36:37.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.062+0000 7f2681400640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f267c02d320 con 0x7f267c02c9d0 2024-04-25T12:36:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.065+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f267c02d320 con 0x7f267c02c9d0 2024-04-25T12:36:37.065 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:37.066 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:37.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.066+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2650001640 0x7f2650003b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.066+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f2650006bf0 con 0x7f2650001640 2024-04-25T12:36:37.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.067+0000 7f267b400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2650001640 0x7f2650003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.067+0000 7f267b400640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2650001640 0x7f2650003b50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.068+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2650006bf0 con 0x7f2650001640 2024-04-25T12:36:37.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.097+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2650005910 con 0x7f2650001640 2024-04-25T12:36:37.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.100+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2650005910 con 0x7f2650001640 2024-04-25T12:36:37.101 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:37.102 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:37.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.103+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2650004850 0x7f2650004d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.103+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f26500066c0 con 0x7f2650004850 2024-04-25T12:36:37.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.103+0000 7f2680a00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2650004850 0x7f2650004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.106+0000 7f2680a00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2650004850 0x7f2650004d30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.107+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26500066c0 con 0x7f2650004850 2024-04-25T12:36:37.118 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:37.119+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:37.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.139+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2650000bb0 con 0x7f2650004850 2024-04-25T12:36:37.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.142+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2650000bb0 con 0x7f2650004850 2024-04-25T12:36:37.141 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:37.142 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:37.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.142+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f26500058c0 0x7f2650005cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.142+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f265000d590 con 0x7f26500058c0 2024-04-25T12:36:37.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.143+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f26500058c0 0x7f2650005cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.144+0000 7f267be00640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f26500058c0 0x7f2650005cd0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.145+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f265000d590 con 0x7f26500058c0 2024-04-25T12:36:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.175+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f265000ad00 con 0x7f26500058c0 2024-04-25T12:36:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.178+0000 7f2673e00640 1 -- 172.21.15.123:0/712321231 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f265000ad00 con 0x7f26500058c0 2024-04-25T12:36:37.177 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:37.178 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:37.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f267c0c5f30 msgr2=0x7f267c14c3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f267c0c5f30 0x7f267c14c3d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f267c0c52b0 msgr2=0x7f267c0c66e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f267c0c52b0 0x7f267c0c66e0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f267c0c7130 msgr2=0x7f267c062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f267c0c7130 0x7f267c062fe0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f267c14cc80 msgr2=0x7f267c14d090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f267c14cc80 0x7f267c14d090 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f267c02c9d0 msgr2=0x7f267c02cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f267c02c9d0 0x7f267c02cde0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2650001640 msgr2=0x7f2650003b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f2650001640 0x7f2650003b50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2650004850 msgr2=0x7f2650004d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f2650004850 0x7f2650004d30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f26500058c0 msgr2=0x7f2650005cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f26500058c0 0x7f2650005cd0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 reap_dead start 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.179+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 reap_dead start 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 reap_dead start 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 reap_dead start 2024-04-25T12:36:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2680a00640 1 -- 172.21.15.123:0/712321231 reap_dead start 2024-04-25T12:36:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f265c0744b0 msgr2=0x7f265c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f265c0744b0 0x7f265c0769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f266c000f60 tx=0x7f266c009040 comp rx=0 tx=0).stop 2024-04-25T12:36:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c156990 msgr2=0x7f267c14be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c156990 0x7f267c14be90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2674009eb0 tx=0x7f2674004a40 comp rx=0 tx=0).stop 2024-04-25T12:36:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 shutdown_connections 2024-04-25T12:36:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f265c0744b0 0x7f265c0769c0 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f266c000f60 tx=0x7f266c009040 comp rx=0 tx=0).stop 2024-04-25T12:36:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 --2- 172.21.15.123:0/712321231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f267c156990 0x7f267c14be90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.180+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 >> 172.21.15.123:0/712321231 conn(0x7f267c0ca170 msgr2=0x7f267c154bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.181+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 shutdown_connections 2024-04-25T12:36:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.181+0000 7f2671600640 1 -- 172.21.15.123:0/712321231 wait complete. 2024-04-25T12:36:37.207 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:37.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:37.208 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:37.208 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:37.208 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:37.223 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:37.224+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:37.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:37.321 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:37.321+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:37.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.327+0000 7fee87600640 1 Processor -- start 2024-04-25T12:36:37.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.327+0000 7fee87600640 1 -- start start 2024-04-25T12:36:37.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.329+0000 7fee87600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee800c6970 0x7fee800cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.329+0000 7fee87600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee80156ad0 0x7fee8006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.329+0000 7fee87600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee800cc3b0 con 0x7fee800cde90 2024-04-25T12:36:37.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.329+0000 7fee87600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fee801609b0 con 0x7fee80156ad0 2024-04-25T12:36:37.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.329+0000 7fee87600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fee800cc650 con 0x7fee800c6970 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.330+0000 7fee7fe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee80156ad0 0x7fee8006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.330+0000 7fee84c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee800c6970 0x7fee800cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.330+0000 7fee7fe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee80156ad0 0x7fee8006d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:58942/0 (socket says 172.21.15.123:58942) 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.330+0000 7fee7fe00640 1 -- 172.21.15.123:0/923812015 learned_addr learned my addr 172.21.15.123:0/923812015 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.331+0000 7fee84c00640 1 -- 172.21.15.123:0/923812015 >> v2:172.21.15.153:3300/0 conn(0x7fee80156ad0 msgr2=0x7fee8006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.331+0000 7fee84c00640 1 --2- 172.21.15.123:0/923812015 >> v2:172.21.15.153:3300/0 conn(0x7fee80156ad0 0x7fee8006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.331+0000 7fee84c00640 1 -- 172.21.15.123:0/923812015 >> v1:172.21.15.123:6789/0 conn(0x7fee800cde90 legacy=0x7fee8015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:37.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.331+0000 7fee84c00640 1 -- 172.21.15.123:0/923812015 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee800cc930 con 0x7fee800c6970 2024-04-25T12:36:37.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.332+0000 7fee84c00640 1 --2- 172.21.15.123:0/923812015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee800c6970 0x7fee800cd410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fee740097a0 tx=0x7fee7402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7957ce1c81dbee73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.333+0000 7fee7f400640 1 -- 172.21.15.123:0/923812015 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74037070 con 0x7fee800c6970 2024-04-25T12:36:37.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.333+0000 7fee7f400640 1 -- 172.21.15.123:0/923812015 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee74032440 con 0x7fee800c6970 2024-04-25T12:36:37.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.334+0000 7fee7f400640 1 -- 172.21.15.123:0/923812015 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74032810 con 0x7fee800c6970 2024-04-25T12:36:37.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.335+0000 7fee87600640 1 -- 172.21.15.123:0/923812015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee800c6970 msgr2=0x7fee800cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.335+0000 7fee87600640 1 --2- 172.21.15.123:0/923812015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee800c6970 0x7fee800cd410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fee740097a0 tx=0x7fee7402a210 comp rx=0 tx=0).stop 2024-04-25T12:36:37.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.336+0000 7fee87600640 1 -- 172.21.15.123:0/923812015 shutdown_connections 2024-04-25T12:36:37.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.336+0000 7fee87600640 1 --2- 172.21.15.123:0/923812015 >> v2:172.21.15.153:3300/0 conn(0x7fee80156ad0 0x7fee8006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.336+0000 7fee87600640 1 --2- 172.21.15.123:0/923812015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee800c6970 0x7fee800cd410 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.336+0000 7fee87600640 1 -- 172.21.15.123:0/923812015 >> 172.21.15.123:0/923812015 conn(0x7fee800d1a30 msgr2=0x7fee800d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:37.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.336+0000 7fee87600640 1 -- 172.21.15.123:0/923812015 shutdown_connections 2024-04-25T12:36:37.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.336+0000 7fee87600640 1 -- 172.21.15.123:0/923812015 wait complete. 2024-04-25T12:36:37.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.340+0000 7fee87600640 1 Processor -- start 2024-04-25T12:36:37.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.340+0000 7fee87600640 1 -- start start 2024-04-25T12:36:37.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.343+0000 7fee87600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee800c6970 0x7fee801f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.343+0000 7fee87600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee800cde90 0x7fee801f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.343+0000 7fee84c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee800c6970 0x7fee801f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.343+0000 7fee84c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee800c6970 0x7fee801f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51310/0 (socket says 172.21.15.123:51310) 2024-04-25T12:36:37.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.344+0000 7fee84c00640 1 -- 172.21.15.123:0/3848391769 learned_addr learned my addr 172.21.15.123:0/3848391769 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:37.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.344+0000 7fee87600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee80156ad0 0x7fee8014f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.344+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee8006e6e0 con 0x7fee800c6970 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.344+0000 7fee7fe00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee800cde90 0x7fee801f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.344+0000 7fee85600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee80156ad0 0x7fee8014f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.345+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee800ccef0 con 0x7fee800cde90 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.345+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fee800cd950 con 0x7fee80156ad0 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.345+0000 7fee7fe00640 1 -- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee80156ad0 msgr2=0x7fee8014f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.345+0000 7fee7fe00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee80156ad0 0x7fee8014f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.345+0000 7fee7fe00640 1 -- 172.21.15.123:0/3848391769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee800c6970 msgr2=0x7fee801f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.345+0000 7fee7fe00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee800c6970 0x7fee801f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.345+0000 7fee7fe00640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee801f6f30 con 0x7fee800cde90 2024-04-25T12:36:37.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.346+0000 7fee84c00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee800c6970 0x7fee801f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:37.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.347+0000 7fee7fe00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee800cde90 0x7fee801f69f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee80069630 tx=0x7fee6c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.347+0000 7fee7cc00640 1 -- 172.21.15.123:0/3848391769 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c017070 con 0x7fee800cde90 2024-04-25T12:36:37.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.347+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee801f3ab0 con 0x7fee800cde90 2024-04-25T12:36:37.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.348+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801f37d0 con 0x7fee800cde90 2024-04-25T12:36:37.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.348+0000 7fee7cc00640 1 -- 172.21.15.123:0/3848391769 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee6c00c730 con 0x7fee800cde90 2024-04-25T12:36:37.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.348+0000 7fee7cc00640 1 -- 172.21.15.123:0/3848391769 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00cb00 con 0x7fee800cde90 2024-04-25T12:36:37.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.349+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c005290 con 0x7fee800cde90 2024-04-25T12:36:37.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.352+0000 7fa9abc00640 1 Processor -- start 2024-04-25T12:36:37.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.353+0000 7fa9abc00640 1 -- start start 2024-04-25T12:36:37.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.353+0000 7fee7cc00640 1 -- 172.21.15.123:0/3848391769 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee6c01a070 con 0x7fee800cde90 2024-04-25T12:36:37.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.355+0000 7fee7cc00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee600744e0 0x7fee600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.355+0000 7fee7cc00640 1 -- 172.21.15.123:0/3848391769 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6685+0+0 (secure 0 0 0) 0x7fee6c095680 con 0x7fee800cde90 2024-04-25T12:36:37.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.355+0000 7fee84c00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee600744e0 0x7fee600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.355+0000 7fa9abc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a40c8170 0x7fa9a40ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.355+0000 7fa9abc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a4074400 con 0x7fa9a40c8fd0 2024-04-25T12:36:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.355+0000 7fa9abc00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a41595e0 con 0x7fa9a41426a0 2024-04-25T12:36:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.355+0000 7fa9abc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a4074670 con 0x7fa9a40c8170 2024-04-25T12:36:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.356+0000 7fa9a9200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa9a40c8fd0 0x7fa9a4153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55522/0 (socket says 172.21.15.123:55522) 2024-04-25T12:36:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.356+0000 7fa9a9200640 1 -- 172.21.15.123:0/3190815677 learned_addr learned my addr 172.21.15.123:0/3190815677 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.356+0000 7fa9a9c00640 1 --2- 172.21.15.123:0/3190815677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a40c8170 0x7fa9a40ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.356+0000 7fa9a9c00640 1 -- 172.21.15.123:0/3190815677 >> v1:172.21.15.153:6789/0 conn(0x7fa9a41426a0 legacy=0x7fa9a4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.356+0000 7fa9a9c00640 1 -- 172.21.15.123:0/3190815677 >> v1:172.21.15.123:6789/0 conn(0x7fa9a40c8fd0 legacy=0x7fa9a4153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:37.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.356+0000 7fa9a9c00640 1 -- 172.21.15.123:0/3190815677 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a4074950 con 0x7fa9a40c8170 2024-04-25T12:36:37.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.356+0000 7fee84c00640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee600744e0 0x7fee600769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee740325c0 tx=0x7fee7402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.357+0000 7fee7cc00640 1 -- 172.21.15.123:0/3848391769 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee6c05f350 con 0x7fee800cde90 2024-04-25T12:36:37.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.358+0000 7fa9a9c00640 1 --2- 172.21.15.123:0/3190815677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a40c8170 0x7fa9a40ca200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009dc0 tx=0x7fa99c02c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570b4fd1731ea6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.358+0000 7fa9a3400640 1 -- 172.21.15.123:0/3190815677 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c032750 con 0x7fa9a40c8170 2024-04-25T12:36:37.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.358+0000 7fa9a3400640 1 -- 172.21.15.123:0/3190815677 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99c030050 con 0x7fa9a40c8170 2024-04-25T12:36:37.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.358+0000 7fa9a3400640 1 -- 172.21.15.123:0/3190815677 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c032b10 con 0x7fa9a40c8170 2024-04-25T12:36:37.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.360+0000 7fa9abc00640 1 -- 172.21.15.123:0/3190815677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a40c8170 msgr2=0x7fa9a40ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.360+0000 7fa9abc00640 1 --2- 172.21.15.123:0/3190815677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a40c8170 0x7fa9a40ca200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009dc0 tx=0x7fa99c02c890 comp rx=0 tx=0).stop 2024-04-25T12:36:37.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.360+0000 7fa9abc00640 1 -- 172.21.15.123:0/3190815677 shutdown_connections 2024-04-25T12:36:37.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.360+0000 7fa9abc00640 1 --2- 172.21.15.123:0/3190815677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a40c8170 0x7fa9a40ca200 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.360+0000 7fa9abc00640 1 -- 172.21.15.123:0/3190815677 >> 172.21.15.123:0/3190815677 conn(0x7fa9a406f440 msgr2=0x7fa9a406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:37.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.360+0000 7fa9abc00640 1 -- 172.21.15.123:0/3190815677 shutdown_connections 2024-04-25T12:36:37.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.360+0000 7fa9abc00640 1 -- 172.21.15.123:0/3190815677 wait complete. 2024-04-25T12:36:37.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.364+0000 7fa9abc00640 1 Processor -- start 2024-04-25T12:36:37.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.364+0000 7fa9abc00640 1 -- start start 2024-04-25T12:36:37.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.366+0000 7fa9abc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a40c8170 0x7fa9a40c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.366+0000 7fa9abc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9a40c8fd0 0x7fa9a40c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.366+0000 7fa9abc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a41426a0 0x7fa9a41538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.366+0000 7fa9abc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a40c8580 con 0x7fa9a40c8170 2024-04-25T12:36:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.367+0000 7fa9abc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a40c4880 con 0x7fa9a40c8fd0 2024-04-25T12:36:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.367+0000 7fa9abc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a40c8700 con 0x7fa9a41426a0 2024-04-25T12:36:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.367+0000 7fa9a9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a40c8170 0x7fa9a40c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.367+0000 7fa9a9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a40c8170 0x7fa9a40c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51318/0 (socket says 172.21.15.123:51318) 2024-04-25T12:36:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.367+0000 7fa9a9200640 1 -- 172.21.15.123:0/2948257819 learned_addr learned my addr 172.21.15.123:0/2948257819 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:37.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.368+0000 7fa9a3e00640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9a40c8fd0 0x7fa9a40c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.368+0000 7fa9a9200640 1 -- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a41426a0 msgr2=0x7fa9a41538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.368+0000 7fa9a9200640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa9a41426a0 0x7fa9a41538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.368+0000 7fa9a9200640 1 -- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9a40c8fd0 msgr2=0x7fa9a40c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.368+0000 7fa9a9200640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9a40c8fd0 0x7fa9a40c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.368+0000 7fa9a9200640 1 -- 172.21.15.123:0/2948257819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a40c6a20 con 0x7fa9a40c8170 2024-04-25T12:36:37.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.369+0000 7fa9a9200640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a40c8170 0x7fa9a40c7440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa99800b460 tx=0x7fa99800b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.369+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998015020 con 0x7fa9a40c8170 2024-04-25T12:36:37.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.369+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa998011830 con 0x7fa9a40c8170 2024-04-25T12:36:37.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.369+0000 7fa9abc00640 1 -- 172.21.15.123:0/2948257819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a41f7960 con 0x7fa9a40c8170 2024-04-25T12:36:37.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.370+0000 7fa9abc00640 1 -- 172.21.15.123:0/2948257819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a41f7f20 con 0x7fa9a40c8170 2024-04-25T12:36:37.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.370+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998011c00 con 0x7fa9a40c8170 2024-04-25T12:36:37.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.371+0000 7fa9abc00640 1 -- 172.21.15.123:0/2948257819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa980001030 con 0x7fa9a40c8170 2024-04-25T12:36:37.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.372+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa998018020 con 0x7fa9a40c8170 2024-04-25T12:36:37.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.374+0000 7fa9a0c00640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa984074500 0x7fa984076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.374+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6685+0+0 (secure 0 0 0) 0x7fa998095370 con 0x7fa9a40c8170 2024-04-25T12:36:37.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.374+0000 7fa9a0c00640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa98407dee0 0x7fa9840803d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.375+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa99800de50 con 0x7fa98407dee0 2024-04-25T12:36:37.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.375+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fa99809c050 con 0x7fa9a40c8170 2024-04-25T12:36:37.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.375+0000 7fa9a3e00640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa984074500 0x7fa984076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.375+0000 7fa9a9c00640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa98407dee0 0x7fa9840803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.376+0000 7fa9a9c00640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fa98407dee0 0x7fa9840803d0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.377+0000 7fa9a3e00640 1 --2- 172.21.15.123:0/2948257819 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa984074500 0x7fa984076a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40ce730 tx=0x7fa990009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.377+0000 7fa9a0c00640 1 -- 172.21.15.123:0/2948257819 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa99800de50 con 0x7fa98407dee0 2024-04-25T12:36:37.423 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:37.422+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:37.428 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:37.429 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:37.429 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:37.454 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:37.454 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:37.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-25T12:36:37.522 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:37.523+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:37.608 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:37.608 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 138.65606721899999, 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012628781, 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951638+0000", 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951659+0000", 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1027999999999999e-05 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951691+0000", 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2267000000000001e-05 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951710+0000", 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9063999999999999e-05 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952026+0000", 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000315719 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952367+0000", 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034072600000000002 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955799+0000", 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034325969999999999 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962852+0000", 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070528889999999997 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964076+0000", 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012234119999999999 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964178+0000", 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102166 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964264+0000", 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6175999999999996e-05 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 138.655929388, 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012746265999999999, 2024-04-25T12:36:37.614 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951774+0000", 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-06 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951782+0000", 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2090000000000005e-06 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951796+0000", 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3559e-05 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951808+0000", 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2183e-05 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956591+0000", 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047823550000000003 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956696+0000", 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105696 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:37.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956907+0000", 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000211165 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962907+0000", 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059995650000000001 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964283+0000", 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013758189999999999 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964381+0000", 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7977000000000001e-05 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964519+0000", 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138675 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:31 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 138.65476066400001, 2024-04-25T12:36:37.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076541370000000001, 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952943+0000", 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952999+0000", 2024-04-25T12:36:37.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5968999999999999e-05 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953032+0000", 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2598999999999997e-05 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953058+0000", 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6585999999999999e-05 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953207+0000", 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014848300000000001 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953263+0000", 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6005999999999999e-05 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960434+0000", 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071716219999999999 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960596+0000", 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016173899999999999 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.624 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 138.65457874500001, 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010940785, 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:37.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953125+0000", 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.22e-06 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953138+0000", 2024-04-25T12:36:37.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2996000000000001e-05 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953158+0000", 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0173000000000002e-05 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953176+0000", 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7927e-05 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954316+0000", 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011402599999999999 2024-04-25T12:36:37.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954375+0000", 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963968+0000", 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095929269999999994 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964065+0000", 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6628999999999999e-05 2024-04-25T12:36:37.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 138.65447992099999, 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080627230000000008, 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953225+0000", 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3640000000000002e-06 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953259+0000", 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3955999999999998e-05 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953292+0000", 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3549e-05 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953320+0000", 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7892e-05 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953495+0000", 2024-04-25T12:36:37.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017444800000000001 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953705+0000", 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021011000000000001 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953862+0000", 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015660399999999999 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959861+0000", 2024-04-25T12:36:37.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059993959999999997 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961071+0000", 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012101900000000001 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961192+0000", 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121279 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961285+0000", 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2935000000000004e-05 2024-04-25T12:36:37.633 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 138.65430318400001, 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010652488999999999, 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953401+0000", 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.513e-06 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953412+0000", 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209000000000001e-05 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953473+0000", 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0668999999999998e-05 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953496+0000", 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3246000000000001e-05 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956356+0000", 2024-04-25T12:36:37.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00286002 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956620+0000", 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026411099999999998 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.957005+0000", 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038473499999999998 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962757+0000", 2024-04-25T12:36:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057517080000000003 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963927+0000", 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00117023 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964002+0000", 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5483999999999998e-05 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964052+0000", 2024-04-25T12:36:37.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9564000000000002e-05 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 138.65417473400001, 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011827087999999999, 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.639 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953529+0000", 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953594+0000", 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4807999999999993e-05 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953617+0000", 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3220000000000001e-05 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953683+0000", 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5328999999999997e-05 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954191+0000", 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000508428 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040280099999999999 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954922+0000", 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032838300000000002 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960432+0000", 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055099989999999998 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965226+0000", 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047938700000000004 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965297+0000", 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0992000000000001e-05 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965355+0000", 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7695000000000002e-05 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:19 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.491270934, 2024-04-25T12:36:37.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010949354999999999, 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116434+0000", 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116475+0000", 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1486000000000002e-05 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116505+0000", 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9709e-05 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116529+0000", 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3839999999999999e-05 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116651+0000", 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122013 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116685+0000", 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4513e-05 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127273+0000", 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010587653000000001 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127381+0000", 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108204 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:20 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.491115807, 2024-04-25T12:36:37.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014061374, 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116588+0000", 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0499999999999999e-06 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116602+0000", 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3793e-05 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116621+0000", 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9545999999999999e-05 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116637+0000", 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5696e-05 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117243+0000", 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060668400000000002 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117280+0000", 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6619000000000003e-05 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130559+0000", 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013278570999999999 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130648+0000", 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9414999999999997e-05 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:27 2.9 e25/18 2:91bd965c:::501.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:37.653 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.49068915700001, 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013688128000000001, 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117015+0000", 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3200000000000001e-06 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117121+0000", 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106451 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117145+0000", 2024-04-25T12:36:37.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.404e-05 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117210+0000", 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4314999999999997e-05 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120564+0000", 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033548050000000002 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120624+0000", 2024-04-25T12:36:37.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9234999999999998e-05 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130654+0000", 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010030437999999999 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130702+0000", 2024-04-25T12:36:37.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7524000000000001e-05 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:18 2.13 e25/18 2:ca4dd7f5:::611.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.49033585799999, 2024-04-25T12:36:37.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010076254, 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117369+0000", 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.88e-06 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117385+0000", 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6670000000000001e-05 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117413+0000", 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7691999999999998e-05 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117435+0000", 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2416e-05 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117549+0000", 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113436 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1509e-05 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127347+0000", 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097767759999999992 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127443+0000", 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5874999999999999e-05 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.48912418499999, 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012295627, 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.663 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118582+0000", 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000001e-06 2024-04-25T12:36:37.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118599+0000", 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7176e-05 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2505999999999997e-05 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118659+0000", 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7559000000000002e-05 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120040+0000", 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001380956 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120305+0000", 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026497700000000001 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120707+0000", 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040189499999999999 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128035+0000", 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073284049999999996 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130711+0000", 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026752910000000002 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130787+0000", 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6384999999999995e-05 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130874+0000", 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7020000000000004e-05 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.488968185, 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012319366, 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.668 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:37.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118747+0000", 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1286e-05 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118772+0000", 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5150000000000001e-05 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118793+0000", 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0778999999999999e-05 2024-04-25T12:36:37.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121911+0000", 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031179409999999999 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122331+0000", 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041955999999999997 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124441+0000", 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021107629999999999 2024-04-25T12:36:37.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130205+0000", 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057636140000000002 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130888+0000", 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068289100000000003 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130961+0000", 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3026000000000002e-05 2024-04-25T12:36:37.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131054+0000", 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2855999999999993e-05 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:21 2.8 2:1470b2f8:::613.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.488879457, 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012409474, 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.673 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118824+0000", 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118834+0000", 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118855+0000", 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999999e-05 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118875+0000", 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9809999999999998e-05 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119612+0000", 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073729299999999998 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119912+0000", 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029931000000000001 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120202+0000", 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000289907 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130342+0000", 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010140188 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131091+0000", 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074893900000000001 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131166+0000", 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5272999999999996e-05 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131233+0000", 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6625000000000007e-05 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.48877071, 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012992702, 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.680 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118933+0000", 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3069999999999999e-06 2024-04-25T12:36:37.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118943+0000", 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7200000000000001e-06 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118967+0000", 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3828999999999998e-05 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118986+0000", 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9057000000000001e-05 2024-04-25T12:36:37.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124788+0000", 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00580271 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125092+0000", 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030394999999999999 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125479+0000", 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038660000000000002 2024-04-25T12:36:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131299+0000", 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058200559999999997 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131823+0000", 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052382700000000004 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131888+0000", 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4740999999999996e-05 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131925+0000", 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6905e-05 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:5 2.7 e25/18 2:e455cf45:::60b.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 107.488647262, 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011712752, 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.685 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119057+0000", 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.296e-06 2024-04-25T12:36:37.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119068+0000", 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.189e-05 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119091+0000", 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2802999999999998e-05 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119110+0000", 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8473000000000001e-05 2024-04-25T12:36:37.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121085+0000", 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019750689999999999 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121149+0000", 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3979000000000005e-05 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130709+0000", 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009559817 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130768+0000", 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9425000000000001e-05 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:21 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.452857974, 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066623919999999996, 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.689 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154846+0000", 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.06e-06 2024-04-25T12:36:37.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154856+0000", 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0713000000000001e-05 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154878+0000", 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1180000000000001e-05 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154896+0000", 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8343e-05 2024-04-25T12:36:37.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155332+0000", 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043663000000000003 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155389+0000", 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6818999999999999e-05 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161381+0000", 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059914440000000003 2024-04-25T12:36:37.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161507+0000", 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126203 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:22 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.452584203, 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064678330000000001, 2024-04-25T12:36:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155120+0000", 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155131+0000", 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1793e-05 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155155+0000", 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3306e-05 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155174+0000", 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9383000000000001e-05 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157101+0000", 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019271570000000001 2024-04-25T12:36:37.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157161+0000", 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9945000000000003e-05 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161512+0000", 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043506389999999999 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161586+0000", 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4250000000000002e-05 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:26 2.11 2:893cd530:::mds2_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.45164723000001, 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064127289999999998, 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156057+0000", 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8330000000000001e-06 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156090+0000", 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2719000000000003e-05 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156116+0000", 2024-04-25T12:36:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6531999999999999e-05 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156140+0000", 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3453999999999999e-05 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156350+0000", 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021060999999999999 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156467+0000", 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116823 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156594+0000", 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012698900000000001 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:37.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161255+0000", 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046609570000000003 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162360+0000", 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001105167 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162416+0000", 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6076999999999999e-05 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162468+0000", 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1567999999999998e-05 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 102.451496581, 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066007829999999998, 2024-04-25T12:36:37.705 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:37.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156208+0000", 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5570000000000001e-06 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156216+0000", 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0900000000000005e-06 2024-04-25T12:36:37.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156237+0000", 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1452999999999999e-05 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156255+0000", 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8091999999999999e-05 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156841+0000", 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00058573200000000005 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156983+0000", 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000141676 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:37.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162807+0000", 2024-04-25T12:36:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058241830000000001 2024-04-25T12:36:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:37.710 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:37.710 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:37.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:36:37.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:37.713 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:37.713+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:37.759 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:36:37.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.763+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fee5c005580 con 0x7fee800cde90 2024-04-25T12:36:37.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.803+0000 7faee2200640 1 Processor -- start 2024-04-25T12:36:37.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.803+0000 7faee2200640 1 -- start start 2024-04-25T12:36:37.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.806+0000 7faee2200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faedc0c48a0 0x7faedc15a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.806+0000 7faee2200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc1543b0 0x7faedc0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.806+0000 7faee2200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faedc15ac70 con 0x7faedc0cbc70 2024-04-25T12:36:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.806+0000 7faee2200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7faedc15b510 con 0x7faedc0c48a0 2024-04-25T12:36:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.806+0000 7faee2200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faedc15e000 con 0x7faedc1543b0 2024-04-25T12:36:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.807+0000 7faedaa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7faedc0cbc70 0x7faedc15e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55536/0 (socket says 172.21.15.123:55536) 2024-04-25T12:36:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.807+0000 7faedaa00640 1 -- 172.21.15.123:0/2333606674 learned_addr learned my addr 172.21.15.123:0/2333606674 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:37.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.807+0000 7faedb400640 1 --2- 172.21.15.123:0/2333606674 >> v2:172.21.15.123:3300/0 conn(0x7faedc0c48a0 0x7faedc15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.808+0000 7faedbe00640 1 --2- 172.21.15.123:0/2333606674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc1543b0 0x7faedc0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.808+0000 7faedbe00640 1 -- 172.21.15.123:0/2333606674 >> v2:172.21.15.123:3300/0 conn(0x7faedc0c48a0 msgr2=0x7faedc15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:37.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.808+0000 7faedbe00640 1 --2- 172.21.15.123:0/2333606674 >> v2:172.21.15.123:3300/0 conn(0x7faedc0c48a0 0x7faedc15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.808+0000 7faedbe00640 1 -- 172.21.15.123:0/2333606674 >> v1:172.21.15.123:6789/0 conn(0x7faedc0cbc70 legacy=0x7faedc15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.808+0000 7faedbe00640 1 -- 172.21.15.123:0/2333606674 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc15af50 con 0x7faedc1543b0 2024-04-25T12:36:37.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.810+0000 7faedbe00640 1 --2- 172.21.15.123:0/2333606674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc1543b0 0x7faedc0cfcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faed4009d90 tx=0x7faed402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b88a47c01c92d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.810+0000 7faeda000640 1 -- 172.21.15.123:0/2333606674 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4031600 con 0x7faedc1543b0 2024-04-25T12:36:37.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.811+0000 7faeda000640 1 -- 172.21.15.123:0/2333606674 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed40317a0 con 0x7faedc1543b0 2024-04-25T12:36:37.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.811+0000 7faeda000640 1 -- 172.21.15.123:0/2333606674 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4031600 con 0x7faedc1543b0 2024-04-25T12:36:37.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.812+0000 7faee2200640 1 -- 172.21.15.123:0/2333606674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc1543b0 msgr2=0x7faedc0cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.812+0000 7faee2200640 1 --2- 172.21.15.123:0/2333606674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc1543b0 0x7faedc0cfcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faed4009d90 tx=0x7faed402a5d0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.811 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:37.813+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.813+0000 7faee2200640 1 -- 172.21.15.123:0/2333606674 shutdown_connections 2024-04-25T12:36:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.813+0000 7faee2200640 1 --2- 172.21.15.123:0/2333606674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc1543b0 0x7faedc0cfcd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.813+0000 7faee2200640 1 --2- 172.21.15.123:0/2333606674 >> v2:172.21.15.123:3300/0 conn(0x7faedc0c48a0 0x7faedc15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.813+0000 7faee2200640 1 -- 172.21.15.123:0/2333606674 >> 172.21.15.123:0/2333606674 conn(0x7faedc068600 msgr2=0x7faedc068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:37.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.816+0000 7faee2200640 1 -- 172.21.15.123:0/2333606674 shutdown_connections 2024-04-25T12:36:37.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.816+0000 7faee2200640 1 -- 172.21.15.123:0/2333606674 wait complete. 2024-04-25T12:36:37.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.819+0000 7faee2200640 1 Processor -- start 2024-04-25T12:36:37.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.820+0000 7faee2200640 1 -- start start 2024-04-25T12:36:37.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.822+0000 7faee2200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc0c48a0 0x7faedc147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.822+0000 7faee2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faedc0cbc70 0x7faedc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.822+0000 7faee2200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faedc1543b0 0x7faedc1474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.822+0000 7faee2200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faedc0c4f60 con 0x7faedc0cbc70 2024-04-25T12:36:37.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.822+0000 7faee2200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faedc15dc90 con 0x7faedc1543b0 2024-04-25T12:36:37.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.822+0000 7faee2200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faedc0c50e0 con 0x7faedc0c48a0 2024-04-25T12:36:37.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faedc0cbc70 0x7faedc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faedc1543b0 0x7faedc1474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faedc1543b0 0x7faedc1474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:58972/0 (socket says 172.21.15.123:58972) 2024-04-25T12:36:37.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedbe00640 1 -- 172.21.15.123:0/586347275 learned_addr learned my addr 172.21.15.123:0/586347275 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:37.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedb400640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc0c48a0 0x7faedc147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedbe00640 1 -- 172.21.15.123:0/586347275 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc0c48a0 msgr2=0x7faedc147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedbe00640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faedc0c48a0 0x7faedc147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.823+0000 7faedbe00640 1 -- 172.21.15.123:0/586347275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faedc0cbc70 msgr2=0x7faedc0b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.824+0000 7faedbe00640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faedc0cbc70 0x7faedc0b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.824+0000 7faedbe00640 1 -- 172.21.15.123:0/586347275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc0b53e0 con 0x7faedc1543b0 2024-04-25T12:36:37.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.824+0000 7faedaa00640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faedc0cbc70 0x7faedc0b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:37.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.824+0000 7faedbe00640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faedc1543b0 0x7faedc1474f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faed402abc0 tx=0x7faed4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.824+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4009450 con 0x7faedc1543b0 2024-04-25T12:36:37.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.825+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4031b70 con 0x7faedc1543b0 2024-04-25T12:36:37.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.825+0000 7faee2200640 1 -- 172.21.15.123:0/586347275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc148630 con 0x7faedc1543b0 2024-04-25T12:36:37.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.825+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed40330d0 con 0x7faedc1543b0 2024-04-25T12:36:37.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.826+0000 7faee2200640 1 -- 172.21.15.123:0/586347275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc148910 con 0x7faedc1543b0 2024-04-25T12:36:37.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.827+0000 7faee2200640 1 -- 172.21.15.123:0/586347275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faedc076700 con 0x7faedc1543b0 2024-04-25T12:36:37.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.828+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faed4033270 con 0x7faedc1543b0 2024-04-25T12:36:37.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.829+0000 7faed3400640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faebc0744e0 0x7faebc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.829+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6685+0+0 (secure 0 0 0) 0x7faed40b5d50 con 0x7faedc1543b0 2024-04-25T12:36:37.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.830+0000 7faed3400640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faebc07ddc0 0x7faebc0802b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:37.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.830+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faebc080980 con 0x7faebc07ddc0 2024-04-25T12:36:37.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.830+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7faed40b6110 con 0x7faedc1543b0 2024-04-25T12:36:37.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.830+0000 7faedb400640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faebc0744e0 0x7faebc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.830+0000 7faedaa00640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faebc07ddc0 0x7faebc0802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:37.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.831+0000 7faedb400640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faebc0744e0 0x7faebc0769f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faecc009970 tx=0x7faecc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.831+0000 7faedaa00640 1 --2- 172.21.15.123:0/586347275 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7faebc07ddc0 0x7faebc0802b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:37.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.832+0000 7faed3400640 1 -- 172.21.15.123:0/586347275 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faebc080980 con 0x7faebc07ddc0 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:37.885 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:37.906 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:37.906 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:37.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-04-25T12:36:37.913 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:37.914+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:37.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.969+0000 7fee7cc00640 1 -- 172.21.15.123:0/3848391769 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v63) ==== 87+0+0 (secure 0 0 0) 0x7fee6c063650 con 0x7fee800cde90 2024-04-25T12:36:37.968 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:36:37.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.974+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee600744e0 msgr2=0x7fee600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.974+0000 7fee87600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee600744e0 0x7fee600769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee740325c0 tx=0x7fee7402e000 comp rx=0 tx=0).stop 2024-04-25T12:36:37.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.974+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee800cde90 msgr2=0x7fee801f69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:37.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.974+0000 7fee87600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee800cde90 0x7fee801f69f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee80069630 tx=0x7fee6c00ecd0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 shutdown_connections 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee600744e0 0x7fee600769f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee80156ad0 0x7fee8014f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee800cde90 0x7fee801f69f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 --2- 172.21.15.123:0/3848391769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee800c6970 0x7fee801f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 >> 172.21.15.123:0/3848391769 conn(0x7fee800d1a30 msgr2=0x7fee800b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 shutdown_connections 2024-04-25T12:36:37.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:37.975+0000 7fee87600640 1 -- 172.21.15.123:0/3848391769 wait complete. 2024-04-25T12:36:38.014 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:38.015+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:38.095 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:38.095 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:38.095 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:38.095 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:38.114 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:38.115+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:38.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:38.215 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:38.216+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:38.316 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:38.317+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:38.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.349+0000 7fa7ff200640 1 Processor -- start 2024-04-25T12:36:38.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.349+0000 7fa7ff200640 1 -- start start 2024-04-25T12:36:38.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.351+0000 7fa7ff200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa8000c6800 0x7fa8000ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.351+0000 7fa7ff200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8000cedf0 con 0x7fa8001568c0 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.351+0000 7fa7ff200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8000c7df0 con 0x7fa8000cdca0 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.351+0000 7fa7ff200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa8000cebd0 con 0x7fa8000c6800 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.351+0000 7fa7fe800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa8000c6800 0x7fa8000ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.351+0000 7fa7fd400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa8000cdca0 0x7fa80015c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:36174/0 (socket says 172.21.15.123:36174) 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.351+0000 7fa7fd400640 1 -- 172.21.15.123:0/4231047094 learned_addr learned my addr 172.21.15.123:0/4231047094 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.352+0000 7fa7fca00640 1 -- 172.21.15.123:0/4231047094 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872714567 0 0) 0x7fa8000c7df0 con 0x7fa8000cdca0 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.352+0000 7fa7fca00640 1 -- 172.21.15.123:0/4231047094 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7e4003520 con 0x7fa8000cdca0 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.352+0000 7fa7fe800640 1 -- 172.21.15.123:0/4231047094 >> v1:172.21.15.153:6789/0 conn(0x7fa8000cdca0 legacy=0x7fa80015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.352+0000 7fa7fe800640 1 -- 172.21.15.123:0/4231047094 >> v1:172.21.15.123:6789/0 conn(0x7fa8001568c0 legacy=0x7fa8000ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:38.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.352+0000 7fa7fe800640 1 -- 172.21.15.123:0/4231047094 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8000c80d0 con 0x7fa8000c6800 2024-04-25T12:36:38.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.354+0000 7fa7fe800640 1 --2- 172.21.15.123:0/4231047094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 0x7fa8000ce690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009d90 tx=0x7fa7f802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ab45628a03cac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.354+0000 7fa7fca00640 1 -- 172.21.15.123:0/4231047094 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8007740 con 0x7fa8000c6800 2024-04-25T12:36:38.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.354+0000 7fa7fca00640 1 -- 172.21.15.123:0/4231047094 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f80078e0 con 0x7fa8000c6800 2024-04-25T12:36:38.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.354+0000 7fa7fca00640 1 -- 172.21.15.123:0/4231047094 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8007c90 con 0x7fa8000c6800 2024-04-25T12:36:38.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.355+0000 7fa7ff200640 1 -- 172.21.15.123:0/4231047094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 msgr2=0x7fa8000ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.355+0000 7fa7ff200640 1 --2- 172.21.15.123:0/4231047094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 0x7fa8000ce690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009d90 tx=0x7fa7f802a6f0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.355+0000 7fa7ff200640 1 -- 172.21.15.123:0/4231047094 shutdown_connections 2024-04-25T12:36:38.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.355+0000 7fa7ff200640 1 --2- 172.21.15.123:0/4231047094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 0x7fa8000ce690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.355+0000 7fa7ff200640 1 -- 172.21.15.123:0/4231047094 >> 172.21.15.123:0/4231047094 conn(0x7fa80006dcd0 msgr2=0x7fa80006e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:38.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.356+0000 7fa7ff200640 1 -- 172.21.15.123:0/4231047094 shutdown_connections 2024-04-25T12:36:38.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.356+0000 7fa7ff200640 1 -- 172.21.15.123:0/4231047094 wait complete. 2024-04-25T12:36:38.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.358+0000 7fa7ff200640 1 Processor -- start 2024-04-25T12:36:38.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.358+0000 7fa7ff200640 1 -- start start 2024-04-25T12:36:38.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.359+0000 7fa7ff200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 0x7fa8001531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.359+0000 7fa7ff200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8000cdca0 0x7fa8001f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.359+0000 7fa7ff200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8000c8ab0 0x7fa80014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.359+0000 7fa7ff200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000c8720 con 0x7fa8000c8ab0 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.359+0000 7fa7ff200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000cf0d0 con 0x7fa8000c6800 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.359+0000 7fa7ff200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa8000cd630 con 0x7fa8000cdca0 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.360+0000 7fa7fe800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8000c8ab0 0x7fa80014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.360+0000 7fa7fe800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8000c8ab0 0x7fa80014c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51330/0 (socket says 172.21.15.123:51330) 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.360+0000 7fa7fe800640 1 -- 172.21.15.123:0/3568704979 learned_addr learned my addr 172.21.15.123:0/3568704979 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.360+0000 7fa7fd400640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8000cdca0 0x7fa8001f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.360+0000 7fa7fde00640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 0x7fa8001531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7fe800640 1 -- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8000cdca0 msgr2=0x7fa8001f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7fe800640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8000cdca0 0x7fa8001f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7fe800640 1 -- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 msgr2=0x7fa8001531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7fe800640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8000c6800 0x7fa8001531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7fe800640 1 -- 172.21.15.123:0/3568704979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8001f64e0 con 0x7fa8000c8ab0 2024-04-25T12:36:38.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7fe800640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8000c8ab0 0x7fa80014c600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7f802ace0 tx=0x7fa7f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8009450 con 0x7fa8000c8ab0 2024-04-25T12:36:38.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f8004240 con 0x7fa8000c8ab0 2024-04-25T12:36:38.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8004630 con 0x7fa8000c8ab0 2024-04-25T12:36:38.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.361+0000 7fa7ff200640 1 -- 172.21.15.123:0/3568704979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8001f2180 con 0x7fa8000c8ab0 2024-04-25T12:36:38.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.362+0000 7fa7ff200640 1 -- 172.21.15.123:0/3568704979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8001f2420 con 0x7fa8000c8ab0 2024-04-25T12:36:38.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.363+0000 7fa7ff200640 1 -- 172.21.15.123:0/3568704979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa80006a030 con 0x7fa8000c8ab0 2024-04-25T12:36:38.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.365+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa7f803b410 con 0x7fa8000c8ab0 2024-04-25T12:36:38.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.366+0000 7fa7f6000640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7e0074500 0x7fa7e0076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.366+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6685+0+0 (secure 0 0 0) 0x7fa7f80b9400 con 0x7fa8000c8ab0 2024-04-25T12:36:38.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.366+0000 7fa7f6000640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa7e007dee0 0x7fa7e00803d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.366+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7e0080aa0 con 0x7fa7e007dee0 2024-04-25T12:36:38.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.366+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa7f80c0050 con 0x7fa8000c8ab0 2024-04-25T12:36:38.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.367+0000 7fa7fd400640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa7e007dee0 0x7fa7e00803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.367+0000 7fa7fde00640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7e0074500 0x7fa7e0076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.367+0000 7fa7fd400640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fa7e007dee0 0x7fa7e00803d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.368+0000 7fa7fde00640 1 --2- 172.21.15.123:0/3568704979 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa7e0074500 0x7fa7e0076a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0000f60 tx=0x7fa7f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.368+0000 7fa7f6000640 1 -- 172.21.15.123:0/3568704979 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7e0080aa0 con 0x7fa7e007dee0 2024-04-25T12:36:38.378 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:38.379 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:38.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:38.414 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:38.415 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:38.416 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:38.417+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:38.435 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:38.435 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:38.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-04-25T12:36:38.518 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:38.519+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:38.619 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:38.620+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:38.719 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:38.720+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:38.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.724+0000 7f370cc00640 1 Processor -- start 2024-04-25T12:36:38.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.724+0000 7f370cc00640 1 -- start start 2024-04-25T12:36:38.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f370cc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c5490 0x7f37081565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f370cc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f370806d680 0x7f37080cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f370cc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f370815c4b0 con 0x7f37080c6480 2024-04-25T12:36:38.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f370cc00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f37081601b0 con 0x7f370806d680 2024-04-25T12:36:38.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f370cc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f37080cae20 con 0x7f37080c5490 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f3707400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c5490 0x7f37081565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f3707400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c5490 0x7f37081565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:46096/0 (socket says 172.21.15.123:46096) 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.727+0000 7f3707400640 1 -- 172.21.15.123:0/3213086250 learned_addr learned my addr 172.21.15.123:0/3213086250 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.728+0000 7f3707400640 1 -- 172.21.15.123:0/3213086250 >> v2:172.21.15.123:3300/0 conn(0x7f370806d680 msgr2=0x7f37080cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.728+0000 7f3707e00640 1 --2- 172.21.15.123:0/3213086250 >> v2:172.21.15.123:3300/0 conn(0x7f370806d680 0x7f37080cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.728+0000 7f3707400640 1 --2- 172.21.15.123:0/3213086250 >> v2:172.21.15.123:3300/0 conn(0x7f370806d680 0x7f37080cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.728+0000 7f3707400640 1 -- 172.21.15.123:0/3213086250 >> v1:172.21.15.123:6789/0 conn(0x7f37080c6480 legacy=0x7f370815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.728+0000 7f3707400640 1 -- 172.21.15.123:0/3213086250 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37080cb100 con 0x7f37080c5490 2024-04-25T12:36:38.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.728+0000 7f3707e00640 1 --2- 172.21.15.123:0/3213086250 >> v2:172.21.15.123:3300/0 conn(0x7f370806d680 0x7f37080cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:38.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.730+0000 7f3707400640 1 --2- 172.21.15.123:0/3213086250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c5490 0x7f37081565d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009870 tx=0x7f36f802a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb225a468c23ef34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.731+0000 7f3706000640 1 -- 172.21.15.123:0/3213086250 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8037070 con 0x7f37080c5490 2024-04-25T12:36:38.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.731+0000 7f3706000640 1 -- 172.21.15.123:0/3213086250 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f802adf0 con 0x7f37080c5490 2024-04-25T12:36:38.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.731+0000 7f3706000640 1 -- 172.21.15.123:0/3213086250 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8032660 con 0x7f37080c5490 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 -- 172.21.15.123:0/3213086250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c5490 msgr2=0x7f37081565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 --2- 172.21.15.123:0/3213086250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c5490 0x7f37081565d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009870 tx=0x7f36f802a160 comp rx=0 tx=0).stop 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 -- 172.21.15.123:0/3213086250 shutdown_connections 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 --2- 172.21.15.123:0/3213086250 >> v2:172.21.15.123:3300/0 conn(0x7f370806d680 0x7f37080cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 --2- 172.21.15.123:0/3213086250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c5490 0x7f37081565d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 -- 172.21.15.123:0/3213086250 >> 172.21.15.123:0/3213086250 conn(0x7f3708068600 msgr2=0x7f3708068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 -- 172.21.15.123:0/3213086250 shutdown_connections 2024-04-25T12:36:38.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.732+0000 7f370cc00640 1 -- 172.21.15.123:0/3213086250 wait complete. 2024-04-25T12:36:38.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.736+0000 7f370cc00640 1 Processor -- start 2024-04-25T12:36:38.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.736+0000 7f370cc00640 1 -- start start 2024-04-25T12:36:38.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.738+0000 7f370cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370806d680 0x7f3708155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.738+0000 7f370cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37080c5490 0x7f3708154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.738+0000 7f370cc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c6480 0x7f3708150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.738+0000 7f370cc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f370815cd30 con 0x7f370806d680 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.738+0000 7f370cc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f370815ca50 con 0x7f37080c5490 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.738+0000 7f370cc00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f37080cb4f0 con 0x7f37080c6480 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37080c5490 0x7f3708154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37080c5490 0x7f3708154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:59004/0 (socket says 172.21.15.123:59004) 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 -- 172.21.15.123:0/4042517338 learned_addr learned my addr 172.21.15.123:0/4042517338 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3707e00640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c6480 0x7f3708150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3707400640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370806d680 0x7f3708155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 -- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c6480 msgr2=0x7f3708150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37080c6480 0x7f3708150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 -- 172.21.15.123:0/4042517338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370806d680 msgr2=0x7f3708155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:38.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f370806d680 0x7f3708155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:38.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.739+0000 7f3706a00640 1 -- 172.21.15.123:0/4042517338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37081f24d0 con 0x7f37080c5490 2024-04-25T12:36:38.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.740+0000 7f3706a00640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37080c5490 0x7f3708154170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c4a0 tx=0x7f36f400c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.740+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4017070 con 0x7f37080c5490 2024-04-25T12:36:38.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.740+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f4005480 con 0x7f37080c5490 2024-04-25T12:36:38.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.740+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4003830 con 0x7f37080c5490 2024-04-25T12:36:38.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.741+0000 7f370cc00640 1 -- 172.21.15.123:0/4042517338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37081f2770 con 0x7f37080c5490 2024-04-25T12:36:38.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.741+0000 7f370cc00640 1 -- 172.21.15.123:0/4042517338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081511b0 con 0x7f37080c5490 2024-04-25T12:36:38.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.742+0000 7f370cc00640 1 -- 172.21.15.123:0/4042517338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3708076da0 con 0x7f37080c5490 2024-04-25T12:36:38.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.744+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f36f401a070 con 0x7f37080c5490 2024-04-25T12:36:38.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.746+0000 7f36ff400640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36e8074530 0x7f36e8076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.746+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6685+0+0 (secure 0 0 0) 0x7f36f40953b0 con 0x7f37080c5490 2024-04-25T12:36:38.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.747+0000 7f36ff400640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f36e807df10 0x7f36e8080400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:38.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.747+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36f4009eb0 con 0x7f36e807df10 2024-04-25T12:36:38.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.747+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f36f4095760 con 0x7f37080c5490 2024-04-25T12:36:38.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.747+0000 7f3707400640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36e8074530 0x7f36e8076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.747+0000 7f3707e00640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f36e807df10 0x7f36e8080400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:38.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.748+0000 7f3707e00640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f36e807df10 0x7f36e8080400 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.748+0000 7f3707400640 1 --2- 172.21.15.123:0/4042517338 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f36e8074530 0x7f36e8076a40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f36f8034000 tx=0x7f36f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:38.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:38.749+0000 7f36ff400640 1 -- 172.21.15.123:0/4042517338 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36f4009eb0 con 0x7f36e807df10 2024-04-25T12:36:38.758 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:38.758 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:38.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:38.758 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:38.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.758 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 150.733249215, 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53759151699999996, 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024333+0000", 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024334+0000", 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7179999999999999e-06 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024345+0000", 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0608000000000001e-05 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024373+0000", 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7966e-05 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024392+0000", 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024490+0000", 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7823000000000002e-05 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for peered", 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.024520+0000", 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0056000000000002e-05 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.559770+0000", 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53525036000000004 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560001+0000", 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023139100000000001 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.560330+0000", 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032824500000000001 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561700+0000", 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013704730000000001 2024-04-25T12:36:38.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561742+0000", 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1826000000000002e-05 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561855+0000", 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011287599999999999 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.561924+0000", 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9484000000000001e-05 2024-04-25T12:36:38.764 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 150.170325996, 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095393000000000006, 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.765 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587256+0000", 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587257+0000", 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9100000000000003e-07 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587490+0000", 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000233186 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587502+0000", 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2499e-05 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587512+0000", 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7699999999999996e-06 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.591891+0000", 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043783770000000001 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592036+0000", 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014543999999999999 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.592306+0000", 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026964399999999998 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595185+0000", 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028791730000000001 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596722+0000", 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001537131 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596762+0000", 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0491000000000002e-05 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.596795+0000", 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2798e-05 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:38.770 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 150.17005307700001, 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012300247, 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 24 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587529+0000", 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587530+0000", 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3799999999999996e-07 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587688+0000", 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015835999999999999 2024-04-25T12:36:38.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587699+0000", 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0727000000000001e-05 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587710+0000", 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0987e-05 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593399+0000", 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005688935 2024-04-25T12:36:38.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593685+0000", 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028662199999999999 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.593893+0000", 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020758500000000001 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595599+0000", 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017058209999999999 2024-04-25T12:36:38.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599668+0000", 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040696439999999999 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599769+0000", 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100752 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599829+0000", 2024-04-25T12:36:38.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0075999999999999e-05 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 150.16983101899999, 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012351434, 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 25 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587751+0000", 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587752+0000", 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587758+0000", 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2199999999999997e-06 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587770+0000", 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2129e-05 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.587778+0000", 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594340+0000", 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065618270000000001 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594415+0000", 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4535000000000004e-05 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.594510+0000", 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5100999999999996e-05 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.595607+0000", 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010968239999999999 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.599926+0000", 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043194469999999997 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600027+0000", 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100961 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.600102+0000", 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5339000000000005e-05 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:38.781 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.96488182900001, 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091591999999999993, 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792700+0000", 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792701+0000", 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.17e-06 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792911+0000", 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000210163 2024-04-25T12:36:38.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792932+0000", 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0472e-05 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.792952+0000", 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0553000000000001e-05 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799157+0000", 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062046030000000004 2024-04-25T12:36:38.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799246+0000", 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8505999999999996e-05 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.799389+0000", 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014319100000000001 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801232+0000", 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018427599999999999 2024-04-25T12:36:38.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801753+0000", 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052153699999999998 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801814+0000", 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1014000000000002e-05 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801859+0000", 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5231000000000001e-05 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.786 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.96444348099999, 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092910930000000003, 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.787 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 52 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793138+0000", 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793140+0000", 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-06 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793151+0000", 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0966e-05 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793168+0000", 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.793184+0000", 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6200000000000001e-05 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801271+0000", 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080863189999999998 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801360+0000", 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9248999999999999e-05 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.801515+0000", 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000154883 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802197+0000", 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068243299999999998 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802339+0000", 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014159 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802393+0000", 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4215000000000002e-05 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:08.802430+0000", 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6434999999999999e-05 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:38.792 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.50644021100001, 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012798129, 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 70 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251142+0000", 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251143+0000", 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2729999999999999e-06 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251153+0000", 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8439999999999993e-06 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251172+0000", 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8906000000000002e-05 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251183+0000", 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1653e-05 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257332+0000", 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061486459999999998 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257490+0000", 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000157947 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.257724+0000", 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023408200000000001 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259695+0000", 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001971169 2024-04-25T12:36:38.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263731+0000", 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035683 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263842+0000", 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110985 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263940+0000", 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7941000000000003e-05 2024-04-25T12:36:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.50638330199999, 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013164844, 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.798 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 71 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251199+0000", 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251200+0000", 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2999999999999999e-07 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251206+0000", 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9399999999999999e-06 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251215+0000", 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251223+0000", 2024-04-25T12:36:38.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2260000000000005e-06 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.259955+0000", 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087315629999999995 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260077+0000", 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122263 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260310+0000", 2024-04-25T12:36:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232702 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.263699+0000", 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033888080000000001 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264218+0000", 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051973199999999996 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264304+0000", 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5433999999999996e-05 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264363+0000", 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9743e-05 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.506341693, 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017184050999999999, 2024-04-25T12:36:38.803 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 72 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251240+0000", 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251241+0000", 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.23e-07 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251477+0000", 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023584499999999999 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251504+0000", 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7633000000000001e-05 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251516+0000", 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.164e-05 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.260993+0000", 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094771119999999993 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261122+0000", 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012831400000000001 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.261319+0000", 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019727199999999999 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264078+0000", 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027589120000000001 2024-04-25T12:36:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268258+0000", 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041806569999999999 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268357+0000", 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8721000000000006e-05 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268424+0000", 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7222000000000001e-05 2024-04-25T12:36:38.808 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.50604675, 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017105597, 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.809 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 73 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251535+0000", 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251536+0000", 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4300000000000001e-07 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251543+0000", 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3840000000000002e-06 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251552+0000", 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6860000000000001e-06 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251560+0000", 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0499999999999992e-06 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262615+0000", 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011054494 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262757+0000", 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142521 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.262999+0000", 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024147400000000001 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264190+0000", 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011911840000000001 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268472+0000", 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042823940000000001 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268571+0000", 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8218000000000001e-05 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268641+0000", 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0248999999999998e-05 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.814 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.506009764, 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017309201999999999, 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 74 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:38.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251572+0000", 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251573+0000", 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9999999999999997e-07 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251579+0000", 2024-04-25T12:36:38.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6330000000000001e-06 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251592+0000", 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3869999999999999e-05 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251621+0000", 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8966e-05 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264505+0000", 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012883307 2024-04-25T12:36:38.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264627+0000", 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012204 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.264832+0000", 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020571899999999999 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268227+0000", 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003394308 2024-04-25T12:36:38.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268663+0000", 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043630699999999998 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268761+0000", 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7774000000000003e-05 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268881+0000", 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120578 2024-04-25T12:36:38.819 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.50593965499999, 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01751983, 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.820 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 75 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251642+0000", 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251643+0000", 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5300000000000003e-07 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251849+0000", 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000206345 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251863+0000", 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3077e-05 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251873+0000", 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.08e-05 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265525+0000", 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013651876 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265667+0000", 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014156699999999999 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.265939+0000", 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027219700000000001 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268931+0000", 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029921990000000001 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269008+0000", 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6619000000000006e-05 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269082+0000", 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4401999999999997e-05 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.269162+0000", 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9895000000000007e-05 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.50569198700001, 2024-04-25T12:36:38.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018357248, 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 76 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251890+0000", 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251891+0000", 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7300000000000005e-07 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251905+0000", 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4279e-05 2024-04-25T12:36:38.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251915+0000", 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.251935+0000", 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.961e-05 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267152+0000", 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015217341000000001 2024-04-25T12:36:38.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267276+0000", 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012396 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.267509+0000", 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023329500000000001 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.268988+0000", 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014781709999999999 2024-04-25T12:36:38.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270041+0000", 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001053806 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270161+0000", 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119494 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.270247+0000", 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6382000000000001e-05 2024-04-25T12:36:38.830 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.48646747800001, 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099439890000000003, 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.831 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 80 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271114+0000", 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271115+0000", 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271221+0000", 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105713 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271236+0000", 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5415999999999999e-05 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.271240+0000", 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3449999999999999e-06 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275441+0000", 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042009090000000001 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275579+0000", 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013740700000000001 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.275795+0000", 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021634900000000001 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.277926+0000", 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021311220000000001 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.280958+0000", 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030320490000000002 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281016+0000", 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8232000000000001e-05 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.281058+0000", 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1993999999999998e-05 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:38.836 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.131451113, 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094693499999999996, 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 92 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:38.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626131+0000", 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626132+0000", 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4399999999999999e-07 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626137+0000", 2024-04-25T12:36:38.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4929999999999998e-06 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626147+0000", 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5300000000000002e-06 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626155+0000", 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4130000000000006e-06 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629532+0000", 2024-04-25T12:36:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033774830000000001 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629626+0000", 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3394999999999996e-05 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.629792+0000", 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016611100000000001 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633343+0000", 2024-04-25T12:36:38.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00355055 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635469+0000", 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021262009999999999 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635548+0000", 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9167999999999998e-05 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635600+0000", 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2262000000000002e-05 2024-04-25T12:36:38.841 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.131414777, 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096152330000000008, 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 93 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626167+0000", 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626168+0000", 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9699999999999995e-07 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626290+0000", 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121941 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626304+0000", 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3729e-05 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626321+0000", 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7062999999999999e-05 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630147+0000", 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003826 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630229+0000", 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1953000000000006e-05 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.630372+0000", 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014345199999999999 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.633348+0000", 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029755789999999999 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635672+0000", 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023242829999999999 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635733+0000", 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0899000000000001e-05 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635782+0000", 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9636999999999999e-05 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4175.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 149.13124085699999, 2024-04-25T12:36:38.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097157200000000006, 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4175", 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.153:0/2365170275", 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 94 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626341+0000", 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626342+0000", 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4300000000000002e-07 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626348+0000", 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0059999999999998e-06 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626358+0000", 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.626367+0000", 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8939999999999999e-06 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631146+0000", 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047795429999999998 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631307+0000", 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016041699999999999 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.631539+0000", 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232258 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635089+0000", 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035503980000000002 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635880+0000", 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079070200000000001 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.635984+0000", 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104221 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:09.636057+0000", 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2597999999999998e-05 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.63953130500001, 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010749583, 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:38.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118051+0000", 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118054+0000", 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118075+0000", 2024-04-25T12:36:38.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118119+0000", 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4372000000000001e-05 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118163+0000", 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3766000000000001e-05 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119484+0000", 2024-04-25T12:36:38.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001320494 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119803+0000", 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031975599999999998 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120121+0000", 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031758999999999997 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128491+0000", 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083702940000000003 2024-04-25T12:36:38.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128605+0000", 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113158 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128733+0000", 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012861099999999999 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128800+0000", 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7086000000000001e-05 2024-04-25T12:36:38.857 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.639332319, 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011301215999999999, 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118250+0000", 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118251+0000", 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6160000000000001e-06 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118263+0000", 2024-04-25T12:36:38.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2243e-05 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118305+0000", 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1838999999999997e-05 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118327+0000", 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2082999999999998e-05 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120981+0000", 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002653201 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121221+0000", 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000240058 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121655+0000", 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043405400000000002 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128501+0000", 2024-04-25T12:36:38.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068465519999999997 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128984+0000", 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048240400000000003 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129487+0000", 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050289499999999995 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129551+0000", 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4270999999999995e-05 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.639224888, 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012520414000000001, 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118357+0000", 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118358+0000", 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3230000000000001e-06 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118368+0000", 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3899999999999999e-06 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2481999999999999e-05 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118410+0000", 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9304e-05 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126108+0000", 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076979409999999998 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126379+0000", 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027136700000000001 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126653+0000", 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274238 2024-04-25T12:36:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129707+0000", 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030541800000000001 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130686+0000", 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097908900000000004 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130791+0000", 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104824 2024-04-25T12:36:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130877+0000", 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6275999999999999e-05 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:38.868 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:38.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:38.871 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:38.871 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:38.872 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:38.872 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:38.872 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:38.872 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:38.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-04-25T12:36:38.872 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:38.871+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:38.971 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:38.972+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:39.071 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:39.072+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.171+0000 7f5586400640 1 Processor -- start 2024-04-25T12:36:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.171+0000 7f5586400640 1 -- start start 2024-04-25T12:36:39.171 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f5586400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f55800c88b0 0x7f558015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f5586400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5580161450 con 0x7f55800748a0 2024-04-25T12:36:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f5586400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55801616f0 con 0x7f55800c2720 2024-04-25T12:36:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f5586400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f55800c4030 con 0x7f55800c88b0 2024-04-25T12:36:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f557fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f55800748a0 0x7f55800c81a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55582/0 (socket says 172.21.15.123:55582) 2024-04-25T12:36:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f557fe00640 1 -- 172.21.15.123:0/558249569 learned_addr learned my addr 172.21.15.123:0/558249569 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.173+0000 7f5585a00640 1 --2- 172.21.15.123:0/558249569 >> v2:172.21.15.153:3300/0 conn(0x7f55800c88b0 0x7f558015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.174+0000 7f5585a00640 1 -- 172.21.15.123:0/558249569 >> v1:172.21.15.153:6789/0 conn(0x7f55800c2720 legacy=0x7f558006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.174+0000 7f5585a00640 1 -- 172.21.15.123:0/558249569 >> v1:172.21.15.123:6789/0 conn(0x7f55800748a0 legacy=0x7f55800c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.174+0000 7f5585a00640 1 -- 172.21.15.123:0/558249569 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55800c4310 con 0x7f55800c88b0 2024-04-25T12:36:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.174+0000 7f557f400640 1 -- 172.21.15.123:0/558249569 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120608583 0 0) 0x7f5580161450 con 0x7f55800748a0 2024-04-25T12:36:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.175+0000 7f5585a00640 1 --2- 172.21.15.123:0/558249569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55800c88b0 0x7f558015de60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557802a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7691b854d7b1502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.175+0000 7f557f400640 1 -- 172.21.15.123:0/558249569 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578007740 con 0x7f55800c88b0 2024-04-25T12:36:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.175+0000 7f557f400640 1 -- 172.21.15.123:0/558249569 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55780078e0 con 0x7f55800c88b0 2024-04-25T12:36:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.175+0000 7f557f400640 1 -- 172.21.15.123:0/558249569 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578007cb0 con 0x7f55800c88b0 2024-04-25T12:36:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.176+0000 7f5586400640 1 -- 172.21.15.123:0/558249569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55800c88b0 msgr2=0x7f558015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.176+0000 7f5586400640 1 --2- 172.21.15.123:0/558249569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55800c88b0 0x7f558015de60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557802a6a0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.176+0000 7f5586400640 1 -- 172.21.15.123:0/558249569 shutdown_connections 2024-04-25T12:36:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.176+0000 7f5586400640 1 --2- 172.21.15.123:0/558249569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55800c88b0 0x7f558015de60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.176+0000 7f5586400640 1 -- 172.21.15.123:0/558249569 >> 172.21.15.123:0/558249569 conn(0x7f5580070b90 msgr2=0x7f5580070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.177+0000 7f5586400640 1 -- 172.21.15.123:0/558249569 shutdown_connections 2024-04-25T12:36:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.177+0000 7f5586400640 1 -- 172.21.15.123:0/558249569 wait complete. 2024-04-25T12:36:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.179+0000 7f5586400640 1 Processor -- start 2024-04-25T12:36:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.179+0000 7f5586400640 1 -- start start 2024-04-25T12:36:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5586400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55800748a0 0x7f55801551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5586400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55800c2720 0x7f558014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5586400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55800c88b0 0x7f558014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5586400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f558006d040 con 0x7f55800c2720 2024-04-25T12:36:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5586400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f558006cec0 con 0x7f55800748a0 2024-04-25T12:36:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5586400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f55800742f0 con 0x7f55800c88b0 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f557fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55800c2720 0x7f558014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f557fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55800c2720 0x7f558014f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51364/0 (socket says 172.21.15.123:51364) 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f557fe00640 1 -- 172.21.15.123:0/4017961075 learned_addr learned my addr 172.21.15.123:0/4017961075 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5585000640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55800748a0 0x7f55801551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.181+0000 7f5585a00640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55800c88b0 0x7f558014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.182+0000 7f5585000640 1 -- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55800c88b0 msgr2=0x7f558014f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.182+0000 7f5585000640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55800c88b0 0x7f558014f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.182+0000 7f5585000640 1 -- 172.21.15.123:0/4017961075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55800c2720 msgr2=0x7f558014f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.182+0000 7f5585000640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55800c2720 0x7f558014f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.182+0000 7f5585000640 1 -- 172.21.15.123:0/4017961075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5580156300 con 0x7f55800748a0 2024-04-25T12:36:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.182+0000 7f5585000640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55800748a0 0x7f55801551c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f557400b3e0 tx=0x7f557400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.183+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574015030 con 0x7f55800748a0 2024-04-25T12:36:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.183+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5574004020 con 0x7f55800748a0 2024-04-25T12:36:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.183+0000 7f5586400640 1 -- 172.21.15.123:0/4017961075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5580155700 con 0x7f55800748a0 2024-04-25T12:36:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.183+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574004410 con 0x7f55800748a0 2024-04-25T12:36:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.183+0000 7f5586400640 1 -- 172.21.15.123:0/4017961075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55801546e0 con 0x7f55800748a0 2024-04-25T12:36:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.184+0000 7f5572000640 1 -- 172.21.15.123:0/4017961075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55801541b0 con 0x7f55800748a0 2024-04-25T12:36:39.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.189+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5574018070 con 0x7f55800748a0 2024-04-25T12:36:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.190+0000 7f557cc00640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55600744e0 0x7f55600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.190+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6685+0+0 (secure 0 0 0) 0x7f55740957e0 con 0x7f55800748a0 2024-04-25T12:36:39.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.190+0000 7f557cc00640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f556007de40 0x7f5560080330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.190+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5560080a00 con 0x7f556007de40 2024-04-25T12:36:39.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.190+0000 7f557fe00640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55600744e0 0x7f55600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.190+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f5574095b90 con 0x7f55800748a0 2024-04-25T12:36:39.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.191+0000 7f5585a00640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f556007de40 0x7f5560080330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.191+0000 7f557fe00640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55600744e0 0x7f55600769f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f556c000f60 tx=0x7f556c007ea0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.191+0000 7f5585a00640 1 --2- 172.21.15.123:0/4017961075 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f556007de40 0x7f5560080330 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.194+0000 7f557cc00640 1 -- 172.21.15.123:0/4017961075 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5560080a00 con 0x7f556007de40 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:39.220 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:39.221 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:39.221 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:39.221 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:39.224 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:39.239 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:39.239 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:39.240 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-04-25T12:36:39.260 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:39.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:39.260 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:39.260 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:39.273 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:39.274+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:39.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:39.373 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:39.374+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:39.474 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:39.475+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:39.574 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:39.575+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:39.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.597+0000 7f8b0f200640 1 Processor -- start 2024-04-25T12:36:39.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.597+0000 7f8b0f200640 1 -- start start 2024-04-25T12:36:39.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.600+0000 7f8b0f200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8b101543f0 0x7f8b100cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.600+0000 7f8b0f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b100d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.601+0000 7f8b0f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b1015b0b0 con 0x7f8b100b23b0 2024-04-25T12:36:39.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.601+0000 7f8b0f200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8b100cb370 con 0x7f8b101543f0 2024-04-25T12:36:39.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.601+0000 7f8b0f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8b100cc010 con 0x7f8b100c4500 2024-04-25T12:36:39.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.601+0000 7f8b0de00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8b101543f0 0x7f8b100cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.601+0000 7f8b0e800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8b100b23b0 0x7f8b100cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55596/0 (socket says 172.21.15.123:55596) 2024-04-25T12:36:39.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.601+0000 7f8b0e800640 1 -- 172.21.15.123:0/2331396357 learned_addr learned my addr 172.21.15.123:0/2331396357 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:39.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.601+0000 7f8b0d400640 1 --2- 172.21.15.123:0/2331396357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b100d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.602+0000 7f8b0ca00640 1 -- 172.21.15.123:0/2331396357 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405569087 0 0) 0x7f8b1015b0b0 con 0x7f8b100b23b0 2024-04-25T12:36:39.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.602+0000 7f8b0d400640 1 -- 172.21.15.123:0/2331396357 >> v2:172.21.15.153:3300/0 conn(0x7f8b101543f0 msgr2=0x7f8b100cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:39.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.602+0000 7f8b0d400640 1 --2- 172.21.15.123:0/2331396357 >> v2:172.21.15.153:3300/0 conn(0x7f8b101543f0 0x7f8b100cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.602+0000 7f8b0d400640 1 -- 172.21.15.123:0/2331396357 >> v1:172.21.15.123:6789/0 conn(0x7f8b100b23b0 legacy=0x7f8b100cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.602+0000 7f8b0d400640 1 -- 172.21.15.123:0/2331396357 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b100cadb0 con 0x7f8b100c4500 2024-04-25T12:36:39.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.602+0000 7f8b0de00640 1 --2- 172.21.15.123:0/2331396357 >> v2:172.21.15.153:3300/0 conn(0x7f8b101543f0 0x7f8b100cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:39.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.604+0000 7f8b0d400640 1 --2- 172.21.15.123:0/2331396357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b100d0090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0097a0 tx=0x7f8afc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9895b3413602bd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.604+0000 7f8b0ca00640 1 -- 172.21.15.123:0/2331396357 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc037070 con 0x7f8b100c4500 2024-04-25T12:36:39.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.604+0000 7f8b0ca00640 1 -- 172.21.15.123:0/2331396357 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8afc032440 con 0x7f8b100c4500 2024-04-25T12:36:39.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.604+0000 7f8b0ca00640 1 -- 172.21.15.123:0/2331396357 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc037070 con 0x7f8b100c4500 2024-04-25T12:36:39.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.605+0000 7f8b0f200640 1 -- 172.21.15.123:0/2331396357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 msgr2=0x7f8b100d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.605+0000 7f8b0f200640 1 --2- 172.21.15.123:0/2331396357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b100d0090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0097a0 tx=0x7f8afc02a250 comp rx=0 tx=0).stop 2024-04-25T12:36:39.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.606+0000 7f8b0f200640 1 -- 172.21.15.123:0/2331396357 shutdown_connections 2024-04-25T12:36:39.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.606+0000 7f8b0f200640 1 --2- 172.21.15.123:0/2331396357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b100d0090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.606+0000 7f8b0f200640 1 --2- 172.21.15.123:0/2331396357 >> v2:172.21.15.153:3300/0 conn(0x7f8b101543f0 0x7f8b100cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.606+0000 7f8b0f200640 1 -- 172.21.15.123:0/2331396357 >> 172.21.15.123:0/2331396357 conn(0x7f8b10068810 msgr2=0x7f8b10068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:39.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.606+0000 7f8b0f200640 1 -- 172.21.15.123:0/2331396357 shutdown_connections 2024-04-25T12:36:39.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.606+0000 7f8b0f200640 1 -- 172.21.15.123:0/2331396357 wait complete. 2024-04-25T12:36:39.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.610+0000 7f8b0f200640 1 Processor -- start 2024-04-25T12:36:39.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.610+0000 7f8b0f200640 1 -- start start 2024-04-25T12:36:39.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b100b23b0 0x7f8b1015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b101fa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b101543f0 0x7f8b1015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100cb590 con 0x7f8b101543f0 2024-04-25T12:36:39.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100cbca0 con 0x7f8b100b23b0 2024-04-25T12:36:39.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8b100cb710 con 0x7f8b100c4500 2024-04-25T12:36:39.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b100b23b0 0x7f8b1015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b101543f0 0x7f8b1015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.614+0000 7f8b0de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b100b23b0 0x7f8b1015b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:59046/0 (socket says 172.21.15.123:59046) 2024-04-25T12:36:39.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.614+0000 7f8b0de00640 1 -- 172.21.15.123:0/490690884 learned_addr learned my addr 172.21.15.123:0/490690884 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:39.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.613+0000 7f8b0d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b101fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.614+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 msgr2=0x7f8b101fa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.614+0000 7f8b0e800640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b100c4500 0x7f8b101fa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.614+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b100b23b0 msgr2=0x7f8b1015b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.614+0000 7f8b0e800640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b100b23b0 0x7f8b1015b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.614+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1015e220 con 0x7f8b101543f0 2024-04-25T12:36:39.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.615+0000 7f8b0e800640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b101543f0 0x7f8b1015be30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c750 tx=0x7f8b0800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.615+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08018070 con 0x7f8b101543f0 2024-04-25T12:36:39.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.615+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b08004d70 con 0x7f8b101543f0 2024-04-25T12:36:39.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.615+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800a880 con 0x7f8b101543f0 2024-04-25T12:36:39.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.615+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b101593b0 con 0x7f8b101543f0 2024-04-25T12:36:39.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.616+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1015bb70 con 0x7f8b101543f0 2024-04-25T12:36:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.617+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8b1015b9a0 con 0x7f8b101543f0 2024-04-25T12:36:39.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.619+0000 7fa15b200640 1 Processor -- start 2024-04-25T12:36:39.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.619+0000 7fa15b200640 1 -- start start 2024-04-25T12:36:39.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.620+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8b0800aa20 con 0x7f8b101543f0 2024-04-25T12:36:39.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.621+0000 7f8b06000640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8af00744b0 0x7f8af00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.621+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6685+0+0 (secure 0 0 0) 0x7f8b0805b2f0 con 0x7f8b101543f0 2024-04-25T12:36:39.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.621+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v63) ==== 54+0+16 (secure 0 0 0) 0x7f8b08067340 con 0x7f8b101543f0 2024-04-25T12:36:39.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.621+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8af00744b0 0x7f8af00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7fa15b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa154073460 0x7fa15415d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8b101fa750 0x7f8b101577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b10076da0 con 0x7f8b101fa750 2024-04-25T12:36:39.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7fa15b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa154073c10 con 0x7fa1540ca060 2024-04-25T12:36:39.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7fa15b200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa154074100 con 0x7fa1540b2a10 2024-04-25T12:36:39.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7fa15b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa15415d7b0 con 0x7fa154073460 2024-04-25T12:36:39.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7fa153400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa1540b2a10 0x7fa1540c3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:36190/0 (socket says 172.21.15.123:36190) 2024-04-25T12:36:39.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7fa153400640 1 -- 172.21.15.123:0/1339234598 learned_addr learned my addr 172.21.15.123:0/1339234598 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:39.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.622+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8af00744b0 0x7f8af00769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009af0 tx=0x7f8b00008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635873704 0 0) 0x7fa154074100 con 0x7fa1540b2a10 2024-04-25T12:36:39.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa138003520 con 0x7fa1540b2a10 2024-04-25T12:36:39.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7fa153e00640 1 --2- 172.21.15.123:0/1339234598 >> v2:172.21.15.123:3300/0 conn(0x7fa154073460 0x7fa15415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 759226929 0 0) 0x7fa138003520 con 0x7fa1540b2a10 2024-04-25T12:36:39.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 >> v2:172.21.15.123:3300/0 conn(0x7fa154073460 msgr2=0x7fa15415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:39.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7f8b0d400640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8b101fa750 0x7f8b101577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7fa152a00640 1 --2- 172.21.15.123:0/1339234598 >> v2:172.21.15.123:3300/0 conn(0x7fa154073460 0x7fa15415d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.623+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 >> v1:172.21.15.123:6789/0 conn(0x7fa1540ca060 legacy=0x7fa1540ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa154070a20 con 0x7fa1540b2a10 2024-04-25T12:36:39.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7fa140001670 con 0x7fa1540b2a10 2024-04-25T12:36:39.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 >> v1:172.21.15.153:6789/0 conn(0x7fa1540b2a10 legacy=0x7fa1540c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1380039b0 0x7fa138003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa138004800 0x7fa138004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa138005270 0x7fa13800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa15415d7b0 con 0x7fa138004800 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa13800c200 con 0x7fa1380039b0 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa13800c400 con 0x7fa138005270 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa153e00640 1 --2- 172.21.15.123:0/1339234598 >> v2:172.21.15.123:3300/0 conn(0x7fa154073460 0x7fa15415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa159200640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa138005270 0x7fa13800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.624+0000 7fa153e00640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1380039b0 0x7fa138003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa153400640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa138004800 0x7fa138004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa153e00640 1 -- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa138005270 msgr2=0x7fa13800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:39.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7f8b0d400640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8b101fa750 0x7f8b101577e0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa153e00640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa138005270 0x7fa13800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa153e00640 1 -- 172.21.15.123:0/1339234598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa138004800 msgr2=0x7fa138004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:39.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa153e00640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa138004800 0x7fa138004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa153e00640 1 -- 172.21.15.123:0/1339234598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa154070a20 con 0x7fa1380039b0 2024-04-25T12:36:39.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa159200640 1 -- 172.21.15.123:0/1339234598 reap_dead start 2024-04-25T12:36:39.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.625+0000 7fa153400640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa138004800 0x7fa138004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:39.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.626+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b10076da0 con 0x7f8b101fa750 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.627+0000 7fa153e00640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1380039b0 0x7fa138003e20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1440098a0 tx=0x7fa138005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99755a716936a243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.627+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f040 con 0x7fa1380039b0 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.627+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14402aca0 con 0x7fa1380039b0 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.627+0000 7fa152a00640 1 -- 172.21.15.123:0/1339234598 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144004250 con 0x7fa1380039b0 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.628+0000 7fa15b200640 1 -- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1380039b0 msgr2=0x7fa138003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.628+0000 7fa15b200640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1380039b0 0x7fa138003e20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1440098a0 tx=0x7fa138005830 comp rx=0 tx=0).stop 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.629+0000 7fa15b200640 1 -- 172.21.15.123:0/1339234598 shutdown_connections 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.629+0000 7fa15b200640 1 --2- 172.21.15.123:0/1339234598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1380039b0 0x7fa138003e20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.629+0000 7fa15b200640 1 -- 172.21.15.123:0/1339234598 >> 172.21.15.123:0/1339234598 conn(0x7fa15406f440 msgr2=0x7fa15406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.629+0000 7fa15b200640 1 -- 172.21.15.123:0/1339234598 shutdown_connections 2024-04-25T12:36:39.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.629+0000 7fa15b200640 1 -- 172.21.15.123:0/1339234598 wait complete. 2024-04-25T12:36:39.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.633+0000 7fa15b200640 1 Processor -- start 2024-04-25T12:36:39.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.633+0000 7fa15b200640 1 -- start start 2024-04-25T12:36:39.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.635+0000 7fa15b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1380039b0 0x7fa1541e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.635+0000 7fa15b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1540c78a0 0x7fa154155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.635+0000 7fa15b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa15414cfa0 0x7fa15414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.635+0000 7fa15b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa154071310 con 0x7fa1380039b0 2024-04-25T12:36:39.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.635+0000 7fa15b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa154156fb0 con 0x7fa1540c78a0 2024-04-25T12:36:39.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.635+0000 7fa15b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa154073930 con 0x7fa15414cfa0 2024-04-25T12:36:39.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.636+0000 7fa153e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1380039b0 0x7fa1541e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.636+0000 7fa153400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1540c78a0 0x7fa154155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.636+0000 7fa153400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1540c78a0 0x7fa154155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:59070/0 (socket says 172.21.15.123:59070) 2024-04-25T12:36:39.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.636+0000 7fa153400640 1 -- 172.21.15.123:0/3414461155 learned_addr learned my addr 172.21.15.123:0/3414461155 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:39.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.636+0000 7fa159200640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa15414cfa0 0x7fa15414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.637+0000 7fa153400640 1 -- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa15414cfa0 msgr2=0x7fa15414d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.637+0000 7fa153400640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa15414cfa0 0x7fa15414d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.637+0000 7fa153400640 1 -- 172.21.15.123:0/3414461155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1380039b0 msgr2=0x7fa1541e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.637+0000 7fa153400640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1380039b0 0x7fa1541e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.637+0000 7fa153400640 1 -- 172.21.15.123:0/3414461155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1541e6b20 con 0x7fa1540c78a0 2024-04-25T12:36:39.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.638+0000 7fa153400640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1540c78a0 0x7fa154155ad0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa140007290 tx=0x7fa140007750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.638+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14001a070 con 0x7fa1540c78a0 2024-04-25T12:36:39.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.638+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14000fa00 con 0x7fa1540c78a0 2024-04-25T12:36:39.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.638+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140007d50 con 0x7fa1540c78a0 2024-04-25T12:36:39.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.638+0000 7fa15b200640 1 -- 172.21.15.123:0/3414461155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa154157810 con 0x7fa1540c78a0 2024-04-25T12:36:39.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.638+0000 7fa15b200640 1 -- 172.21.15.123:0/3414461155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa154157dd0 con 0x7fa1540c78a0 2024-04-25T12:36:39.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.640+0000 7fa15b200640 1 -- 172.21.15.123:0/3414461155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1540c41d0 con 0x7fa1540c78a0 2024-04-25T12:36:39.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.642+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa14001d070 con 0x7fa1540c78a0 2024-04-25T12:36:39.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.644+0000 7fa14be00640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa134074500 0x7fa134076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.645+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6685+0+0 (secure 0 0 0) 0x7fa1400955a0 con 0x7fa1540c78a0 2024-04-25T12:36:39.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.645+0000 7fa153e00640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa134074500 0x7fa134076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.645+0000 7fa14be00640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa13407dee0 0x7fa1340803d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.645+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa134080aa0 con 0x7fa13407dee0 2024-04-25T12:36:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.646+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa140095950 con 0x7fa1540c78a0 2024-04-25T12:36:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.646+0000 7fa159200640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa13407dee0 0x7fa1340803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.646+0000 7fa153e00640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa134074500 0x7fa134076a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa144033000 tx=0x7fa14402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.647+0000 7fa159200640 1 --2- 172.21.15.123:0/3414461155 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fa13407dee0 0x7fa1340803d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.648+0000 7fa14be00640 1 -- 172.21.15.123:0/3414461155 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa134080aa0 con 0x7fa13407dee0 2024-04-25T12:36:39.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.658+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b100cb8e0 con 0x7f8b101fa750 2024-04-25T12:36:39.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.661+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b100cb8e0 con 0x7f8b101fa750 2024-04-25T12:36:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:39.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.661+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8b1006f1b0 0x7f8b1015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.661+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8b100cf230 con 0x7f8b1006f1b0 2024-04-25T12:36:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.662+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8b1006f1b0 0x7f8b1015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.663+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8b1006f1b0 0x7f8b1015c370 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.665+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b100cf230 con 0x7f8b1006f1b0 2024-04-25T12:36:39.676 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:39.677+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-25T12:36:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-25T12:36:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-25T12:36:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-25T12:36:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-25T12:36:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-25T12:36:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:bench [] [] [] [] 2024-04-25T12:36:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-25T12:36:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-25T12:36:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-25T12:36:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-25T12:36:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:Error EINVAL: invalid command 2024-04-25T12:36:39.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.695+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b100cef50 con 0x7f8b1006f1b0 2024-04-25T12:36:39.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.698+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b100cef50 con 0x7f8b1006f1b0 2024-04-25T12:36:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.699+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8b1015c8b0 0x7f8b101f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.699+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8b101f73d0 con 0x7f8b1015c8b0 2024-04-25T12:36:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.699+0000 7f8b0d400640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8b1015c8b0 0x7f8b101f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.700+0000 7f8b0d400640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8b1015c8b0 0x7f8b101f6e90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.702+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b101f73d0 con 0x7f8b1015c8b0 2024-04-25T12:36:39.703 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T12:36:39.703 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-25T12:36:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:39.717 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:39.717 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:39.717 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:39.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.720+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b10062fe0 con 0x7f8b1015c8b0 2024-04-25T12:36:39.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.723+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b10062fe0 con 0x7f8b1015c8b0 2024-04-25T12:36:39.722 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:39.722 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:39.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.723+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8b10063ac0 0x7f8b100cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.723+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8b10063ed0 con 0x7f8b10063ac0 2024-04-25T12:36:39.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.724+0000 7f8b0e800640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8b10063ac0 0x7f8b100cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.724+0000 7f8b0e800640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8b10063ac0 0x7f8b100cd4d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.725+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b10063ed0 con 0x7f8b10063ac0 2024-04-25T12:36:39.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.740+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b100cdce0 con 0x7f8b10063ac0 2024-04-25T12:36:39.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.742+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b100cdce0 con 0x7f8b10063ac0 2024-04-25T12:36:39.741 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.742+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8b100ce170 0x7f8b1002c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.742+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8b1020ca60 con 0x7f8b100ce170 2024-04-25T12:36:39.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.742+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8b100ce170 0x7f8b1002c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.743+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8b100ce170 0x7f8b1002c9d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.744+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b1020ca60 con 0x7f8b100ce170 2024-04-25T12:36:39.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:39.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.773+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b1002cf10 con 0x7f8b100ce170 2024-04-25T12:36:39.777 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:39.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.775+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b1002cf10 con 0x7f8b100ce170 2024-04-25T12:36:39.777 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:39.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.775+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8b1002d630 0x7f8b1002da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.775+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8b1020fbf0 con 0x7f8b1002d630 2024-04-25T12:36:39.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.776+0000 7f8b0d400640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8b1002d630 0x7f8b1002da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.776+0000 7f8b0d400640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8b1002d630 0x7f8b1002da40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.777+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b1020fbf0 con 0x7f8b1002d630 2024-04-25T12:36:39.778 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:39.778+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:39.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.806+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b1020d360 con 0x7f8b1002d630 2024-04-25T12:36:39.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.808+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b1020d360 con 0x7f8b1002d630 2024-04-25T12:36:39.807 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:39.808 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:39.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.808+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8b1020d9e0 0x7f8b1020fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.808+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8b10212e60 con 0x7f8b1020d9e0 2024-04-25T12:36:39.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.809+0000 7f8b0e800640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8b1020d9e0 0x7f8b1020fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.810+0000 7f8b0e800640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8b1020d9e0 0x7f8b1020fe00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.811+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b10212e60 con 0x7f8b1020d9e0 2024-04-25T12:36:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.831+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b10000950 con 0x7f8b1020d9e0 2024-04-25T12:36:39.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.834+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b10000950 con 0x7f8b1020d9e0 2024-04-25T12:36:39.833 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.834+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8b102120b0 0x7f8b10212540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.834+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8b10216230 con 0x7f8b102120b0 2024-04-25T12:36:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.835+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8b102120b0 0x7f8b10212540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.835+0000 7f8b0de00640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8b102120b0 0x7f8b10212540 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.836+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b10216230 con 0x7f8b102120b0 2024-04-25T12:36:39.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.852+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8b102150b0 con 0x7f8b102120b0 2024-04-25T12:36:39.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.854+0000 7f8b06000640 1 -- 172.21.15.123:0/490690884 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8b102150b0 con 0x7f8b102120b0 2024-04-25T12:36:39.853 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:39.854 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:39.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.854+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8b101fa750 msgr2=0x7f8b101577e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.854+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f8b101fa750 0x7f8b101577e0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8b1006f1b0 msgr2=0x7f8b1015c370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f8b1006f1b0 0x7f8b1015c370 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8b1015c8b0 msgr2=0x7f8b101f6e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f8b1015c8b0 0x7f8b101f6e90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8b10063ac0 msgr2=0x7f8b100cd4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f8b10063ac0 0x7f8b100cd4d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8b100ce170 msgr2=0x7f8b1002c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f8b100ce170 0x7f8b1002c9d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8b1002d630 msgr2=0x7f8b1002da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8b1002d630 0x7f8b1002da40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8b1020d9e0 msgr2=0x7f8b1020fe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f8b1020d9e0 0x7f8b1020fe00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8b102120b0 msgr2=0x7f8b10212540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f8b102120b0 0x7f8b10212540 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 reap_dead start 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 reap_dead start 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 reap_dead start 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 reap_dead start 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0e800640 1 -- 172.21.15.123:0/490690884 reap_dead start 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8af00744b0 msgr2=0x7f8af00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8af00744b0 0x7f8af00769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009af0 tx=0x7f8b00008040 comp rx=0 tx=0).stop 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b101543f0 msgr2=0x7f8b1015be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b101543f0 0x7f8b1015be30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c750 tx=0x7f8b0800cc10 comp rx=0 tx=0).stop 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 shutdown_connections 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8af00744b0 0x7f8af00769c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 --2- 172.21.15.123:0/490690884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b101543f0 0x7f8b1015be30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:39.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.855+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 >> 172.21.15.123:0/490690884 conn(0x7f8b10068810 msgr2=0x7f8b100cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:39.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.856+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 shutdown_connections 2024-04-25T12:36:39.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:39.856+0000 7f8b0f200640 1 -- 172.21.15.123:0/490690884 wait complete. 2024-04-25T12:36:39.877 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:39.878+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:39.978 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:39.979+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:40.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:36:40.052 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:40.052 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:40.079 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:40.080+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:40.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:40.179 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:40.180+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:40.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.278+0000 7fb7b4e00640 1 Processor -- start 2024-04-25T12:36:40.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.279+0000 7fb7b4e00640 1 -- start start 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.280+0000 7fb7b4e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 0x7fb7b006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.280+0000 7fb7b4e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb7b006cab0 0x7fb7b015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.280+0000 7fb7b4e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb7b015cfb0 0x7fb7b00c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.280+0000 7fb7b4e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b006ecd0 con 0x7fb7b006cab0 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.280+0000 7fb7b4e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b006efb0 con 0x7fb7b015cfb0 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.280+0000 7fb7b4e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b006f290 con 0x7fb7b01567b0 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.281+0000 7fb7afe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb7b015cfb0 0x7fb7b00c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.281+0000 7fb7aea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb7b006cab0 0x7fb7b015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.281+0000 7fb7afe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb7b015cfb0 0x7fb7b00c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:59082/0 (socket says 172.21.15.123:59082) 2024-04-25T12:36:40.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.281+0000 7fb7afe00640 1 -- 172.21.15.123:0/1910429978 learned_addr learned my addr 172.21.15.123:0/1910429978 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:40.281 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:40.280+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:40.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.281+0000 7fb7afe00640 1 -- 172.21.15.123:0/1910429978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 msgr2=0x7fb7b006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:40.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.281+0000 7fb7afe00640 1 --2- 172.21.15.123:0/1910429978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 0x7fb7b006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.282+0000 7fb7afe00640 1 -- 172.21.15.123:0/1910429978 >> v2:172.21.15.123:3300/0 conn(0x7fb7b006cab0 msgr2=0x7fb7b015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:40.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.282+0000 7fb7afe00640 1 --2- 172.21.15.123:0/1910429978 >> v2:172.21.15.123:3300/0 conn(0x7fb7b006cab0 0x7fb7b015ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.282+0000 7fb7afe00640 1 -- 172.21.15.123:0/1910429978 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b006f570 con 0x7fb7b015cfb0 2024-04-25T12:36:40.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.282+0000 7fb7aea00640 1 --2- 172.21.15.123:0/1910429978 >> v2:172.21.15.123:3300/0 conn(0x7fb7b006cab0 0x7fb7b015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:40.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.283+0000 7fb7afe00640 1 --2- 172.21.15.123:0/1910429978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b015cfb0 0x7fb7b00c82b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ce4b0 tx=0x7fb7a802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a230edf5ee52cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:40.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.283+0000 7fb7ae000640 1 -- 172.21.15.123:0/1910429978 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8036070 con 0x7fb7b015cfb0 2024-04-25T12:36:40.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.283+0000 7fb7ae000640 1 -- 172.21.15.123:0/1910429978 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a8031600 con 0x7fb7b015cfb0 2024-04-25T12:36:40.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.283+0000 7fb7ae000640 1 -- 172.21.15.123:0/1910429978 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a80319d0 con 0x7fb7b015cfb0 2024-04-25T12:36:40.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.284+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1910429978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b015cfb0 msgr2=0x7fb7b00c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:40.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.284+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1910429978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b015cfb0 0x7fb7b00c82b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ce4b0 tx=0x7fb7a802a730 comp rx=0 tx=0).stop 2024-04-25T12:36:40.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.285+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1910429978 shutdown_connections 2024-04-25T12:36:40.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.285+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1910429978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b015cfb0 0x7fb7b00c82b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.285+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1910429978 >> v2:172.21.15.123:3300/0 conn(0x7fb7b006cab0 0x7fb7b015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.285+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1910429978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 0x7fb7b006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.285+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1910429978 >> 172.21.15.123:0/1910429978 conn(0x7fb7b00cd600 msgr2=0x7fb7b00cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:40.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.285+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1910429978 shutdown_connections 2024-04-25T12:36:40.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.285+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1910429978 wait complete. 2024-04-25T12:36:40.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.287+0000 7fb7b4e00640 1 Processor -- start 2024-04-25T12:36:40.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.288+0000 7fb7b4e00640 1 -- start start 2024-04-25T12:36:40.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.289+0000 7fb7b4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b006cab0 0x7fb7b01f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:40.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.289+0000 7fb7b4e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 0x7fb7b0155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:40.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.289+0000 7fb7b4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7b015cfb0 0x7fb7b014aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:40.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.289+0000 7fb7b4e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00c9d20 con 0x7fb7b015cfb0 2024-04-25T12:36:40.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.289+0000 7fb7b4e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b015d3c0 con 0x7fb7b006cab0 2024-04-25T12:36:40.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.289+0000 7fb7b4e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b0156bc0 con 0x7fb7b01567b0 2024-04-25T12:36:40.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.289+0000 7fb7af400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b006cab0 0x7fb7b01f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:40.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7af400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b006cab0 0x7fb7b01f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:59098/0 (socket says 172.21.15.123:59098) 2024-04-25T12:36:40.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7af400640 1 -- 172.21.15.123:0/1125514032 learned_addr learned my addr 172.21.15.123:0/1125514032 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:40.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7aea00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 0x7fb7b0155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:40.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7afe00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7b015cfb0 0x7fb7b014aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7af400640 1 -- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 msgr2=0x7fb7b0155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7af400640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 0x7fb7b0155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7af400640 1 -- 172.21.15.123:0/1125514032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7b015cfb0 msgr2=0x7fb7b014aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7af400640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7b015cfb0 0x7fb7b014aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7af400640 1 -- 172.21.15.123:0/1125514032 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b01f30c0 con 0x7fb7b006cab0 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.290+0000 7fb7afe00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7b015cfb0 0x7fb7b014aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.291+0000 7fb7af400640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b006cab0 0x7fb7b01f2b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b440 tx=0x7fb7a000b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.291+0000 7fb7a7400640 1 -- 172.21.15.123:0/1125514032 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0015020 con 0x7fb7b006cab0 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.291+0000 7fb7a7400640 1 -- 172.21.15.123:0/1125514032 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a0011890 con 0x7fb7b006cab0 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.291+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b0154850 con 0x7fb7b006cab0 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.291+0000 7fb7a7400640 1 -- 172.21.15.123:0/1125514032 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0011c80 con 0x7fb7b006cab0 2024-04-25T12:36:40.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.291+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b01f32d0 con 0x7fb7b006cab0 2024-04-25T12:36:40.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.293+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b0070470 con 0x7fb7b006cab0 2024-04-25T12:36:40.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.294+0000 7fb7a7400640 1 -- 172.21.15.123:0/1125514032 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7a0018020 con 0x7fb7b006cab0 2024-04-25T12:36:40.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.295+0000 7fb7a7400640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb7900744e0 0x7fb7900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:40.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.296+0000 7fb7a7400640 1 -- 172.21.15.123:0/1125514032 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6685+0+0 (secure 0 0 0) 0x7fb7a00957c0 con 0x7fb7b006cab0 2024-04-25T12:36:40.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.296+0000 7fb7aea00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb7900744e0 0x7fb7900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:40.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.296+0000 7fb7aea00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb7900744e0 0x7fb7900769f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00b2190 tx=0x7fb79c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:40.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.300+0000 7fb7a7400640 1 -- 172.21.15.123:0/1125514032 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7b0070470 con 0x7fb7b006cab0 2024-04-25T12:36:40.381 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:40.382+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:40.482 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:40.483+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:40.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.505+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fb7b0076da0 con 0x7fb7b006cab0 2024-04-25T12:36:40.521 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:40.521 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:40.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:40.594 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:40.595+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:40.684 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:40.685+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:40.785 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:40.785+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:40.885 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:40.886+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:40.890 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:40.890 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:40.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.985+0000 7fb7a7400640 1 -- 172.21.15.123:0/1125514032 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v64) ==== 97+0+0 (secure 0 0 0) 0x7fb7a001f020 con 0x7fb7b006cab0 2024-04-25T12:36:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:36:40.986 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:40.986+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb7900744e0 msgr2=0x7fb7900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb7900744e0 0x7fb7900769f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00b2190 tx=0x7fb79c008040 comp rx=0 tx=0).stop 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b006cab0 msgr2=0x7fb7b01f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b006cab0 0x7fb7b01f2b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b440 tx=0x7fb7a000b900 comp rx=0 tx=0).stop 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 shutdown_connections 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb7900744e0 0x7fb7900769f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7b015cfb0 0x7fb7b014aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7b01567b0 0x7fb7b0155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 --2- 172.21.15.123:0/1125514032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7b006cab0 0x7fb7b01f2b80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 >> 172.21.15.123:0/1125514032 conn(0x7fb7b00cd600 msgr2=0x7fb7b014d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.989+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 shutdown_connections 2024-04-25T12:36:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:40.990+0000 7fb7b4e00640 1 -- 172.21.15.123:0/1125514032 wait complete. 2024-04-25T12:36:41.087 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:41.087+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:41.188 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:41.188+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:41.278 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:41.279 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:41.288 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:41.289+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:41.311 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:41.389 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:41.390+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:41.436 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:41.436 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:41.446 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:41.489 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:41.491+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:41.590 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:41.591+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:41.598 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:41.598 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:41.608 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:41.692 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:41.693+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:41.759 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:41.759 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:41.768 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:41.792 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:41.793+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:41.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:41.920 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:41.920 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:41.928 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:41.929+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:41.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:41.994 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:41.995+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:42.044 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:42.045 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:42.054 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:42.095 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:42.096+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:42.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.159+0000 7f34ed600640 1 Processor -- start 2024-04-25T12:36:42.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.159+0000 7f34ed600640 1 -- start start 2024-04-25T12:36:42.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.161+0000 7f34ed600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34e815ce60 0x7f34e806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.161+0000 7f34ed600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34e806e0b0 0x7f34e8166d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.161+0000 7f34ed600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e8072010 con 0x7f34e8156760 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.161+0000 7f34ed600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f34e806d6e0 con 0x7f34e815ce60 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.161+0000 7f34ed600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f34e8156b70 con 0x7f34e806e0b0 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34ecc00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34e806e0b0 0x7f34e8166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.161+0000 7f34e7e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f34e8156760 0x7f34e80c4f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:36194/0 (socket says 172.21.15.123:36194) 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34e7e00640 1 -- 172.21.15.123:0/2263811714 learned_addr learned my addr 172.21.15.123:0/2263811714 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:42.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34e7400640 1 --2- 172.21.15.123:0/2263811714 >> v2:172.21.15.123:3300/0 conn(0x7f34e815ce60 0x7f34e806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34e6a00640 1 -- 172.21.15.123:0/2263811714 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490083133 0 0) 0x7f34e8072010 con 0x7f34e8156760 2024-04-25T12:36:42.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34e6a00640 1 -- 172.21.15.123:0/2263811714 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34cc003520 con 0x7f34e8156760 2024-04-25T12:36:42.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34ecc00640 1 -- 172.21.15.123:0/2263811714 >> v2:172.21.15.123:3300/0 conn(0x7f34e815ce60 msgr2=0x7f34e806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:42.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2263811714 >> v2:172.21.15.123:3300/0 conn(0x7f34e815ce60 0x7f34e806db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34ecc00640 1 -- 172.21.15.123:0/2263811714 >> v1:172.21.15.153:6789/0 conn(0x7f34e8156760 legacy=0x7f34e80c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.162+0000 7f34ecc00640 1 -- 172.21.15.123:0/2263811714 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e80cae10 con 0x7f34e806e0b0 2024-04-25T12:36:42.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.163+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2263811714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e806e0b0 0x7f34e8166d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009d90 tx=0x7f34e002a740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e32713e7a87ef28e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.163+0000 7f34e6a00640 1 -- 172.21.15.123:0/2263811714 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0007740 con 0x7f34e806e0b0 2024-04-25T12:36:42.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.163+0000 7f34e6a00640 1 -- 172.21.15.123:0/2263811714 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e00078e0 con 0x7f34e806e0b0 2024-04-25T12:36:42.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.163+0000 7f34e6a00640 1 -- 172.21.15.123:0/2263811714 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0007cb0 con 0x7f34e806e0b0 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.164+0000 7f34ed600640 1 -- 172.21.15.123:0/2263811714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e806e0b0 msgr2=0x7f34e8166d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.164+0000 7f34ed600640 1 --2- 172.21.15.123:0/2263811714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e806e0b0 0x7f34e8166d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009d90 tx=0x7f34e002a740 comp rx=0 tx=0).stop 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.164+0000 7f34ed600640 1 -- 172.21.15.123:0/2263811714 shutdown_connections 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.164+0000 7f34ed600640 1 --2- 172.21.15.123:0/2263811714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e806e0b0 0x7f34e8166d90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.164+0000 7f34ed600640 1 --2- 172.21.15.123:0/2263811714 >> v2:172.21.15.123:3300/0 conn(0x7f34e815ce60 0x7f34e806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.164+0000 7f34ed600640 1 -- 172.21.15.123:0/2263811714 >> 172.21.15.123:0/2263811714 conn(0x7f34e8074520 msgr2=0x7f34e80ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.164+0000 7f34ed600640 1 -- 172.21.15.123:0/2263811714 shutdown_connections 2024-04-25T12:36:42.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.165+0000 7f34ed600640 1 -- 172.21.15.123:0/2263811714 wait complete. 2024-04-25T12:36:42.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.167+0000 7f34ed600640 1 Processor -- start 2024-04-25T12:36:42.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.167+0000 7f34ed600640 1 -- start start 2024-04-25T12:36:42.166 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:42.166 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.168+0000 7f34ed600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e806e0b0 0x7f34e80cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.168+0000 7f34ed600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e8156760 0x7f34e81521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.168+0000 7f34ed600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34e815ce60 0x7f34e8151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.168+0000 7f34ed600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8071a50 con 0x7f34e806e0b0 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.168+0000 7f34ed600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8071770 con 0x7f34e8156760 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.168+0000 7f34ed600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f34e8071d30 con 0x7f34e815ce60 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e806e0b0 0x7f34e80cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e8156760 0x7f34e81521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e8156760 0x7f34e81521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:59110/0 (socket says 172.21.15.123:59110) 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 -- 172.21.15.123:0/2173744120 learned_addr learned my addr 172.21.15.123:0/2173744120 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e806e0b0 0x7f34e80cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51438/0 (socket says 172.21.15.123:51438) 2024-04-25T12:36:42.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34e815ce60 0x7f34e8151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34e815ce60 msgr2=0x7f34e8151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34e815ce60 0x7f34e8151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e806e0b0 msgr2=0x7f34e80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e806e0b0 0x7f34e80cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e814f350 con 0x7f34e8156760 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.169+0000 7f34e7400640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e8156760 0x7f34e81521b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f34d400e870 tx=0x7f34d400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.170+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4017070 con 0x7f34e8156760 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.170+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d400c8e0 con 0x7f34e8156760 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.170+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e814f610 con 0x7f34e8156760 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.170+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e8150850 con 0x7f34e8156760 2024-04-25T12:36:42.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.170+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d400ccd0 con 0x7f34e8156760 2024-04-25T12:36:42.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.171+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f34c0005270 con 0x7f34e8156760 2024-04-25T12:36:42.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.173+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34d401a070 con 0x7f34e8156760 2024-04-25T12:36:42.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.174+0000 7f34dfe00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f34c80744b0 0x7f34c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.174+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6685+0+0 (secure 0 0 0) 0x7f34d403ce00 con 0x7f34e8156760 2024-04-25T12:36:42.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.174+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v64) ==== 54+0+16 (secure 0 0 0) 0x7f34d4095c20 con 0x7f34e8156760 2024-04-25T12:36:42.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.174+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f34c00045e0 0x7f34c00049f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.174+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34c0005550 con 0x7f34c00045e0 2024-04-25T12:36:42.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.174+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f34c80744b0 0x7f34c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.174+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f34c00045e0 0x7f34c00049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.175+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f34c80744b0 0x7f34c80769c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005880 tx=0x7f34d8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.175+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f34c00045e0 0x7f34c00049f0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.176+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c0005550 con 0x7f34c00045e0 2024-04-25T12:36:42.176 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:42.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.191+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c0001270 con 0x7f34c00045e0 2024-04-25T12:36:42.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.193+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c0001270 con 0x7f34c00045e0 2024-04-25T12:36:42.193 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:42.193 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:42.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.194+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f34c0001d30 0x7f34c0002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.194+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f34c0002270 con 0x7f34c0001d30 2024-04-25T12:36:42.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.194+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f34c0001d30 0x7f34c0002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.195+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f34c0001d30 0x7f34c0002990 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.196+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c0002270 con 0x7f34c0001d30 2024-04-25T12:36:42.219 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:42.220+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.223+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c00031a0 con 0x7f34c0001d30 2024-04-25T12:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.227+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c00031a0 con 0x7f34c0001d30 2024-04-25T12:36:42.226 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.227+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f34c0009c80 0x7f34c000c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.227+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f34c000ee70 con 0x7f34c0009c80 2024-04-25T12:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.228+0000 7f34e7400640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f34c0009c80 0x7f34c000c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.229+0000 7f34e7400640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f34c0009c80 0x7f34c000c0a0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.231+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c000ee70 con 0x7f34c0009c80 2024-04-25T12:36:42.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.245+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c000c8b0 con 0x7f34c0009c80 2024-04-25T12:36:42.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.250+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c000c8b0 con 0x7f34c0009c80 2024-04-25T12:36:42.249 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:42.250 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:42.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.250+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f34c000d370 0x7f34c000d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.250+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f34c000dfd0 con 0x7f34c000d370 2024-04-25T12:36:42.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.251+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f34c000d370 0x7f34c000d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.252+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f34c000d370 0x7f34c000d7c0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.253+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c000dfd0 con 0x7f34c000d370 2024-04-25T12:36:42.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.268+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c000e290 con 0x7f34c000d370 2024-04-25T12:36:42.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.271+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c000e290 con 0x7f34c000d370 2024-04-25T12:36:42.270 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:42.271 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:42.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.271+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f34c000e9e0 0x7f34c0013040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.271+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f34c0015fc0 con 0x7f34c000e9e0 2024-04-25T12:36:42.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.272+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f34c000e9e0 0x7f34c0013040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.272+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f34c000e9e0 0x7f34c0013040 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.273+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c0015fc0 con 0x7f34c000e9e0 2024-04-25T12:36:42.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.288+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c0015fc0 con 0x7f34c000e9e0 2024-04-25T12:36:42.287 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:42.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.290+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c0015fc0 con 0x7f34c000e9e0 2024-04-25T12:36:42.289 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:42.290 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:42.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.290+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f34c00152f0 0x7f34c0015700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.291+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f34c0019390 con 0x7f34c00152f0 2024-04-25T12:36:42.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.291+0000 7f34e7400640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f34c00152f0 0x7f34c0015700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.291+0000 7f34e7400640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f34c00152f0 0x7f34c0015700 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.292+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c0019390 con 0x7f34c00152f0 2024-04-25T12:36:42.296 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:42.297+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:42.297 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:42.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.307+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c0018210 con 0x7f34c00152f0 2024-04-25T12:36:42.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.309+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c0018210 con 0x7f34c00152f0 2024-04-25T12:36:42.309 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:42.309 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:42.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.310+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f34c0018930 0x7f34c0018d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.310+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f34c001c7b0 con 0x7f34c0018930 2024-04-25T12:36:42.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.310+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f34c0018930 0x7f34c0018d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.311+0000 7f34ecc00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f34c0018930 0x7f34c0018d60 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.312+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c001c7b0 con 0x7f34c0018930 2024-04-25T12:36:42.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.327+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c0013620 con 0x7f34c0018930 2024-04-25T12:36:42.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.329+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c0013620 con 0x7f34c0018930 2024-04-25T12:36:42.328 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:42.329 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:42.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.329+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f34c0013b30 0x7f34c0013f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:42.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.329+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f34c0014500 con 0x7f34c0013b30 2024-04-25T12:36:42.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.330+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f34c0013b30 0x7f34c0013f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:42.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.330+0000 7f34e7e00640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f34c0013b30 0x7f34c0013f40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:42.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.331+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34c0014500 con 0x7f34c0013b30 2024-04-25T12:36:42.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.346+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f34c0014740 con 0x7f34c0013b30 2024-04-25T12:36:42.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.348+0000 7f34dfe00640 1 -- 172.21.15.123:0/2173744120 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f34c0014740 con 0x7f34c0013b30 2024-04-25T12:36:42.347 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:42.349 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:42.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f34c00045e0 msgr2=0x7f34c00049f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f34c00045e0 0x7f34c00049f0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f34c0001d30 msgr2=0x7f34c0002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f34c0001d30 0x7f34c0002990 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f34c0009c80 msgr2=0x7f34c000c0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f34c0009c80 0x7f34c000c0a0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f34c000d370 msgr2=0x7f34c000d7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f34c000d370 0x7f34c000d7c0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f34c000e9e0 msgr2=0x7f34c0013040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f34c000e9e0 0x7f34c0013040 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f34c00152f0 msgr2=0x7f34c0015700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f34c00152f0 0x7f34c0015700 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f34c0018930 msgr2=0x7f34c0018d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f34c0018930 0x7f34c0018d60 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f34c0013b30 msgr2=0x7f34c0013f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f34c0013b30 0x7f34c0013f40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ecc00640 1 -- 172.21.15.123:0/2173744120 reap_dead start 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ecc00640 1 -- 172.21.15.123:0/2173744120 reap_dead start 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ecc00640 1 -- 172.21.15.123:0/2173744120 reap_dead start 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ecc00640 1 -- 172.21.15.123:0/2173744120 reap_dead start 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ecc00640 1 -- 172.21.15.123:0/2173744120 reap_dead start 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f34c80744b0 msgr2=0x7f34c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f34c80744b0 0x7f34c80769c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005880 tx=0x7f34d8009040 comp rx=0 tx=0).stop 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e8156760 msgr2=0x7f34e81521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e8156760 0x7f34e81521b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f34d400e870 tx=0x7f34d400ed30 comp rx=0 tx=0).stop 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 shutdown_connections 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f34c80744b0 0x7f34c80769c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 --2- 172.21.15.123:0/2173744120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e8156760 0x7f34e81521b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.349+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 >> 172.21.15.123:0/2173744120 conn(0x7f34e8074520 msgr2=0x7f34e8153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.350+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 shutdown_connections 2024-04-25T12:36:42.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:42.350+0000 7f34ed600640 1 -- 172.21.15.123:0/2173744120 wait complete. 2024-04-25T12:36:42.396 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:42.397+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:42.459 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:42.500 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:42.501+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:42.599 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:42.600+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 143.660768977, 2024-04-25T12:36:42.612 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.613 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:36:42.614 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:36:42.615 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:36:42.616 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:36:42.617 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.618 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 143.66068132300001, 2024-04-25T12:36:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.620 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:36:42.621 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:36:42.622 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:36:42.623 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:36:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 143.660336755, 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.625 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.626 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:42.626 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.626 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.626 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.626 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.626 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:42.626 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.627 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.628 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:36:42.629 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.630 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 143.66026428500001, 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.631 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:36:42.632 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.633 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.634 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.635 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.636 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 143.66019030699999, 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.637 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.638 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.639 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:36:42.640 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.641 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.642 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 143.658152935, 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.643 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:36:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:36:42.645 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.646 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:42.647 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 143.657292799, 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:42.648 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.649 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.650 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:36:42.651 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.496607144, 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.652 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.653 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:36:42.654 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:42.655 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.656 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:42.657 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.49653103, 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:42.658 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.659 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.660 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.661 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.496485361, 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:42.663 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.664 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.665 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.666 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.667 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.49644058600001, 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.668 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.669 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.670 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.671 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.48982416200001, 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.672 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.673 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:36:42.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:36:42.675 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.676 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.48976467200001, 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:42.677 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.678 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.679 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.680 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 107.45879438, 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.681 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.682 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.683 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.684 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.685 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 107.458716869, 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:36:42.686 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.687 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:36:42.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:36:42.689 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.690 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:36:42.691 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 107.45866902, 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.692 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:42.693 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.695 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:36:42.696 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.697 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 107.45862713, 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:42.698 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.699 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:36:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.702 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:42.703 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 107.458592958, 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:36:42.705 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:36:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:42.707 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.708 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 107.458310216, 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.709 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:36:42.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.711 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:36:42.712 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 107.454169785, 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:42.713 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:36:42.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:42.716 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:42.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:42.718 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:42.718+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:42.818 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:42.819+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:42.871 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:42.871 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:42.871 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:42.872 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:42.881 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:42.920 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:42.921+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:43.020 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:43.021+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:43.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:36:43.034 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:43.034 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:43.034 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:43.034 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:43.034 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:43.043 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:43.121 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:43.122+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:43.155 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:43.155 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:43.155 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:43.155 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [] 2024-04-25T12:36:43.155 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:43.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:43.222 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:43.223+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:43.322 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:43.323+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:43.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.288+0000 7fabc1000640 1 Processor -- start 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.288+0000 7fabc1000640 1 -- start start 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabc1000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabbc0c2870 0x7fabbc16d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabc1000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabbc169220 0x7fabbc169630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabc1000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabbc073d50 con 0x7fabbc0c4240 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabc1000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fabbc16d6e0 con 0x7fabbc0c2870 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabc1000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fabbc074640 con 0x7fabbc169220 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabb9400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabbc0c2870 0x7fabbc16d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabba800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabbc169220 0x7fabbc169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabb9e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fabbc0c4240 0x7fabbc0c9a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:49368/0 (socket says 172.21.15.123:49368) 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.290+0000 7fabb9e00640 1 -- 172.21.15.123:0/2967790323 learned_addr learned my addr 172.21.15.123:0/2967790323 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.291+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852767987 0 0) 0x7fabbc073d50 con 0x7fabbc0c4240 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.389+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faba0003500 con 0x7fabbc0c4240 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1275157889 0 0) 0x7faba0003500 con 0x7fabbc0c4240 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabbc169220 msgr2=0x7fabbc169630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fabbc169220 0x7fabbc169630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 >> v2:172.21.15.153:3300/0 conn(0x7fabbc0c2870 msgr2=0x7fabbc16d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 --2- 172.21.15.123:0/2967790323 >> v2:172.21.15.153:3300/0 conn(0x7fabbc0c2870 0x7fabbc16d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc0c4710 con 0x7fabbc0c4240 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7fabac001670 con 0x7fabbc0c4240 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 >> v1:172.21.15.153:6789/0 conn(0x7fabbc0c4240 legacy=0x7fabbc0c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba0003b10 0x7faba0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba00045f0 0x7faba0004ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7faba000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc16d6e0 con 0x7faba0003b10 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc074640 con 0x7faba00045f0 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faba000c020 con 0x7faba0005130 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabb9400640 1 --2- 172.21.15.123:0/2967790323 >> v2:172.21.15.153:3300/0 conn(0x7fabbc0c2870 0x7fabbc16d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.390+0000 7fabba800640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7faba000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabb9400640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba00045f0 0x7faba0004ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabb9e00640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba0003b10 0x7faba0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabba800640 1 -- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba00045f0 msgr2=0x7faba0004ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabba800640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba00045f0 0x7faba0004ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabba800640 1 -- 172.21.15.123:0/2967790323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba0003b10 msgr2=0x7faba0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabba800640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba0003b10 0x7faba0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabba800640 1 -- 172.21.15.123:0/2967790323 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba0003500 con 0x7faba0005130 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.391+0000 7fabb9e00640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba0003b10 0x7faba0003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.393+0000 7fabba800640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7faba000b9d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fabbc06e700 tx=0x7faba0003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0d1a2b3030f339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.393+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400b4c0 con 0x7faba0005130 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.393+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb400d2d0 con 0x7faba0005130 2024-04-25T12:36:43.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.393+0000 7fabb8a00640 1 -- 172.21.15.123:0/2967790323 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400b740 con 0x7faba0005130 2024-04-25T12:36:43.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.395+0000 7fabc1000640 1 -- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 msgr2=0x7faba000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:43.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.395+0000 7fabc1000640 1 --2- 172.21.15.123:0/2967790323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7faba000b9d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fabbc06e700 tx=0x7faba0003890 comp rx=0 tx=0).stop 2024-04-25T12:36:43.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.395+0000 7fabba800640 1 -- 172.21.15.123:0/2967790323 reap_dead start 2024-04-25T12:36:43.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.395+0000 7fabc1000640 1 -- 172.21.15.123:0/2967790323 shutdown_connections 2024-04-25T12:36:43.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.395+0000 7fabc1000640 1 -- 172.21.15.123:0/2967790323 >> 172.21.15.123:0/2967790323 conn(0x7fabbc06c750 msgr2=0x7fabbc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:43.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.395+0000 7fabc1000640 1 -- 172.21.15.123:0/2967790323 shutdown_connections 2024-04-25T12:36:43.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.395+0000 7fabc1000640 1 -- 172.21.15.123:0/2967790323 wait complete. 2024-04-25T12:36:43.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.402+0000 7fabc1000640 1 Processor -- start 2024-04-25T12:36:43.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.402+0000 7fabc1000640 1 -- start start 2024-04-25T12:36:43.404 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:43.404 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:43.404 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:43.404 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.406+0000 7fabc1000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7fabbc0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.406+0000 7fabc1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc0c4c00 0x7fabbc0c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.406+0000 7fabc1000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1ff100 0x7fabbc15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.406+0000 7fabc1000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc0c49f0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.406+0000 7fabc1000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc0742d0 con 0x7fabbc1ff100 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.406+0000 7fabc1000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fabbc074ad0 con 0x7faba0005130 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.406+0000 7fabb9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc0c4c00 0x7fabbc0c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc0c4c00 0x7fabbc0c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48650/0 (socket says 172.21.15.123:48650) 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7fabbc0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9400640 1 -- 172.21.15.123:0/419393781 learned_addr learned my addr 172.21.15.123:0/419393781 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:43.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabba800640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1ff100 0x7fabbc15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9400640 1 -- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 msgr2=0x7fabbc0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9400640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7fabbc0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9400640 1 -- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1ff100 msgr2=0x7fabbc15ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9400640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1ff100 0x7fabbc15ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9400640 1 -- 172.21.15.123:0/419393781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc1f4870 con 0x7fabbc0c4c00 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.407+0000 7fabb9e00640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7fabbc0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.408+0000 7fabb9400640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc0c4c00 0x7fabbc0c8a80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faba800a680 tx=0x7faba800ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.408+0000 7fabb2000640 1 -- 172.21.15.123:0/419393781 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8004230 con 0x7fabbc0c4c00 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.408+0000 7fabb2000640 1 -- 172.21.15.123:0/419393781 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba80043f0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.408+0000 7fabb2000640 1 -- 172.21.15.123:0/419393781 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba801a9d0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.408+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc1f4b10 con 0x7fabbc0c4c00 2024-04-25T12:36:43.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.408+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1f5090 con 0x7fabbc0c4c00 2024-04-25T12:36:43.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.410+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc0c3cf0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.412+0000 7fabb2000640 1 -- 172.21.15.123:0/419393781 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faba801ab70 con 0x7fabbc0c4c00 2024-04-25T12:36:43.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.413+0000 7fabb2000640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab9c0744e0 0x7fab9c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.413+0000 7fabb2000640 1 -- 172.21.15.123:0/419393781 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6685+0+0 (secure 0 0 0) 0x7faba809edd0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.414+0000 7fabb9e00640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab9c0744e0 0x7fab9c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.414+0000 7fabb9e00640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab9c0744e0 0x7fab9c0769f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabac001eb0 tx=0x7fabac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:43.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.418+0000 7fabb2000640 1 -- 172.21.15.123:0/419393781 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fabbc0c3cf0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.424 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:43.425+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:43.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:43.524 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:43.525+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:43.625 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:43.626+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:43.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:43.634+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fabbc0b35f0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.727 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:43.728+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:43.761 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:43.761 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:43.761 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:43.761 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:43.762 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:43.782 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:43.827 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:43.828+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:43.929 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:43.929+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:43.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.000+0000 7fabb2000640 1 -- 172.21.15.123:0/419393781 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v65) ==== 91+0+0 (secure 0 0 0) 0x7fabbc0b35f0 con 0x7fabbc0c4c00 2024-04-25T12:36:43.999 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:36:44.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.005+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab9c0744e0 msgr2=0x7fab9c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.005+0000 7fabc1000640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab9c0744e0 0x7fab9c0769f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabac001eb0 tx=0x7fabac009040 comp rx=0 tx=0).stop 2024-04-25T12:36:44.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.005+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc0c4c00 msgr2=0x7fabbc0c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.005+0000 7fabc1000640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc0c4c00 0x7fabbc0c8a80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faba800a680 tx=0x7faba800ab40 comp rx=0 tx=0).stop 2024-04-25T12:36:44.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.006+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 shutdown_connections 2024-04-25T12:36:44.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.006+0000 7fabc1000640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fab9c0744e0 0x7fab9c0769f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.006+0000 7fabc1000640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabbc1ff100 0x7fabbc15ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.006+0000 7fabc1000640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabbc0c4c00 0x7fabbc0c8a80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.006+0000 7fabc1000640 1 --2- 172.21.15.123:0/419393781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faba0005130 0x7fabbc0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.006+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 >> 172.21.15.123:0/419393781 conn(0x7fabbc06c750 msgr2=0x7fabbc167be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:44.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.010+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 shutdown_connections 2024-04-25T12:36:44.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.011+0000 7fabc1000640 1 -- 172.21.15.123:0/419393781 wait complete. 2024-04-25T12:36:44.030 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:44.031+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 145.14203419699999, 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012628781, 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951635+0000", 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951638+0000", 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951659+0000", 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1027999999999999e-05 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951691+0000", 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2267000000000001e-05 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951710+0000", 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9063999999999999e-05 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952026+0000", 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000315719 2024-04-25T12:36:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952367+0000", 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034072600000000002 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.955799+0000", 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034325969999999999 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962852+0000", 2024-04-25T12:36:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070528889999999997 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964076+0000", 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012234119999999999 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964178+0000", 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102166 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964264+0000", 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6175999999999996e-05 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 145.141896366, 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012746265999999999, 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951773+0000", 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951774+0000", 2024-04-25T12:36:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-06 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951782+0000", 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2090000000000005e-06 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951796+0000", 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3559e-05 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951808+0000", 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2183e-05 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956591+0000", 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047823550000000003 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956696+0000", 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105696 2024-04-25T12:36:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956907+0000", 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000211165 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962907+0000", 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059995650000000001 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964283+0000", 2024-04-25T12:36:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013758189999999999 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964381+0000", 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7977000000000001e-05 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964519+0000", 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138675 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:31 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 145.140727642, 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076541370000000001, 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952942+0000", 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952943+0000", 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952999+0000", 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5968999999999999e-05 2024-04-25T12:36:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953032+0000", 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2598999999999997e-05 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953058+0000", 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6585999999999999e-05 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953207+0000", 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014848300000000001 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953263+0000", 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6005999999999999e-05 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960434+0000", 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071716219999999999 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960596+0000", 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016173899999999999 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 145.140545723, 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010940785, 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953124+0000", 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953125+0000", 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.22e-06 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953138+0000", 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2996000000000001e-05 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953158+0000", 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0173000000000002e-05 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953176+0000", 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7927e-05 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954316+0000", 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011402599999999999 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954375+0000", 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:36:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963968+0000", 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095929269999999994 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964065+0000", 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6628999999999999e-05 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 145.14044689900001, 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080627230000000008, 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953223+0000", 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953225+0000", 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3640000000000002e-06 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953259+0000", 2024-04-25T12:36:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3955999999999998e-05 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953292+0000", 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3549e-05 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953320+0000", 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7892e-05 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953495+0000", 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017444800000000001 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953705+0000", 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021011000000000001 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953862+0000", 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015660399999999999 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.959861+0000", 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059993959999999997 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961071+0000", 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012101900000000001 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961192+0000", 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121279 2024-04-25T12:36:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961285+0000", 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2935000000000004e-05 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 145.14027016200001, 2024-04-25T12:36:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010652488999999999, 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953399+0000", 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953401+0000", 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.513e-06 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953412+0000", 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209000000000001e-05 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953473+0000", 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0668999999999998e-05 2024-04-25T12:36:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953496+0000", 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3246000000000001e-05 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956356+0000", 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00286002 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956620+0000", 2024-04-25T12:36:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026411099999999998 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.957005+0000", 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038473499999999998 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962757+0000", 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057517080000000003 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.963927+0000", 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00117023 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964002+0000", 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5483999999999998e-05 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.964052+0000", 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9564000000000002e-05 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 145.140141712, 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011827087999999999, 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953528+0000", 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953529+0000", 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953594+0000", 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4807999999999993e-05 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953617+0000", 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3220000000000001e-05 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953683+0000", 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5328999999999997e-05 2024-04-25T12:36:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954191+0000", 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000508428 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954594+0000", 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040280099999999999 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954922+0000", 2024-04-25T12:36:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032838300000000002 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960432+0000", 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055099989999999998 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965226+0000", 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047938700000000004 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965297+0000", 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0992000000000001e-05 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965355+0000", 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7695000000000002e-05 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:19 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'1, mlcod=0'0)", 2024-04-25T12:36:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.97723791200001, 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010949354999999999, 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116432+0000", 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116434+0000", 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116475+0000", 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1486000000000002e-05 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116505+0000", 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9709e-05 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116529+0000", 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3839999999999999e-05 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116651+0000", 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122013 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116685+0000", 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4513e-05 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127273+0000", 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010587653000000001 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127381+0000", 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108204 2024-04-25T12:36:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:20 2.9 e25/18 2:92e99a1d:::612.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.97708278499999, 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014061374, 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116587+0000", 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116588+0000", 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0499999999999999e-06 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116602+0000", 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3793e-05 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116621+0000", 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9545999999999999e-05 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.116637+0000", 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5696e-05 2024-04-25T12:36:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117243+0000", 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060668400000000002 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117280+0000", 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6619000000000003e-05 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130559+0000", 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013278570999999999 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130648+0000", 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9414999999999997e-05 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:27 2.9 e25/18 2:91bd965c:::501.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.976656135, 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013688128000000001, 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117013+0000", 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117015+0000", 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3200000000000001e-06 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117121+0000", 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106451 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117145+0000", 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.404e-05 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117210+0000", 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4314999999999997e-05 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120564+0000", 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033548050000000002 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120624+0000", 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9234999999999998e-05 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130654+0000", 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010030437999999999 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130702+0000", 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7524000000000001e-05 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:18 2.13 e25/18 2:ca4dd7f5:::611.00000000:head v 25'4, mlcod=24'1)", 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.976302836, 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010076254, 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117367+0000", 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117369+0000", 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.88e-06 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117385+0000", 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6670000000000001e-05 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117413+0000", 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7691999999999998e-05 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117435+0000", 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2416e-05 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117549+0000", 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113436 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117570+0000", 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1509e-05 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127347+0000", 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097767759999999992 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127443+0000", 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5874999999999999e-05 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.975091163, 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012295627, 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118582+0000", 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000001e-06 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118599+0000", 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7176e-05 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2505999999999997e-05 2024-04-25T12:36:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118659+0000", 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7559000000000002e-05 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120040+0000", 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001380956 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120305+0000", 2024-04-25T12:36:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026497700000000001 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120707+0000", 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040189499999999999 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128035+0000", 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073284049999999996 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130711+0000", 2024-04-25T12:36:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026752910000000002 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130787+0000", 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6384999999999995e-05 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130874+0000", 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7020000000000004e-05 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.974935163, 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012319366, 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118734+0000", 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118736+0000", 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118747+0000", 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1286e-05 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118772+0000", 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5150000000000001e-05 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118793+0000", 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0778999999999999e-05 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121911+0000", 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031179409999999999 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122331+0000", 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041955999999999997 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124441+0000", 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021107629999999999 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130205+0000", 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057636140000000002 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130888+0000", 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00068289100000000003 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130961+0000", 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3026000000000002e-05 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131054+0000", 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2855999999999993e-05 2024-04-25T12:36:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:21 2.8 2:1470b2f8:::613.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.974846435, 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012409474, 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118823+0000", 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118824+0000", 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118834+0000", 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118855+0000", 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0829999999999999e-05 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118875+0000", 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9809999999999998e-05 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119612+0000", 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073729299999999998 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119912+0000", 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029931000000000001 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120202+0000", 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000289907 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130342+0000", 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010140188 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131091+0000", 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074893900000000001 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131166+0000", 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5272999999999996e-05 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131233+0000", 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6625000000000007e-05 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.974737688, 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012992702, 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.165 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118932+0000", 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118933+0000", 2024-04-25T12:36:44.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3069999999999999e-06 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118943+0000", 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7200000000000001e-06 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118967+0000", 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3828999999999998e-05 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118986+0000", 2024-04-25T12:36:44.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9057000000000001e-05 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124788+0000", 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00580271 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125092+0000", 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030394999999999999 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:44.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125479+0000", 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00038660000000000002 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131299+0000", 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058200559999999997 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131823+0000", 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052382700000000004 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131888+0000", 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4740999999999996e-05 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131925+0000", 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6905e-05 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.1.9:5 2.7 e25/18 2:e455cf45:::60b.00000000:head v 25'3, mlcod=24'1)", 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 113.97461423999999, 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011712752, 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119055+0000", 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119057+0000", 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.296e-06 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119068+0000", 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.189e-05 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119091+0000", 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2802999999999998e-05 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119110+0000", 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8473000000000001e-05 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121085+0000", 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019750689999999999 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121149+0000", 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3979000000000005e-05 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130709+0000", 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009559817 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130768+0000", 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9425000000000001e-05 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:21 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:44.174 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.938824952, 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066623919999999996, 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154845+0000", 2024-04-25T12:36:44.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154846+0000", 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.06e-06 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154856+0000", 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0713000000000001e-05 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154878+0000", 2024-04-25T12:36:44.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1180000000000001e-05 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154896+0000", 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8343e-05 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155332+0000", 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043663000000000003 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155389+0000", 2024-04-25T12:36:44.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6818999999999999e-05 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161381+0000", 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059914440000000003 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161507+0000", 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126203 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.2.13:22 2.1a e25/18 2:5fd68f60:::61d.00000000:head v 25'3, mlcod=0'0)", 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.93855118099999, 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064678330000000001, 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155118+0000", 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155120+0000", 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3599999999999999e-06 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155131+0000", 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1793e-05 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155155+0000", 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3306e-05 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.155174+0000", 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9383000000000001e-05 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157101+0000", 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019271570000000001 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.157161+0000", 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9945000000000003e-05 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161512+0000", 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043506389999999999 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161586+0000", 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4250000000000002e-05 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:26 2.11 2:893cd530:::mds2_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.937614208, 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064127289999999998, 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156055+0000", 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156057+0000", 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8330000000000001e-06 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156090+0000", 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2719000000000003e-05 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156116+0000", 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6531999999999999e-05 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156140+0000", 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3453999999999999e-05 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156350+0000", 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021060999999999999 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156467+0000", 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116823 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156594+0000", 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012698900000000001 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.161255+0000", 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046609570000000003 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162360+0000", 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001105167 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162416+0000", 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6076999999999999e-05 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162468+0000", 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1567999999999998e-05 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 108.93746355899999, 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066007829999999998, 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:44.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156206+0000", 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156208+0000", 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5570000000000001e-06 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156216+0000", 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0900000000000005e-06 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156237+0000", 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1452999999999999e-05 2024-04-25T12:36:44.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156255+0000", 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8091999999999999e-05 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156841+0000", 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00058573200000000005 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.156983+0000", 2024-04-25T12:36:44.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000141676 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162807+0000", 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058241830000000001 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:44.192 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:44.194 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:44.194 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:44.195+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:44.294 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:44.295+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:44.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:44.396 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:44.397+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:44.467 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:44.467 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:44.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:44.497 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:44.499+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:44.598 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:44.599+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.631+0000 7fe792e00640 1 Processor -- start 2024-04-25T12:36:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.631+0000 7fe792e00640 1 -- start start 2024-04-25T12:36:44.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.633+0000 7fe792e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe78c1426a0 0x7fe78c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.633+0000 7fe792e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe78c074400 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.633+0000 7fe792e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe78c1595e0 con 0x7fe78c0c8170 2024-04-25T12:36:44.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.633+0000 7fe792e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe78c074670 con 0x7fe78c1426a0 2024-04-25T12:36:44.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.633+0000 7fe78be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe78c0c8fd0 0x7fe78c153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56042/0 (socket says 172.21.15.123:56042) 2024-04-25T12:36:44.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.633+0000 7fe78be00640 1 -- 172.21.15.123:0/4142918702 learned_addr learned my addr 172.21.15.123:0/4142918702 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:44.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.633+0000 7fe78b400640 1 --2- 172.21.15.123:0/4142918702 >> v2:172.21.15.153:3300/0 conn(0x7fe78c1426a0 0x7fe78c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127866468 0 0) 0x7fe78c074400 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe770003520 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 269489580 0 0) 0x7fe78c1595e0 con 0x7fe78c0c8170 2024-04-25T12:36:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe78c074400 con 0x7fe78c0c8170 2024-04-25T12:36:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78b400640 1 -- 172.21.15.123:0/4142918702 >> v1:172.21.15.153:6789/0 conn(0x7fe78c0c8170 legacy=0x7fe78c0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78b400640 1 -- 172.21.15.123:0/4142918702 >> v1:172.21.15.123:6789/0 conn(0x7fe78c0c8fd0 legacy=0x7fe78c153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1618811294 0 0) 0x7fe770003520 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.634+0000 7fe78b400640 1 -- 172.21.15.123:0/4142918702 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c074950 con 0x7fe78c1426a0 2024-04-25T12:36:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.636+0000 7fe78b400640 1 --2- 172.21.15.123:0/4142918702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 0x7fe78c159dc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097a0 tx=0x7fe77802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55ef4f684d0c094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.636+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778037070 con 0x7fe78c1426a0 2024-04-25T12:36:44.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.636+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77802ac50 con 0x7fe78c1426a0 2024-04-25T12:36:44.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.636+0000 7fe78aa00640 1 -- 172.21.15.123:0/4142918702 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c001670 con 0x7fe78c1426a0 2024-04-25T12:36:44.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.637+0000 7fe792e00640 1 -- 172.21.15.123:0/4142918702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 msgr2=0x7fe78c159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.637+0000 7fe792e00640 1 --2- 172.21.15.123:0/4142918702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 0x7fe78c159dc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097a0 tx=0x7fe77802a1c0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.637+0000 7fe792e00640 1 -- 172.21.15.123:0/4142918702 shutdown_connections 2024-04-25T12:36:44.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.637+0000 7fe792e00640 1 --2- 172.21.15.123:0/4142918702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 0x7fe78c159dc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.637+0000 7fe792e00640 1 -- 172.21.15.123:0/4142918702 >> 172.21.15.123:0/4142918702 conn(0x7fe78c06f440 msgr2=0x7fe78c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:44.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.637+0000 7fe792e00640 1 -- 172.21.15.123:0/4142918702 shutdown_connections 2024-04-25T12:36:44.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.637+0000 7fe792e00640 1 -- 172.21.15.123:0/4142918702 wait complete. 2024-04-25T12:36:44.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.639+0000 7fe792e00640 1 Processor -- start 2024-04-25T12:36:44.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.640+0000 7fe792e00640 1 -- start start 2024-04-25T12:36:44.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe792e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe78c0c8170 0x7fe78c1f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe792e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe78c0c8fd0 0x7fe78c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe792e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 0x7fe78c0c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe792e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c0c4880 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe792e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c15a300 con 0x7fe78c1426a0 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe792e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe78c0ca910 con 0x7fe78c0c8170 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe78b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe78c0c8fd0 0x7fe78c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe78b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe78c0c8fd0 0x7fe78c070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48656/0 (socket says 172.21.15.123:48656) 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe78b400640 1 -- 172.21.15.123:0/3356202757 learned_addr learned my addr 172.21.15.123:0/3356202757 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe78c0c8170 0x7fe78c1f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.641+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 0x7fe78c0c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe78b400640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe78c0c8170 msgr2=0x7fe78c1f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe78b400640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe78c0c8170 0x7fe78c1f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe78b400640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 msgr2=0x7fe78c0c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe78b400640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 0x7fe78c0c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe78b400640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c1f2be0 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe78c0c8170 0x7fe78c1f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe78c1426a0 0x7fe78c0c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.642+0000 7fe78b400640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe78c0c8fd0 0x7fe78c070c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7780098c0 tx=0x7fe7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.643+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778041070 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.643+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77803c460 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.643+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778037040 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.643+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c1e5340 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.643+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1f2ea0 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.644+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe78c076700 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.646+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe778044070 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.646+0000 7fe783e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe76c0744e0 0x7fe76c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.647+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6685+0+0 (secure 0 0 0) 0x7fe7780b54e0 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.647+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v65) ==== 54+0+16 (secure 0 0 0) 0x7fe77804b070 con 0x7fe78c0c8fd0 2024-04-25T12:36:44.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.647+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe76c0744e0 0x7fe76c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.647+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe78c0711a0 0x7fe78c0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.647+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe78c1e50f0 con 0x7fe78c0711a0 2024-04-25T12:36:44.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.647+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe78c0711a0 0x7fe78c0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.648+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe76c0744e0 0x7fe76c0769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0017e0 tx=0x7fe77c001610 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.648+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe78c0711a0 0x7fe78c0c78f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.649+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c1e50f0 con 0x7fe78c0711a0 2024-04-25T12:36:44.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.665+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c0ca4e0 con 0x7fe78c0711a0 2024-04-25T12:36:44.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.667+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c0ca4e0 con 0x7fe78c0711a0 2024-04-25T12:36:44.666 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:44.667 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:44.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.667+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe78c0afd20 0x7fe78c1f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.668+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe78c0c2a70 con 0x7fe78c0afd20 2024-04-25T12:36:44.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.668+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe78c0afd20 0x7fe78c1f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.669+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe78c0afd20 0x7fe78c1f6100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.669+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c0c2a70 con 0x7fe78c0afd20 2024-04-25T12:36:44.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.686+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c0c2790 con 0x7fe78c0afd20 2024-04-25T12:36:44.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.688+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c0c2790 con 0x7fe78c0afd20 2024-04-25T12:36:44.687 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:44.687 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:44.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.688+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe78c0af810 0x7fe78c1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.688+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe78c1f3250 con 0x7fe78c0af810 2024-04-25T12:36:44.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.688+0000 7fe78b400640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe78c0af810 0x7fe78c1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.689+0000 7fe78b400640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe78c0af810 0x7fe78c1f6b50 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.690+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c1f3250 con 0x7fe78c0af810 2024-04-25T12:36:44.699 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:44.700+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:44.704 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:44.704 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:44.704 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T12:36:44.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-04-25T12:36:44.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.705+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c0632b0 con 0x7fe78c0af810 2024-04-25T12:36:44.709 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:44.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.707+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c0632b0 con 0x7fe78c0af810 2024-04-25T12:36:44.710 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:44.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.707+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe78c157190 0x7fe78c1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.707+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe78c063230 con 0x7fe78c157190 2024-04-25T12:36:44.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.708+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe78c157190 0x7fe78c1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.709+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe78c157190 0x7fe78c1575a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.710+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c063230 con 0x7fe78c157190 2024-04-25T12:36:44.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.725+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c063230 con 0x7fe78c157190 2024-04-25T12:36:44.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.728+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c063230 con 0x7fe78c157190 2024-04-25T12:36:44.727 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:44.727 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:44.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.728+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe78c02c9d0 0x7fe78c02cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.728+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe78c157ff0 con 0x7fe78c02c9d0 2024-04-25T12:36:44.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.729+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe78c02c9d0 0x7fe78c02cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.730+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe78c02c9d0 0x7fe78c02cde0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.731+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c157ff0 con 0x7fe78c02c9d0 2024-04-25T12:36:44.741 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:44.741 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:44.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.746+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c157ff0 con 0x7fe78c02c9d0 2024-04-25T12:36:44.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.748+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c157ff0 con 0x7fe78c02c9d0 2024-04-25T12:36:44.747 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:44.748 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:44.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.749+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fe78c02d830 0x7fe78c2094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.749+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe78c157ff0 con 0x7fe78c02d830 2024-04-25T12:36:44.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.749+0000 7fe78b400640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fe78c02d830 0x7fe78c2094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.750+0000 7fe78b400640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fe78c02d830 0x7fe78c2094e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.751+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c157ff0 con 0x7fe78c02d830 2024-04-25T12:36:44.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:44.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.766+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c157ff0 con 0x7fe78c02d830 2024-04-25T12:36:44.769 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:44.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.769+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c157ff0 con 0x7fe78c02d830 2024-04-25T12:36:44.770 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:44.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.769+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe78c02dc40 0x7fe78c20bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.769+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe78c20ed10 con 0x7fe78c02dc40 2024-04-25T12:36:44.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.769+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe78c02dc40 0x7fe78c20bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.770+0000 7fe790e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe78c02dc40 0x7fe78c20bf40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.771+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c20ed10 con 0x7fe78c02dc40 2024-04-25T12:36:44.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.786+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c20c480 con 0x7fe78c02dc40 2024-04-25T12:36:44.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.788+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c20c480 con 0x7fe78c02dc40 2024-04-25T12:36:44.787 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.789+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe78c20cb00 0x7fe78c20ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.789+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe78c211e30 con 0x7fe78c20cb00 2024-04-25T12:36:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.789+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe78c20cb00 0x7fe78c20ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.790+0000 7fe78be00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe78c20cb00 0x7fe78c20ef20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.791+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe78c211e30 con 0x7fe78c20cb00 2024-04-25T12:36:44.800 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:44.801+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.817+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe78c20f460 con 0x7fe78c20cb00 2024-04-25T12:36:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.819+0000 7fe783e00640 1 -- 172.21.15.123:0/3356202757 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe78c20f460 con 0x7fe78c20cb00 2024-04-25T12:36:44.819 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:44.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe78c0711a0 msgr2=0x7fe78c0c78f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fe78c0711a0 0x7fe78c0c78f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe78c0afd20 msgr2=0x7fe78c1f6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fe78c0afd20 0x7fe78c1f6100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe78c0af810 msgr2=0x7fe78c1f6b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fe78c0af810 0x7fe78c1f6b50 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe78c157190 msgr2=0x7fe78c1575a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fe78c157190 0x7fe78c1575a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe78c02c9d0 msgr2=0x7fe78c02cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fe78c02c9d0 0x7fe78c02cde0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fe78c02d830 msgr2=0x7fe78c2094e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fe78c02d830 0x7fe78c2094e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe78c02dc40 msgr2=0x7fe78c20bf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fe78c02dc40 0x7fe78c20bf40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe78c20cb00 msgr2=0x7fe78c20ef20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fe78c20cb00 0x7fe78c20ef20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe790e00640 1 -- 172.21.15.123:0/3356202757 reap_dead start 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe790e00640 1 -- 172.21.15.123:0/3356202757 reap_dead start 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe790e00640 1 -- 172.21.15.123:0/3356202757 reap_dead start 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe790e00640 1 -- 172.21.15.123:0/3356202757 reap_dead start 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.820+0000 7fe790e00640 1 -- 172.21.15.123:0/3356202757 reap_dead start 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe76c0744e0 msgr2=0x7fe76c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe76c0744e0 0x7fe76c0769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0017e0 tx=0x7fe77c001610 comp rx=0 tx=0).stop 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe78c0c8fd0 msgr2=0x7fe78c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe78c0c8fd0 0x7fe78c070c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7780098c0 tx=0x7fe7780046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 shutdown_connections 2024-04-25T12:36:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe76c0744e0 0x7fe76c0769f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 --2- 172.21.15.123:0/3356202757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe78c0c8fd0 0x7fe78c070c60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 >> 172.21.15.123:0/3356202757 conn(0x7fe78c06f440 msgr2=0x7fe78c0cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 shutdown_connections 2024-04-25T12:36:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.821+0000 7fe792e00640 1 -- 172.21.15.123:0/3356202757 wait complete. 2024-04-25T12:36:44.901 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:44.902+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:44.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.962+0000 7fa013800640 1 Processor -- start 2024-04-25T12:36:44.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.962+0000 7fa013800640 1 -- start start 2024-04-25T12:36:44.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.964+0000 7fa013800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa00c0c7030 0x7fa00c15d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.964+0000 7fa013800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.964+0000 7fa013800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa00c0d1f90 con 0x7fa00c0cdde0 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa013800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa00c15cdc0 con 0x7fa00c0c7030 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa013800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa00c0c7440 con 0x7fa00c156910 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa010e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa00c0cdde0 0x7fa00c160610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56046/0 (socket says 172.21.15.123:56046) 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa010e00640 1 -- 172.21.15.123:0/4038784037 learned_addr learned my addr 172.21.15.123:0/4038784037 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa00be00640 1 --2- 172.21.15.123:0/4038784037 >> v2:172.21.15.153:3300/0 conn(0x7fa00c0c7030 0x7fa00c15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa011800640 1 --2- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa00b400640 1 -- 172.21.15.123:0/4038784037 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1295179578 0 0) 0x7fa00c0d1f90 con 0x7fa00c0cdde0 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa00b400640 1 -- 172.21.15.123:0/4038784037 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ff4003520 con 0x7fa00c0cdde0 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa00be00640 1 -- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 msgr2=0x7fa00c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa00be00640 1 --2- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa00be00640 1 -- 172.21.15.123:0/4038784037 >> v1:172.21.15.123:6789/0 conn(0x7fa00c0cdde0 legacy=0x7fa00c160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa00be00640 1 -- 172.21.15.123:0/4038784037 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c0d2530 con 0x7fa00c0c7030 2024-04-25T12:36:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.965+0000 7fa011800640 1 --2- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.966+0000 7fa00be00640 1 --2- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 0x7fa00c15d250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80097a0 tx=0x7f9ff802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb0db0bab378916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.967+0000 7fa00b400640 1 -- 172.21.15.123:0/4038784037 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8037070 con 0x7fa00c0c7030 2024-04-25T12:36:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.967+0000 7fa00b400640 1 -- 172.21.15.123:0/4038784037 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff8032440 con 0x7fa00c0c7030 2024-04-25T12:36:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.967+0000 7fa00b400640 1 -- 172.21.15.123:0/4038784037 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80327f0 con 0x7fa00c0c7030 2024-04-25T12:36:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 -- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 msgr2=0x7fa00c15d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 --2- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 0x7fa00c15d250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80097a0 tx=0x7f9ff802a180 comp rx=0 tx=0).stop 2024-04-25T12:36:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 -- 172.21.15.123:0/4038784037 shutdown_connections 2024-04-25T12:36:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 --2- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 --2- 172.21.15.123:0/4038784037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 0x7fa00c15d250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 -- 172.21.15.123:0/4038784037 >> 172.21.15.123:0/4038784037 conn(0x7fa00c08d940 msgr2=0x7fa00c08dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 -- 172.21.15.123:0/4038784037 shutdown_connections 2024-04-25T12:36:44.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.968+0000 7fa013800640 1 -- 172.21.15.123:0/4038784037 wait complete. 2024-04-25T12:36:44.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.972+0000 7fa013800640 1 Processor -- start 2024-04-25T12:36:44.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.972+0000 7fa013800640 1 -- start start 2024-04-25T12:36:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.974+0000 7fa013800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 0x7fa00c1f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.974+0000 7fa013800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa00c0cdde0 0x7fa00c151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.974+0000 7fa013800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c1495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.974+0000 7fa013800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c15c770 con 0x7fa00c0cdde0 2024-04-25T12:36:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.974+0000 7fa013800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c0d2250 con 0x7fa00c0c7030 2024-04-25T12:36:44.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.974+0000 7fa013800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa00c15ca50 con 0x7fa00c156910 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa00c0cdde0 0x7fa00c151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa00c0cdde0 0x7fa00c151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48670/0 (socket says 172.21.15.123:48670) 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 -- 172.21.15.123:0/3143608648 learned_addr learned my addr 172.21.15.123:0/3143608648 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa010e00640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 0x7fa00c1f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa011800640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c1495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 -- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 msgr2=0x7fa00c1495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c1495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 -- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 msgr2=0x7fa00c1f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 0x7fa00c1f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:44.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.975+0000 7fa00be00640 1 -- 172.21.15.123:0/3143608648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c1f2da0 con 0x7fa00c0cdde0 2024-04-25T12:36:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.976+0000 7fa00be00640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa00c0cdde0 0x7fa00c151750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80098c0 tx=0x7f9ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.976+0000 7fa008c00640 1 -- 172.21.15.123:0/3143608648 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8037040 con 0x7fa00c0cdde0 2024-04-25T12:36:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.976+0000 7fa008c00640 1 -- 172.21.15.123:0/3143608648 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff80044c0 con 0x7fa00c0cdde0 2024-04-25T12:36:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.976+0000 7fa008c00640 1 -- 172.21.15.123:0/3143608648 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8006e10 con 0x7fa00c0cdde0 2024-04-25T12:36:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.976+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00c15c1d0 con 0x7fa00c0cdde0 2024-04-25T12:36:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.976+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00c14beb0 con 0x7fa00c0cdde0 2024-04-25T12:36:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.978+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00c076700 con 0x7fa00c0cdde0 2024-04-25T12:36:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.979+0000 7fa008c00640 1 -- 172.21.15.123:0/3143608648 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ff8044020 con 0x7fa00c0cdde0 2024-04-25T12:36:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.979+0000 7fa008c00640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9fec0744b0 0x7f9fec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:44.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.980+0000 7fa008c00640 1 -- 172.21.15.123:0/3143608648 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6685+0+0 (secure 0 0 0) 0x7f9ff80b56c0 con 0x7fa00c0cdde0 2024-04-25T12:36:44.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.980+0000 7fa010e00640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9fec0744b0 0x7f9fec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:44.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.981+0000 7fa010e00640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9fec0744b0 0x7f9fec0769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa000005880 tx=0x7fa000009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:44.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:44.985+0000 7fa008c00640 1 -- 172.21.15.123:0/3143608648 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ff807fa80 con 0x7fa00c0cdde0 2024-04-25T12:36:45.002 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:45.003+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:45.014 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:45.014 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:45.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:45.126 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:45.127+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:45.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:45.199+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd out", "ids": ["6"]} v 0) -- 0x7fa00c0cdbc0 con 0x7fa00c0cdde0 2024-04-25T12:36:45.198 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-04-25T12:36:45.199+0000 7fa57d600640 -1 mon.a@0(leader).osd e65 definitely_dead 0 2024-04-25T12:36:45.204 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:45.205+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:45.305 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:45.306+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:45.330 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:45.330 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:45.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:45.405 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:45.406+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:45.506 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:45.507+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:45.607 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:45.608+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:45.653 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:45.653 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:45.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:45.708 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:45.709+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:45.809 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:45.810+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:45.910 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:45.910+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:45.933 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:45.933 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:45.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:46.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.010+0000 7fa008c00640 1 -- 172.21.15.123:0/3143608648 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["6"]}]=0 marked out osd.6. v66) ==== 87+0+0 (secure 0 0 0) 0x7f9ff807f380 con 0x7fa00c0cdde0 2024-04-25T12:36:46.009 INFO:teuthology.orchestra.run.smithi123.stderr:marked out osd.6. 2024-04-25T12:36:46.010 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:46.011+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:46.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.012+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9fec0744b0 msgr2=0x7f9fec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:46.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.013+0000 7fa013800640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9fec0744b0 0x7f9fec0769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa000005880 tx=0x7fa000009040 comp rx=0 tx=0).stop 2024-04-25T12:36:46.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.013+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa00c0cdde0 msgr2=0x7fa00c151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:46.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.013+0000 7fa013800640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa00c0cdde0 0x7fa00c151750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80098c0 tx=0x7f9ff80046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.014+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 shutdown_connections 2024-04-25T12:36:46.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.014+0000 7fa013800640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9fec0744b0 0x7f9fec0769c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.014+0000 7fa013800640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa00c156910 0x7fa00c1495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.014+0000 7fa013800640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa00c0cdde0 0x7fa00c151750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.014+0000 7fa013800640 1 --2- 172.21.15.123:0/3143608648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa00c0c7030 0x7fa00c1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.014+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 >> 172.21.15.123:0/3143608648 conn(0x7fa00c08d940 msgr2=0x7fa00c08dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:46.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.016+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 shutdown_connections 2024-04-25T12:36:46.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.016+0000 7fa013800640 1 -- 172.21.15.123:0/3143608648 wait complete. 2024-04-25T12:36:46.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:36:46.111 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:46.112+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:46.212 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:46.213+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:46.231 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:46.231 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:46.252 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:46.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.290+0000 7ff4fc600640 1 Processor -- start 2024-04-25T12:36:46.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.290+0000 7ff4fc600640 1 -- start start 2024-04-25T12:36:46.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4fc600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4f40ca110 0x7ff4f40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:46.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4fc600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4164e40 0x7ff4f4167260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4fc600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f40c90e0 con 0x7ff4f4071ee0 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4fc600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f40c59d0 con 0x7ff4f40ca110 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4fc600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f40c5450 con 0x7ff4f4164e40 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4f9c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4f40ca110 0x7ff4f40719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4f9c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4f40ca110 0x7ff4f40719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48676/0 (socket says 172.21.15.123:48676) 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4f9c00640 1 -- 172.21.15.123:0/874841483 learned_addr learned my addr 172.21.15.123:0/874841483 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.292+0000 7ff4fa600640 1 --2- 172.21.15.123:0/874841483 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4164e40 0x7ff4f4167260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.293+0000 7ff4f9c00640 1 -- 172.21.15.123:0/874841483 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4164e40 msgr2=0x7ff4f4167260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.293+0000 7ff4f9c00640 1 --2- 172.21.15.123:0/874841483 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4164e40 0x7ff4f4167260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.293+0000 7ff4f9c00640 1 -- 172.21.15.123:0/874841483 >> v1:172.21.15.123:6789/0 conn(0x7ff4f4071ee0 legacy=0x7ff4f40c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.293+0000 7ff4f9c00640 1 -- 172.21.15.123:0/874841483 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f40c56f0 con 0x7ff4f40ca110 2024-04-25T12:36:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.293+0000 7ff4fa600640 1 --2- 172.21.15.123:0/874841483 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4164e40 0x7ff4f4167260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:36:46.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.294+0000 7ff4f9c00640 1 --2- 172.21.15.123:0/874841483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f40719a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009870 tx=0x7ff4e802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c47ff5604792f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:46.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.294+0000 7ff4f3e00640 1 -- 172.21.15.123:0/874841483 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8037070 con 0x7ff4f40ca110 2024-04-25T12:36:46.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.294+0000 7ff4f3e00640 1 -- 172.21.15.123:0/874841483 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e802ae50 con 0x7ff4f40ca110 2024-04-25T12:36:46.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.294+0000 7ff4f3e00640 1 -- 172.21.15.123:0/874841483 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8032860 con 0x7ff4f40ca110 2024-04-25T12:36:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 -- 172.21.15.123:0/874841483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 msgr2=0x7ff4f40719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 --2- 172.21.15.123:0/874841483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f40719a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009870 tx=0x7ff4e802a300 comp rx=0 tx=0).stop 2024-04-25T12:36:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 -- 172.21.15.123:0/874841483 shutdown_connections 2024-04-25T12:36:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 --2- 172.21.15.123:0/874841483 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4164e40 0x7ff4f4167260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 --2- 172.21.15.123:0/874841483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f40719a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 -- 172.21.15.123:0/874841483 >> 172.21.15.123:0/874841483 conn(0x7ff4f4070e80 msgr2=0x7ff4f4071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 -- 172.21.15.123:0/874841483 shutdown_connections 2024-04-25T12:36:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.295+0000 7ff4fc600640 1 -- 172.21.15.123:0/874841483 wait complete. 2024-04-25T12:36:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.297+0000 7ff4fc600640 1 Processor -- start 2024-04-25T12:36:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.298+0000 7ff4fc600640 1 -- start start 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4fc600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4071ee0 0x7ff4f4151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4fc600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4fc600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4f4164e40 0x7ff4f414aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4fc600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4156c70 con 0x7ff4f40ca110 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4fc600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f40c8c50 con 0x7ff4f4164e40 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4fc600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f4068d50 con 0x7ff4f4071ee0 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4f9c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4071ee0 0x7ff4f4151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4f9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4f9c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4071ee0 0x7ff4f4151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:37480/0 (socket says 172.21.15.123:37480) 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4f9c00640 1 -- 172.21.15.123:0/2261990008 learned_addr learned my addr 172.21.15.123:0/2261990008 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.299+0000 7ff4fa600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4f4164e40 0x7ff4f414aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.300+0000 7ff4f9c00640 1 -- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4f4164e40 msgr2=0x7ff4f414aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.300+0000 7ff4f9c00640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4f4164e40 0x7ff4f414aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.300+0000 7ff4f9c00640 1 -- 172.21.15.123:0/2261990008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 msgr2=0x7ff4f414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.300+0000 7ff4f9c00640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f414a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.300+0000 7ff4f9c00640 1 -- 172.21.15.123:0/2261990008 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f4152520 con 0x7ff4f4071ee0 2024-04-25T12:36:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.300+0000 7ff4f9200640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.300+0000 7ff4f9c00640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4071ee0 0x7ff4f4151fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4e802a340 tx=0x7ff4e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.301+0000 7ff4f1600640 1 -- 172.21.15.123:0/2261990008 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8041020 con 0x7ff4f4071ee0 2024-04-25T12:36:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.301+0000 7ff4f1600640 1 -- 172.21.15.123:0/2261990008 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e803c670 con 0x7ff4f4071ee0 2024-04-25T12:36:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.301+0000 7ff4f1600640 1 -- 172.21.15.123:0/2261990008 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8037040 con 0x7ff4f4071ee0 2024-04-25T12:36:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.301+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f4149050 con 0x7ff4f4071ee0 2024-04-25T12:36:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.301+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f4152ad0 con 0x7ff4f4071ee0 2024-04-25T12:36:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.302+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f4076700 con 0x7ff4f4071ee0 2024-04-25T12:36:46.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.303+0000 7ff4f1600640 1 -- 172.21.15.123:0/2261990008 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff4e8044020 con 0x7ff4f4071ee0 2024-04-25T12:36:46.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.304+0000 7ff4f1600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff4d40744b0 0x7ff4d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:46.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.304+0000 7ff4f1600640 1 -- 172.21.15.123:0/2261990008 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(66..66 src has 1..66) ==== 6685+0+0 (secure 0 0 0) 0x7ff4e80b5ae0 con 0x7ff4f4071ee0 2024-04-25T12:36:46.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.305+0000 7ff4f9200640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff4d40744b0 0x7ff4d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:46.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.305+0000 7ff4f9200640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff4d40744b0 0x7ff4d40769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4f414b970 tx=0x7ff4e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:46.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.309+0000 7ff4f1600640 1 -- 172.21.15.123:0/2261990008 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4e804b070 con 0x7ff4f4071ee0 2024-04-25T12:36:46.312 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:46.313+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:46.413 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:46.414+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:46.492 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:46.492 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:46.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:46.512+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7ff4f40c82a0 con 0x7ff4f4071ee0 2024-04-25T12:36:46.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:46.522 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:46.523+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:46.615 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:46.616+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:46.659 INFO:tasks.workunit.client.0.smithi123.stderr:Updating files: 60% (7384/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 98% (12140/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-25T12:36:46.717 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:46.718+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:46.763 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:46.763 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:46.784 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:46.825 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:46.826+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:46.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:46.911 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:46.911 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:46.918 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:46.919+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:46.920 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:47.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.016+0000 7ff4f1600640 1 -- 172.21.15.123:0/2261990008 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v67) ==== 101+0+0 (secure 0 0 0) 0x7ff4e807f7b0 con 0x7ff4f4071ee0 2024-04-25T12:36:47.018 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:36:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.021+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff4d40744b0 msgr2=0x7ff4d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.021+0000 7ff4fc600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff4d40744b0 0x7ff4d40769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4f414b970 tx=0x7ff4e0009040 comp rx=0 tx=0).stop 2024-04-25T12:36:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.021+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4071ee0 msgr2=0x7ff4f4151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.021+0000 7ff4fc600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4071ee0 0x7ff4f4151fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4e802a340 tx=0x7ff4e80046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.023 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:47.021+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.024+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 shutdown_connections 2024-04-25T12:36:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.024+0000 7ff4fc600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff4d40744b0 0x7ff4d40769c0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4f414b970 tx=0x7ff4e0009040 comp rx=0 tx=0).stop 2024-04-25T12:36:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.024+0000 7ff4fc600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4f4164e40 0x7ff4f414aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.024+0000 7ff4fc600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f40ca110 0x7ff4f414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.024+0000 7ff4fc600640 1 --2- 172.21.15.123:0/2261990008 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff4f4071ee0 0x7ff4f4151fe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.024+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 >> 172.21.15.123:0/2261990008 conn(0x7ff4f4070e80 msgr2=0x7ff4f4147df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.024+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 shutdown_connections 2024-04-25T12:36:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.025+0000 7ff4fc600640 1 -- 172.21.15.123:0/2261990008 wait complete. 2024-04-25T12:36:47.090 INFO:teuthology.orchestra.run.smithi153.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:47.090 INFO:teuthology.orchestra.run.smithi153.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:47.100 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:47.120 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:47.121+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.125+0000 7f6f90a00640 1 Processor -- start 2024-04-25T12:36:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.125+0000 7f6f90a00640 1 -- start start 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f90a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6f8c1426a0 0x7f6f8c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f90a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f90a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c074400 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f90a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8c1595e0 con 0x7f6f8c1426a0 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f90a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c074670 con 0x7f6f8c0c8170 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f89e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6f8c0c8fd0 0x7f6f8c153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56070/0 (socket says 172.21.15.123:56070) 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f89e00640 1 -- 172.21.15.123:0/3418385918 learned_addr learned my addr 172.21.15.123:0/3418385918 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f89400640 1 --2- 172.21.15.123:0/3418385918 >> v2:172.21.15.153:3300/0 conn(0x7f6f8c1426a0 0x7f6f8c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.127+0000 7f6f88a00640 1 -- 172.21.15.123:0/3418385918 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3885625997 0 0) 0x7f6f8c074400 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.128+0000 7f6f89400640 1 -- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 msgr2=0x7f6f8c0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.128+0000 7f6f89400640 1 --2- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c0ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.128+0000 7f6f89400640 1 -- 172.21.15.123:0/3418385918 >> v1:172.21.15.123:6789/0 conn(0x7f6f8c0c8fd0 legacy=0x7f6f8c153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.128+0000 7f6f89400640 1 -- 172.21.15.123:0/3418385918 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c074950 con 0x7f6f8c1426a0 2024-04-25T12:36:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.128+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.129+0000 7f6f89400640 1 --2- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c159dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f780097a0 tx=0x7f6f7802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fb59b5e5937a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.129+0000 7f6f88a00640 1 -- 172.21.15.123:0/3418385918 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78037070 con 0x7f6f8c1426a0 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.129+0000 7f6f88a00640 1 -- 172.21.15.123:0/3418385918 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f78032440 con 0x7f6f8c1426a0 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.129+0000 7f6f88a00640 1 -- 172.21.15.123:0/3418385918 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78032810 con 0x7f6f8c1426a0 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.129+0000 7f6f90a00640 1 -- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c1426a0 msgr2=0x7f6f8c159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.129+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c159dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f780097a0 tx=0x7f6f7802a180 comp rx=0 tx=0).stop 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.130+0000 7f6f90a00640 1 -- 172.21.15.123:0/3418385918 shutdown_connections 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.130+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.130+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3418385918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c159dc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.130+0000 7f6f90a00640 1 -- 172.21.15.123:0/3418385918 >> 172.21.15.123:0/3418385918 conn(0x7f6f8c06f440 msgr2=0x7f6f8c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.130+0000 7f6f90a00640 1 -- 172.21.15.123:0/3418385918 shutdown_connections 2024-04-25T12:36:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.130+0000 7f6f90a00640 1 -- 172.21.15.123:0/3418385918 wait complete. 2024-04-25T12:36:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.132+0000 7f6f90a00640 1 Processor -- start 2024-04-25T12:36:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.132+0000 7f6f90a00640 1 -- start start 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.133+0000 7f6f90a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c1f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.133+0000 7f6f90a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c0c8fd0 0x7f6f8c1367d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.133+0000 7f6f90a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.133+0000 7f6f90a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0c4880 con 0x7f6f8c1426a0 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.133+0000 7f6f90a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c15a300 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.133+0000 7f6f90a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0c8580 con 0x7f6f8c0c8170 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f8a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f8a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48686/0 (socket says 172.21.15.123:48686) 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f8a800640 1 -- 172.21.15.123:0/3485955665 learned_addr learned my addr 172.21.15.123:0/3485955665 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c0c8fd0 0x7f6f8c1367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c1f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f89400640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 msgr2=0x7f6f8c1f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f8c0c8170 0x7f6f8c1f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f89400640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f8c1426a0 msgr2=0x7f6f8c151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f89400640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c13c3d0 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.134+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f8c1426a0 0x7f6f8c151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.135+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c0c8fd0 0x7f6f8c1367d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6f78031000 tx=0x7f6f780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.135+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78041020 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.135+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f78030700 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.135+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c1f0f60 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.135+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78037040 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.135+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c1f4a20 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.136+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f6f8c076700 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.138+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6f78044020 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.140+0000 7f6f82000640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6f6c0744b0 0x7f6f6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.140+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6685+0+0 (secure 0 0 0) 0x7f6f780b5ce0 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.140+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6f6c0744b0 0x7f6f6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.140+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v67) ==== 54+0+16 (secure 0 0 0) 0x7f6f7804b030 con 0x7f6f8c0c8fd0 2024-04-25T12:36:47.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.140+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f6f8c135ae0 0x7f6f8c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.140+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c155e70 con 0x7f6f8c135ae0 2024-04-25T12:36:47.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.140+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6f6c0744b0 0x7f6f6c0769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007fd0 tx=0x7f6f7c007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.141+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f6f8c135ae0 0x7f6f8c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.141+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f6f8c135ae0 0x7f6f8c158e00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.142+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c155e70 con 0x7f6f8c135ae0 2024-04-25T12:36:47.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.158+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c159340 con 0x7f6f8c135ae0 2024-04-25T12:36:47.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.160+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c159340 con 0x7f6f8c135ae0 2024-04-25T12:36:47.159 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:47.159 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:47.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.160+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f6f8c140940 0x7f6f8c1394a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.160+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c0c4200 con 0x7f6f8c140940 2024-04-25T12:36:47.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.160+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f6f8c140940 0x7f6f8c1394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.161+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f6f8c140940 0x7f6f8c1394a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.162+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c0c4200 con 0x7f6f8c140940 2024-04-25T12:36:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.177+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c136d10 con 0x7f6f8c140940 2024-04-25T12:36:47.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.179+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c136d10 con 0x7f6f8c140940 2024-04-25T12:36:47.178 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:47.179 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:47.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.180+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f6f8c139c30 0x7f6f8c13de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.180+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c136f20 con 0x7f6f8c139c30 2024-04-25T12:36:47.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.180+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f6f8c139c30 0x7f6f8c13de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.181+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f6f8c139c30 0x7f6f8c13de70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.181+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c136f20 con 0x7f6f8c139c30 2024-04-25T12:36:47.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.196+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c13e680 con 0x7f6f8c139c30 2024-04-25T12:36:47.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.198+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c13e680 con 0x7f6f8c139c30 2024-04-25T12:36:47.197 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:47.197 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:47.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.198+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f6f8c063660 0x7f6f8c063a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.198+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c063fb0 con 0x7f6f8c063660 2024-04-25T12:36:47.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.198+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f6f8c063660 0x7f6f8c063a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.199+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f6f8c063660 0x7f6f8c063a70 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.200+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c063fb0 con 0x7f6f8c063660 2024-04-25T12:36:47.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.214+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c063fb0 con 0x7f6f8c063660 2024-04-25T12:36:47.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.216+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c063fb0 con 0x7f6f8c063660 2024-04-25T12:36:47.215 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:47.216 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:47.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.216+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f6f8c0ce180 0x7f6f8c02c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.216+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c063fb0 con 0x7f6f8c0ce180 2024-04-25T12:36:47.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.217+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f6f8c0ce180 0x7f6f8c02c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.217+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f6f8c0ce180 0x7f6f8c02c9d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.219+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c063fb0 con 0x7f6f8c0ce180 2024-04-25T12:36:47.220 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:47.221+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.233+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c063fb0 con 0x7f6f8c0ce180 2024-04-25T12:36:47.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.236+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c063fb0 con 0x7f6f8c0ce180 2024-04-25T12:36:47.235 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:47.235 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:47.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.236+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f6f8c02d420 0x7f6f8c02d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.236+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c02de40 con 0x7f6f8c02d420 2024-04-25T12:36:47.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.237+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f6f8c02d420 0x7f6f8c02d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.237+0000 7f6f89400640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f6f8c02d420 0x7f6f8c02d830 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.238+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c02de40 con 0x7f6f8c02d420 2024-04-25T12:36:47.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.253+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c0cd8d0 con 0x7f6f8c02d420 2024-04-25T12:36:47.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.256+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c0cd8d0 con 0x7f6f8c02d420 2024-04-25T12:36:47.255 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:47.255 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:47.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.256+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f6f8c209a10 0x7f6f8c20be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.256+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c20ede0 con 0x7f6f8c209a10 2024-04-25T12:36:47.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.256+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f6f8c209a10 0x7f6f8c20be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.257+0000 7f6f8a800640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f6f8c209a10 0x7f6f8c20be30 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.258+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c20ede0 con 0x7f6f8c209a10 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.30938870899999, 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:36:47.259 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.260 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.261 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:36:47.262 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:36:47.263 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.264 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.309301, 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.265 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.266 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:36:47.267 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:36:47.268 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:36:47.269 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.30917507199999, 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.270 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:36:47.271 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:36:47.272 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:47.273 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.274 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.275 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.30911341300001, 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:47.276 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:36:47.277 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.278 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.279 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.30905665, 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:36:47.283 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:36:47.284 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:36:47.285 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.30899317999999, 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.286 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.287 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.288 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.289 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:36:47.290 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.291 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.30894123100001, 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.292 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:36:47.293 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.294 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.295 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.296 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.30888928100001, 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.297 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:36:47.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:36:47.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:36:47.300 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:36:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.308772172, 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:36:47.302 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.303 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:36:47.304 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:36:47.305 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.306 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.307 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.308733725, 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:36:47.309 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:36:47.310 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.311 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.143598672, 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.312 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.313 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.314 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:36:47.315 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:36:47.316 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.143540621, 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.317 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.318 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.319 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.320 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:36:47.321 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.143480928, 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:36:47.322 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.323 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.324 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.325 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.326 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.327 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.10656084599999, 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.328 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.329 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:36:47.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:36:47.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:47.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.10647188, 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.334 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:47.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:36:47.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:36:47.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.10642803899999, 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.346 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.104914135, 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.103994919, 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:36:47.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:36:47.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:36:47.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.10353956599999, 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:36:47.360 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.10351058800001, 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:36:47.361 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:36:47.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.365 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.366 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.366 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:47.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.272+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c20db00 con 0x7f6f8c209a10 2024-04-25T12:36:47.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.274+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c20db00 con 0x7f6f8c209a10 2024-04-25T12:36:47.368 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:47.369 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:47.369 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:47.368+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:47.370 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:47.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.274+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f6f8c20e220 0x7f6f8c20e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:47.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.274+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c2121b0 con 0x7f6f8c20e220 2024-04-25T12:36:47.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.275+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f6f8c20e220 0x7f6f8c20e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:47.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.276+0000 7f6f89e00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f6f8c20e220 0x7f6f8c20e630 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:47.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.277+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c2121b0 con 0x7f6f8c20e220 2024-04-25T12:36:47.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.291+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6f8c211030 con 0x7f6f8c20e220 2024-04-25T12:36:47.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.293+0000 7f6f82000640 1 -- 172.21.15.123:0/3485955665 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c211030 con 0x7f6f8c20e220 2024-04-25T12:36:47.371 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:47.373 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:47.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f6f8c135ae0 msgr2=0x7f6f8c158e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f6f8c135ae0 0x7f6f8c158e00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f6f8c140940 msgr2=0x7f6f8c1394a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f6f8c140940 0x7f6f8c1394a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f6f8c139c30 msgr2=0x7f6f8c13de70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f6f8c139c30 0x7f6f8c13de70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f6f8c063660 msgr2=0x7f6f8c063a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f6f8c063660 0x7f6f8c063a70 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f8a800640 1 -- 172.21.15.123:0/3485955665 reap_dead start 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f6f8c0ce180 msgr2=0x7f6f8c02c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f6f8c0ce180 0x7f6f8c02c9d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f8a800640 1 -- 172.21.15.123:0/3485955665 reap_dead start 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f6f8c02d420 msgr2=0x7f6f8c02d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f6f8c02d420 0x7f6f8c02d830 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f6f8c209a10 msgr2=0x7f6f8c20be30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f6f8c209a10 0x7f6f8c20be30 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f6f8c20e220 msgr2=0x7f6f8c20e630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f6f8c20e220 0x7f6f8c20e630 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6f6c0744b0 msgr2=0x7f6f6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6f6c0744b0 0x7f6f6c0769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007fd0 tx=0x7f6f7c007de0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c0c8fd0 msgr2=0x7f6f8c1367d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 --2- 172.21.15.123:0/3485955665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f8c0c8fd0 0x7f6f8c1367d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6f78031000 tx=0x7f6f780046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f8a800640 1 -- 172.21.15.123:0/3485955665 reap_dead start 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f8a800640 1 -- 172.21.15.123:0/3485955665 reap_dead start 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 shutdown_connections 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.294+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 >> 172.21.15.123:0/3485955665 conn(0x7f6f8c06f440 msgr2=0x7f6f8c1412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.295+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 shutdown_connections 2024-04-25T12:36:47.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:47.295+0000 7f6f90a00640 1 -- 172.21.15.123:0/3485955665 wait complete. 2024-04-25T12:36:47.468 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:47.469+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:47.521 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:47.521 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:47.521 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:47.521 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:47.530 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:47.570 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:47.570+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:47.619 INFO:tasks.workunit.client.0.smithi123.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T12:36:47.619 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr: git switch -c 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr:Or undo this operation with: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr: git switch - 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-04-25T12:36:47.620 INFO:tasks.workunit.client.0.smithi123.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T12:36:47.621 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-04-25T12:36:47.621 INFO:tasks.workunit.client.0.smithi123.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T12:36:47.635 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T12:36:47.670 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:47.671+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:47.682 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:47.682 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:47.682 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:47.682 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:47.682 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:47.691 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:47.739 INFO:tasks.workunit.client.0.smithi123.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T12:36:47.745 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T12:36:47.746 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T12:36:47.771 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:47.772+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:47.849 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 24/18 rep_tid 1 entries 24'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 148.89483898500001, 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075768199999999997, 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:47.850 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954840+0000", 2024-04-25T12:36:47.851 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6869999999999999e-06 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954864+0000", 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3824000000000001e-05 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954902+0000", 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500999999999999e-05 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.852 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954919+0000", 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7618e-05 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955011+0000", 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2250000000000006e-05 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955596+0000", 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058480200000000004 2024-04-25T12:36:47.853 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:25 2.1b e25/18 2:dbae76b5:::mds1_inotable:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.733153903, 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036130300000000001, 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.854 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116525+0000", 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.455e-06 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.855 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116539+0000", 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4e-05 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116563+0000", 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4111999999999999e-05 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116577+0000", 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4059999999999999e-05 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.856 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116689+0000", 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011209000000000001 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116719+0000", 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.989e-05 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120082+0000", 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033628130000000001 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.857 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120137+0000", 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4610000000000001e-05 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 4 entries 25'3 (0'0) error 2:db457a8a:::201.00000001:head by mds.1.9:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.732097362, 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030622480000000001, 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.858 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.859 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117581+0000", 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0279999999999999e-06 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117596+0000", 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5109999999999999e-05 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117631+0000", 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4230000000000003e-05 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.860 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117655+0000", 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4987e-05 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117751+0000", 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5311999999999998e-05 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120642+0000", 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028915809999999998 2024-04-25T12:36:47.861 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 5 entries 25'4 (0'0) error 2:d80f8093:::201.00000002:head by mds.1.9:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.731596131, 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0025905469999999999, 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.862 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118082+0000", 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4000000000000001e-07 2024-04-25T12:36:47.863 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118091+0000", 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9919999999999996e-06 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118108+0000", 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7207999999999999e-05 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118124+0000", 2024-04-25T12:36:47.864 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5378000000000001e-05 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118203+0000", 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8929000000000003e-05 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120672+0000", 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024692999999999998 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.865 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:34 2.1b e25/18 2:db457a8a:::201.00000001:head v 25'5, mlcod=25'3)", 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.728121797, 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016516759999999999, 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.866 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121557+0000", 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.423e-06 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121568+0000", 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0499000000000001e-05 2024-04-25T12:36:47.867 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121585+0000", 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7283e-05 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121597+0000", 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1821000000000001e-05 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121653+0000", 2024-04-25T12:36:47.868 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6533000000000002e-05 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121671+0000", 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7285000000000001e-05 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123158+0000", 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0014869519999999999 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.869 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123207+0000", 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9879999999999997e-05 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:35 2.a 2:54ba6923:::mds1_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.70600948000001, 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016436459999999999, 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.870 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143670+0000", 2024-04-25T12:36:47.871 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4040000000000002e-06 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143685+0000", 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4768e-05 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143703+0000", 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7424000000000001e-05 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143721+0000", 2024-04-25T12:36:47.872 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8685000000000002e-05 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143824+0000", 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010286800000000001 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144032+0000", 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020812700000000001 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:47.873 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144205+0000", 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017276999999999999 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144982+0000", 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077719300000000002 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145196+0000", 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000213308 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145254+0000", 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8619000000000002e-05 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145312+0000", 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7479999999999999e-05 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.1.9:35 2.a e25/18 ondisk, result = 0)", 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:47.875 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.704582913, 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00022686700000000001, 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:47.876 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145096+0000", 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2309999999999999e-06 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145102+0000", 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1639999999999998e-06 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.877 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145115+0000", 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2809000000000001e-05 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145124+0000", 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9989999999999997e-06 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145178+0000", 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4287999999999999e-05 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.878 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145189+0000", 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.084e-05 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145321+0000", 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013253599999999999 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:11 2.c 2:36924187:::618.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:47.879 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.69733364699999, 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070050249999999998, 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.880 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152345+0000", 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6270000000000001e-06 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152355+0000", 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4069999999999999e-06 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.881 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152384+0000", 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9479e-05 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152399+0000", 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152478+0000", 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9702000000000004e-05 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.882 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152615+0000", 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013665500000000001 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152730+0000", 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011504499999999999 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156394+0000", 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036643259999999999 2024-04-25T12:36:47.883 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028370299999999999 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159289+0000", 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7268999999999997e-05 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159349+0000", 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0161999999999998e-05 2024-04-25T12:36:47.884 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.697247073, 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070895239999999998, 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.885 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152431+0000", 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6899999999999996e-07 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.886 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152440+0000", 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3259999999999996e-06 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152450+0000", 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0874e-05 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152470+0000", 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9658999999999999e-05 2024-04-25T12:36:47.887 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153265+0000", 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079454099999999995 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153349+0000", 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4381000000000004e-05 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153485+0000", 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136365 2024-04-25T12:36:47.888 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156400+0000", 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029144990000000001 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159418+0000", 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030182299999999998 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159477+0000", 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:36:47.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159520+0000", 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3127000000000002e-05 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.697183459, 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0068584939999999997, 2024-04-25T12:36:47.890 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152495+0000", 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.16e-07 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152501+0000", 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7819999999999999e-06 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152510+0000", 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9199999999999993e-06 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152518+0000", 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9090000000000006e-06 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152651+0000", 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000132349 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152795+0000", 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014392800000000001 2024-04-25T12:36:47.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152930+0000", 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013561399999999999 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156365+0000", 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034348830000000001 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159266+0000", 2024-04-25T12:36:47.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029006959999999999 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159296+0000", 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0026e-05 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159353+0000", 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6570999999999998e-05 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.895 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.697097496, 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069239999999999996, 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152581+0000", 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3399999999999998e-07 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152587+0000", 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4049999999999997e-06 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152598+0000", 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1043000000000001e-05 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152607+0000", 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9409999999999996e-06 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153448+0000", 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00084054699999999997 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153563+0000", 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000115584 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153719+0000", 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015566199999999999 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156373+0000", 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002653617 2024-04-25T12:36:47.899 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159423+0000", 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030508060000000001 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159472+0000", 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9042000000000002e-05 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159504+0000", 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1519000000000001e-05 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.900 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.69701461299999, 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0086667419999999998, 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.901 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152664+0000", 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0100000000000003e-07 2024-04-25T12:36:47.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152671+0000", 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8870000000000003e-06 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152679+0000", 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2730000000000002e-06 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152696+0000", 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7373000000000001e-05 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153747+0000", 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001050647 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153967+0000", 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00021982699999999999 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.904 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154122+0000", 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015472399999999999 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156405+0000", 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022838110000000002 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161248+0000", 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048425669999999999 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161283+0000", 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5166000000000002e-05 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161330+0000", 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6566000000000001e-05 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.906 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.2 epoch 25/18 rep_tid 20 entries 25'1 (0'0) error 2:40dbbc66:::202.00000003:head by mds.2.13:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.695408732, 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0021562719999999999, 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.907 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154270+0000", 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.638e-06 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154290+0000", 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392e-05 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154315+0000", 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4858000000000002e-05 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154335+0000", 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0044e-05 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154711+0000", 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037622099999999999 2024-04-25T12:36:47.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.910 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.910 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.910 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156425+0000", 2024-04-25T12:36:47.910 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017141190000000001 2024-04-25T12:36:47.910 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.910 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.910 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.913 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:11 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:47.913 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.690552838, 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00023418899999999999, 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:47.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159126+0000", 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7699999999999992e-07 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159131+0000", 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8109999999999999e-06 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.915 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159143+0000", 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1745e-05 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159153+0000", 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6520000000000001e-06 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159211+0000", 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8582999999999998e-05 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159225+0000", 2024-04-25T12:36:47.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3088000000000001e-05 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159359+0000", 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013433299999999999 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:15 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.690502442, 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000189507, 2024-04-25T12:36:47.917 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159176+0000", 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6899999999999998e-07 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159180+0000", 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1320000000000004e-06 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159188+0000", 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9319999999999996e-06 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159199+0000", 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1165999999999999e-05 2024-04-25T12:36:47.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5071000000000002e-05 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159262+0000", 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0160000000000008e-06 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159365+0000", 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010272099999999999 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.920 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:12 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.69046351199999, 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000315354, 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:47.921 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2099999999999998e-07 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159217+0000", 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.875e-06 2024-04-25T12:36:47.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159223+0000", 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2999999999999998e-06 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159229+0000", 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6740000000000002e-06 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159401+0000", 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017201599999999999 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159412+0000", 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0349e-05 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159529+0000", 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000117819 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:16 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:47.924 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.690435243, 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00026996799999999999, 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159243+0000", 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0100000000000001e-07 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159246+0000", 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.687e-06 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.072e-06 2024-04-25T12:36:47.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159261+0000", 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2169999999999999e-06 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159408+0000", 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000147637 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159417+0000", 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999996e-06 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159512+0000", 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5163999999999999e-05 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:23 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 112.688521088, 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000182072, 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:47.928 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161157+0000", 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7400000000000002e-07 2024-04-25T12:36:47.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161164+0000", 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161180+0000", 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6189999999999999e-05 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161191+0000", 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0981e-05 2024-04-25T12:36:47.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161238+0000", 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7509999999999997e-05 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161244+0000", 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.694e-06 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161338+0000", 2024-04-25T12:36:47.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4432999999999995e-05 2024-04-25T12:36:47.932 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.932 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.932 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.932 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:47.932 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:47.932 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:47.933 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:47.934 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:47.934+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:48.034 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:48.035+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:48.088 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:48.088 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:48.088 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:48.088 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:48.097 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:48.135 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:48.136+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:48.208 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:48.208 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:48.208 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:48.208 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:48.208 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:48.217 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:48.235 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:48.237+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:48.330 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:48.330 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:48.330 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:48.330 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:48.330 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.330 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 149.37899497800001, 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.331 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:36:48.332 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:36:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:36:48.334 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.335 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 149.37890732400001, 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.336 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.337 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.338 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.339 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.340 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 149.37856275600001, 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.341 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.342 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.343 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.344 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:36:48.345 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 149.37849028599999, 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.346 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:36:48.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:36:48.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 149.378416308, 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:48.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:36:48.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 149.37637893600001, 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:36:48.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 149.37551880000001, 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:36:48.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.214833145, 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:36:48.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:36:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.368 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.214757031, 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:36:48.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:36:48.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.214711362, 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.374 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:48.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:36:48.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:36:48.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:36:48.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.214666587, 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.379 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.380 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.381 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.382 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.208050163, 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.383 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.384 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.385 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.386 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.207990673, 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:48.387 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:36:48.388 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.389 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:36:48.390 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 113.17702038100001, 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.391 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.392 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.393 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.395 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 113.17694287, 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.396 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:36:48.397 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:48.398 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:36:48.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.400 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 113.17689502100001, 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035552270000000002, 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:48.401 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153105+0000", 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5600000000000005e-07 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.402 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153110+0000", 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2409999999999997e-06 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153118+0000", 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7689999999999996e-06 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153126+0000", 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1659999999999994e-06 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153194+0000", 2024-04-25T12:36:48.403 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8023999999999998e-05 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153351+0000", 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000156439 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153484+0000", 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000133008 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155770+0000", 2024-04-25T12:36:48.404 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022867830000000001 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156527+0000", 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075693600000000005 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156583+0000", 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5871000000000001e-05 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156661+0000", 2024-04-25T12:36:48.405 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7334000000000002e-05 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 113.176853131, 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.406 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:48.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:36:48.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.409 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.410 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 113.176818959, 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.411 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.412 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:36:48.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:36:48.414 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.415 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.416 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 113.17653621700001, 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.417 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.418 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:36:48.419 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.420 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 113.172395786, 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:48.421 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:36:48.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.423 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.424 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.424 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:48.424 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:48.424 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:48.425 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:48.426 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:48.426+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:48.526 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:48.527+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:48.537 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:48.537 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:48.546 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:48.564 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T12:36:48.626 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:48.627+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:48.660 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:48.660 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:48.660 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:48.660 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:48.660 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:48.670 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:48.674 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T12:36:48.754 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T12:36:48.754 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:48.755+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:48.759 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T12:36:48.759 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T12:36:48.822 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:48.822 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:48.823 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:48.823 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [] 2024-04-25T12:36:48.823 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:48.829 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:48.830+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:48.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T12:36:48.840 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T12:36:48.848 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:36:48.849 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T12:36:48.930 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:48.931+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:48.933 INFO:tasks.workunit:Running workunits matching fs/misc/trivial_sync.sh on client.0... 2024-04-25T12:36:48.934 INFO:tasks.workunit:Running workunit fs/misc/trivial_sync.sh... 2024-04-25T12:36:48.934 DEBUG:teuthology.orchestra.run.smithi123:workunit test fs/misc/trivial_sync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh 2024-04-25T12:36:49.031 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:49.032+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:49.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T12:36:49.096 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:49.096 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:49.096 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-25T12:36:49.096 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:49.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T12:36:49.160 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:49.161+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:49.232 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:49.234+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:49.333 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:49.334+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.345+0000 7f7d3b000640 1 Processor -- start 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.345+0000 7f7d3b000640 1 -- start start 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.347+0000 7f7d3b000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d340c4240 0x7f7d3416d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.347+0000 7f7d3b000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d340c2870 0x7f7d340c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.347+0000 7f7d3b000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d34073d50 con 0x7f7d34169220 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.347+0000 7f7d3b000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3416d6e0 con 0x7f7d340c2870 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.348+0000 7f7d3b000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7d34074640 con 0x7f7d340c4240 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.348+0000 7f7d2be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d340c2870 0x7f7d340c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.348+0000 7f7d33e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d340c4240 0x7f7d3416d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.348+0000 7f7d2be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d340c2870 0x7f7d340c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:47036/0 (socket says 172.21.15.123:47036) 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.348+0000 7f7d2be00640 1 -- 172.21.15.123:0/1117047118 learned_addr learned my addr 172.21.15.123:0/1117047118 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.348+0000 7f7d39000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7d34169220 0x7f7d340cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:49388/0 (socket says 172.21.15.123:49388) 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.366+0000 7f7d33e00640 1 -- 172.21.15.123:0/1117047118 >> v2:172.21.15.153:3300/0 conn(0x7f7d340c2870 msgr2=0x7f7d340c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.366+0000 7f7d33e00640 1 --2- 172.21.15.123:0/1117047118 >> v2:172.21.15.153:3300/0 conn(0x7f7d340c2870 0x7f7d340c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.366+0000 7f7d33e00640 1 -- 172.21.15.123:0/1117047118 >> v1:172.21.15.153:6789/0 conn(0x7f7d34169220 legacy=0x7f7d340cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.366+0000 7f7d33e00640 1 -- 172.21.15.123:0/1117047118 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d340c4710 con 0x7f7d340c4240 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.366+0000 7f7d2be00640 1 --2- 172.21.15.123:0/1117047118 >> v2:172.21.15.153:3300/0 conn(0x7f7d340c2870 0x7f7d340c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:49.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.366+0000 7f7d33400640 1 -- 172.21.15.123:0/1117047118 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307650088 0 0) 0x7f7d34073d50 con 0x7f7d34169220 2024-04-25T12:36:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.367+0000 7f7d33e00640 1 --2- 172.21.15.123:0/1117047118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d340c4240 0x7f7d3416d1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400c860 tx=0x7f7d2402a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32abb4e88335fbfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.367+0000 7f7d33400640 1 -- 172.21.15.123:0/1117047118 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24033610 con 0x7f7d340c4240 2024-04-25T12:36:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.367+0000 7f7d33400640 1 -- 172.21.15.123:0/1117047118 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d240337b0 con 0x7f7d340c4240 2024-04-25T12:36:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.367+0000 7f7d33400640 1 -- 172.21.15.123:0/1117047118 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24033b60 con 0x7f7d340c4240 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.368+0000 7f7d3b000640 1 -- 172.21.15.123:0/1117047118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d340c4240 msgr2=0x7f7d3416d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.368+0000 7f7d3b000640 1 --2- 172.21.15.123:0/1117047118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d340c4240 0x7f7d3416d1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400c860 tx=0x7f7d2402a8b0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.368+0000 7f7d3b000640 1 -- 172.21.15.123:0/1117047118 shutdown_connections 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.368+0000 7f7d3b000640 1 --2- 172.21.15.123:0/1117047118 >> v2:172.21.15.153:3300/0 conn(0x7f7d340c2870 0x7f7d340c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.368+0000 7f7d3b000640 1 --2- 172.21.15.123:0/1117047118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d340c4240 0x7f7d3416d1a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.368+0000 7f7d3b000640 1 -- 172.21.15.123:0/1117047118 >> 172.21.15.123:0/1117047118 conn(0x7f7d3406c750 msgr2=0x7f7d3406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.369+0000 7f7d3b000640 1 -- 172.21.15.123:0/1117047118 shutdown_connections 2024-04-25T12:36:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.369+0000 7f7d3b000640 1 -- 172.21.15.123:0/1117047118 wait complete. 2024-04-25T12:36:49.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.372+0000 7f7d3b000640 1 Processor -- start 2024-04-25T12:36:49.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.372+0000 7f7d3b000640 1 -- start start 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d3b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d340c2870 0x7f7d340c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d3b000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340c4240 0x7f7d34202a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d3b000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d34169220 0x7f7d34167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d3b000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d340c49f0 con 0x7f7d340c4240 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d3b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d340742d0 con 0x7f7d340c2870 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d3b000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7d34074ad0 con 0x7f7d34169220 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d33e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d340c2870 0x7f7d340c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d33e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d340c2870 0x7f7d340c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:47050/0 (socket says 172.21.15.123:47050) 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d33e00640 1 -- 172.21.15.123:0/2837213035 learned_addr learned my addr 172.21.15.123:0/2837213035 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:49.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d39000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d34169220 0x7f7d34167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.374+0000 7f7d2be00640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340c4240 0x7f7d34202a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d39000640 1 -- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d340c2870 msgr2=0x7f7d340c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d39000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d340c2870 0x7f7d340c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d39000640 1 -- 172.21.15.123:0/2837213035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340c4240 msgr2=0x7f7d34202a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d39000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340c4240 0x7f7d34202a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d39000640 1 -- 172.21.15.123:0/2837213035 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d34202fa0 con 0x7f7d34169220 2024-04-25T12:36:49.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d39000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d34169220 0x7f7d34167aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406e700 tx=0x7f7d2c00cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d30c00640 1 -- 172.21.15.123:0/2837213035 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c007270 con 0x7f7d34169220 2024-04-25T12:36:49.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d30c00640 1 -- 172.21.15.123:0/2837213035 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2c00a640 con 0x7f7d34169220 2024-04-25T12:36:49.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.375+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d342031b0 con 0x7f7d34169220 2024-04-25T12:36:49.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.376+0000 7f7d30c00640 1 -- 172.21.15.123:0/2837213035 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00aa10 con 0x7f7d34169220 2024-04-25T12:36:49.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.376+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341f3ec0 con 0x7f7d34169220 2024-04-25T12:36:49.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.377+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34076700 con 0x7f7d34169220 2024-04-25T12:36:49.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T12:36:49.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.379+0000 7f7d30c00640 1 -- 172.21.15.123:0/2837213035 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d2c00abb0 con 0x7f7d34169220 2024-04-25T12:36:49.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.380+0000 7f7d30c00640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7d100744b0 0x7f7d100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.381+0000 7f7d30c00640 1 -- 172.21.15.123:0/2837213035 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(67..67 src has 1..67) ==== 6685+0+0 (secure 0 0 0) 0x7f7d2c0952a0 con 0x7f7d34169220 2024-04-25T12:36:49.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.381+0000 7f7d33e00640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7d100744b0 0x7f7d100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.382+0000 7f7d33e00640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7d100744b0 0x7f7d100769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d2402a8f0 tx=0x7f7d24035040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.385+0000 7f7d30c00640 1 -- 172.21.15.123:0/2837213035 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d2c05f000 con 0x7f7d34169220 2024-04-25T12:36:49.413 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:49.413 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-25T12:36:49.414 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-04-25T12:36:49.414 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-25T12:36:49.414 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:49.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T12:36:49.434 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:49.435+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:49.535 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:49.536+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:49.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.593+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f7d3406cee0 con 0x7f7d34169220 2024-04-25T12:36:49.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.612+0000 7f09dfe00640 1 Processor -- start 2024-04-25T12:36:49.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.612+0000 7f09dfe00640 1 -- start start 2024-04-25T12:36:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dfe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e0165a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dfe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e00b9d90 con 0x7f09e0072d60 2024-04-25T12:36:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dfe00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e015a410 con 0x7f09e0154350 2024-04-25T12:36:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dfe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f09e015d780 con 0x7f09e015e320 2024-04-25T12:36:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09de000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f09e0072d60 0x7f09e015dc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56074/0 (socket says 172.21.15.123:56074) 2024-04-25T12:36:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09de000640 1 -- 172.21.15.123:0/4026160830 learned_addr learned my addr 172.21.15.123:0/4026160830 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09df400640 1 --2- 172.21.15.123:0/4026160830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e0165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dd600640 1 -- 172.21.15.123:0/4026160830 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203213502 0 0) 0x7f09e00b9d90 con 0x7f09e0072d60 2024-04-25T12:36:49.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dd600640 1 -- 172.21.15.123:0/4026160830 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09c4003520 con 0x7f09e0072d60 2024-04-25T12:36:49.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dd600640 1 -- 172.21.15.123:0/4026160830 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2908213429 0 0) 0x7f09e015a410 con 0x7f09e0154350 2024-04-25T12:36:49.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09dd600640 1 -- 172.21.15.123:0/4026160830 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09e00b9d90 con 0x7f09e0154350 2024-04-25T12:36:49.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.614+0000 7f09df400640 1 -- 172.21.15.123:0/4026160830 >> v1:172.21.15.153:6789/0 conn(0x7f09e0154350 legacy=0x7f09e00c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.615+0000 7f09df400640 1 -- 172.21.15.123:0/4026160830 >> v1:172.21.15.123:6789/0 conn(0x7f09e0072d60 legacy=0x7f09e015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.615+0000 7f09df400640 1 -- 172.21.15.123:0/4026160830 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e015a6d0 con 0x7f09e015e320 2024-04-25T12:36:49.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.616+0000 7f09df400640 1 --2- 172.21.15.123:0/4026160830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e0165a90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09e015d900 tx=0x7f09d802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d329d93374621de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.616+0000 7f09dd600640 1 -- 172.21.15.123:0/4026160830 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8007740 con 0x7f09e015e320 2024-04-25T12:36:49.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.616+0000 7f09dd600640 1 -- 172.21.15.123:0/4026160830 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d80078e0 con 0x7f09e015e320 2024-04-25T12:36:49.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.616+0000 7f09dd600640 1 -- 172.21.15.123:0/4026160830 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8007cb0 con 0x7f09e015e320 2024-04-25T12:36:49.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.617+0000 7f09dfe00640 1 -- 172.21.15.123:0/4026160830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 msgr2=0x7f09e0165a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.617+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4026160830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e0165a90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09e015d900 tx=0x7f09d802a6c0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.617+0000 7f09dfe00640 1 -- 172.21.15.123:0/4026160830 shutdown_connections 2024-04-25T12:36:49.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.617+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4026160830 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e0165a90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.617+0000 7f09dfe00640 1 -- 172.21.15.123:0/4026160830 >> 172.21.15.123:0/4026160830 conn(0x7f09e006cba0 msgr2=0x7f09e00cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:49.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.617+0000 7f09dfe00640 1 -- 172.21.15.123:0/4026160830 shutdown_connections 2024-04-25T12:36:49.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.617+0000 7f09dfe00640 1 -- 172.21.15.123:0/4026160830 wait complete. 2024-04-25T12:36:49.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.619+0000 7f09dfe00640 1 Processor -- start 2024-04-25T12:36:49.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.619+0000 7f09dfe00640 1 -- start start 2024-04-25T12:36:49.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.620+0000 7f09dfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e0072d60 0x7f09e0151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09e0154350 0x7f09e015c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dfe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e01f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09e015af70 con 0x7f09e0072d60 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dfe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09e015ac90 con 0x7f09e0154350 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dfe00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f09e015a100 con 0x7f09e015e320 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e0072d60 0x7f09e0151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e0072d60 0x7f09e0151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48708/0 (socket says 172.21.15.123:48708) 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dea00640 1 -- 172.21.15.123:0/4019950122 learned_addr learned my addr 172.21.15.123:0/4019950122 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e01f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09de000640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09e0154350 0x7f09e015c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09df400640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09e0154350 msgr2=0x7f09e015c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09e0154350 0x7f09e015c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09df400640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e0072d60 msgr2=0x7f09e0151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e0072d60 0x7f09e0151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09df400640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e015cc20 con 0x7f09e015e320 2024-04-25T12:36:49.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.621+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e0072d60 0x7f09e0151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:49.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.622+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e01f3a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09d8031000 tx=0x7f09d8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.622+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d803f070 con 0x7f09e015e320 2024-04-25T12:36:49.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.622+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d8031c00 con 0x7f09e015e320 2024-04-25T12:36:49.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.622+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e015b640 con 0x7f09e015e320 2024-04-25T12:36:49.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.622+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8009450 con 0x7f09e015e320 2024-04-25T12:36:49.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.622+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e015bc00 con 0x7f09e015e320 2024-04-25T12:36:49.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.623+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f09e0076da0 con 0x7f09e015e320 2024-04-25T12:36:49.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.625+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09d8044460 con 0x7f09e015e320 2024-04-25T12:36:49.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.626+0000 7f09d6a00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09c00744e0 0x7f09c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.626+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(67..67 src has 1..67) ==== 6685+0+0 (secure 0 0 0) 0x7f09d80b6b20 con 0x7f09e015e320 2024-04-25T12:36:49.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.627+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09c00744e0 0x7f09c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.627+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v67) ==== 54+0+16 (secure 0 0 0) 0x7f09d8049060 con 0x7f09e015e320 2024-04-25T12:36:49.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.627+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f09e015cdf0 0x7f09e01f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.627+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09e014e410 con 0x7f09e015cdf0 2024-04-25T12:36:49.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.627+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09c00744e0 0x7f09c00769f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f09d0007fd0 tx=0x7f09d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.627+0000 7f09de000640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f09e015cdf0 0x7f09e01f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.628+0000 7f09de000640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f09e015cdf0 0x7f09e01f3fb0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.629+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e014e410 con 0x7f09e015cdf0 2024-04-25T12:36:49.636 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:49.637+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:49.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.645+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e015b3f0 con 0x7f09e015cdf0 2024-04-25T12:36:49.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.648+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e015b3f0 con 0x7f09e015cdf0 2024-04-25T12:36:49.647 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:49.648 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:49.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.649+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f09e01f44f0 0x7f09e0149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.649+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f09e0073850 con 0x7f09e01f44f0 2024-04-25T12:36:49.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.649+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f09e01f44f0 0x7f09e0149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.650+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f09e01f44f0 0x7f09e0149c80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.650+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e0073850 con 0x7f09e01f44f0 2024-04-25T12:36:49.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.667+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e00c8a90 con 0x7f09e01f44f0 2024-04-25T12:36:49.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.669+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e00c8a90 con 0x7f09e01f44f0 2024-04-25T12:36:49.668 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:49.668 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:49.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.669+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f09e01f0320 0x7f09e01f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.669+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f09e01f0c70 con 0x7f09e01f0320 2024-04-25T12:36:49.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.669+0000 7f09de000640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f09e01f0320 0x7f09e01f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.670+0000 7f09de000640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f09e01f0320 0x7f09e01f0730 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.671+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e01f0c70 con 0x7f09e01f0320 2024-04-25T12:36:49.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.686+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e01f1180 con 0x7f09e01f0320 2024-04-25T12:36:49.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.689+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e01f1180 con 0x7f09e01f0320 2024-04-25T12:36:49.688 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:49.688 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:49.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.689+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f09e00638b0 0x7f09e00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.689+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f09e01f1100 con 0x7f09e00638b0 2024-04-25T12:36:49.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.689+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f09e00638b0 0x7f09e00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.690+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f09e00638b0 0x7f09e00cb230 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.691+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e01f1100 con 0x7f09e00638b0 2024-04-25T12:36:49.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.706+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e0063f90 con 0x7f09e00638b0 2024-04-25T12:36:49.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.708+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e0063f90 con 0x7f09e00638b0 2024-04-25T12:36:49.707 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:49.707 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:49.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.708+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f09e00cbc80 0x7f09e002c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.708+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f09e00cc090 con 0x7f09e00cbc80 2024-04-25T12:36:49.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.709+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f09e00cbc80 0x7f09e002c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.710+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f09e00cbc80 0x7f09e002c9d0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.711+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e00cc090 con 0x7f09e00cbc80 2024-04-25T12:36:49.721 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:49.721 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:49.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.725+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e00cc090 con 0x7f09e00cbc80 2024-04-25T12:36:49.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.728+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e00cc090 con 0x7f09e00cbc80 2024-04-25T12:36:49.727 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:49.728 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:49.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.728+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f09e002d420 0x7f09e002d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.728+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f09e002deb0 con 0x7f09e002d420 2024-04-25T12:36:49.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.729+0000 7f09de000640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f09e002d420 0x7f09e002d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.729+0000 7f09de000640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f09e002d420 0x7f09e002d830 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.730+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e002deb0 con 0x7f09e002d420 2024-04-25T12:36:49.737 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:49.738+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:49.741 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T12:36:49.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.745+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e002deb0 con 0x7f09e002d420 2024-04-25T12:36:49.748 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:49.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.747+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e002deb0 con 0x7f09e002d420 2024-04-25T12:36:49.749 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:49.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.747+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f09e02073a0 0x7f09e02077b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.747+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f09e020c8b0 con 0x7f09e02073a0 2024-04-25T12:36:49.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.747+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f09e02073a0 0x7f09e02077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.748+0000 7f09df400640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f09e02073a0 0x7f09e02077b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.749+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e020c8b0 con 0x7f09e02073a0 2024-04-25T12:36:49.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.763+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e0207de0 con 0x7f09e02073a0 2024-04-25T12:36:49.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.765+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e0207de0 con 0x7f09e02073a0 2024-04-25T12:36:49.764 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:49.765 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:49.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.765+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f09e0208530 0x7f09e0208940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:49.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.765+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f09e0208e80 con 0x7f09e0208530 2024-04-25T12:36:49.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.766+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f09e0208530 0x7f09e0208940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:49.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.766+0000 7f09dea00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f09e0208530 0x7f09e0208940 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:49.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.767+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e0208e80 con 0x7f09e0208530 2024-04-25T12:36:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.784+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f09e0208e80 con 0x7f09e0208530 2024-04-25T12:36:49.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.786+0000 7f09d6a00640 1 -- 172.21.15.123:0/4019950122 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f09e0208e80 con 0x7f09e0208530 2024-04-25T12:36:49.785 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f09e015cdf0 msgr2=0x7f09e01f3fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7f09e015cdf0 0x7f09e01f3fb0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f09e01f44f0 msgr2=0x7f09e0149c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f09e01f44f0 0x7f09e0149c80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f09e01f0320 msgr2=0x7f09e01f0730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f09e01f0320 0x7f09e01f0730 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f09e00638b0 msgr2=0x7f09e00cb230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7f09e00638b0 0x7f09e00cb230 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f09e00cbc80 msgr2=0x7f09e002c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09df400640 1 -- 172.21.15.123:0/4019950122 reap_dead start 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f09e00cbc80 0x7f09e002c9d0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09df400640 1 -- 172.21.15.123:0/4019950122 reap_dead start 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f09e002d420 msgr2=0x7f09e002d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f09e002d420 0x7f09e002d830 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f09e02073a0 msgr2=0x7f09e02077b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f09e02073a0 0x7f09e02077b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f09e0208530 msgr2=0x7f09e0208940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f09e0208530 0x7f09e0208940 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09c00744e0 msgr2=0x7f09c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09c00744e0 0x7f09c00769f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f09d0007fd0 tx=0x7f09d0009040 comp rx=0 tx=0).stop 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 msgr2=0x7f09e01f3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09df400640 1 -- 172.21.15.123:0/4019950122 reap_dead start 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e01f3a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09d8031000 tx=0x7f09d8004a40 comp rx=0 tx=0).stop 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 shutdown_connections 2024-04-25T12:36:49.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 --2- 172.21.15.123:0/4019950122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09e015e320 0x7f09e01f3a70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:49.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 >> 172.21.15.123:0/4019950122 conn(0x7f09e006cba0 msgr2=0x7f09e0158730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:49.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 shutdown_connections 2024-04-25T12:36:49.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:49.787+0000 7f09dfe00640 1 -- 172.21.15.123:0/4019950122 wait complete. 2024-04-25T12:36:49.838 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:49.839+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:49.938 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:49.939+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:49.990 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:49.990 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:50.011 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T12:36:50.061 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:50.062+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:50.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.093+0000 7f7d30c00640 1 -- 172.21.15.123:0/2837213035 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v68) ==== 87+0+0 (secure 0 0 0) 0x7f7d2c063300 con 0x7f7d34169220 2024-04-25T12:36:50.092 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is set 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.096+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7d100744b0 msgr2=0x7f7d100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.096+0000 7f7d3b000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7d100744b0 0x7f7d100769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d2402a8f0 tx=0x7f7d24035040 comp rx=0 tx=0).stop 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.096+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d34169220 msgr2=0x7f7d34167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.096+0000 7f7d3b000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d34169220 0x7f7d34167aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406e700 tx=0x7f7d2c00cbe0 comp rx=0 tx=0).stop 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 shutdown_connections 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7d100744b0 0x7f7d100769c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d34169220 0x7f7d34167aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340c4240 0x7f7d34202a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 --2- 172.21.15.123:0/2837213035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d340c2870 0x7f7d340c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:50.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 >> 172.21.15.123:0/2837213035 conn(0x7f7d3406c750 msgr2=0x7f7d340c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:50.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 shutdown_connections 2024-04-25T12:36:50.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:50.097+0000 7f7d3b000640 1 -- 172.21.15.123:0/2837213035 wait complete. 2024-04-25T12:36:50.163 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:50.164+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:50.241 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:50.242+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:50.265 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:50.265 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:50.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T12:36:50.375 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:50.376+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:50.443 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:50.444+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:50.544 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:50.545+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:50.584 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:50.585 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:50.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T12:36:50.645 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:50.645+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:50.746 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:50.747+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:50.847 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:50.848+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:50.918 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:50.919 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:50.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T12:36:50.988 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:50.989+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:51.049 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:51.050+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:51.059 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-04-25T12:36:51.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-25T12:36:51.149 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-04-25T12:36:51.150+0000 7f40872a1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34705) UID: 0 2024-04-25T12:36:51.159 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T12:36:51.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T12:36:51.212 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:51.212 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:51.236 INFO:tasks.workunit:Stopping ['fs/misc/trivial_sync.sh'] on client.0... 2024-04-25T12:36:51.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T12:36:51.237 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T12:36:51.250 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-04-25T12:36:51.251+0000 7f699b4ba640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34710) UID: 0 2024-04-25T12:36:51.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.325+0000 7f7feb200640 1 Processor -- start 2024-04-25T12:36:51.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.325+0000 7f7feb200640 1 -- start start 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.326+0000 7f7feb200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7fec15c420 0x7f7fec0c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7feb200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7fec0caa70 0x7f7fec0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7feb200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fec0c43d0 con 0x7f7fec074840 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7feb200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7fec06c7b0 con 0x7f7fec15c420 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7feb200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7fec06ca00 con 0x7f7fec0caa70 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7fe9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7fec15c420 0x7f7fec0c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7fea800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7fec074840 0x7f7fec15f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56090/0 (socket says 172.21.15.123:56090) 2024-04-25T12:36:51.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7fea800640 1 -- 172.21.15.123:0/3072928721 learned_addr learned my addr 172.21.15.123:0/3072928721 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:51.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7fe9400640 1 --2- 172.21.15.123:0/3072928721 >> v2:172.21.15.153:3300/0 conn(0x7f7fec0caa70 0x7f7fec0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:51.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7fe9e00640 1 -- 172.21.15.123:0/3072928721 >> v2:172.21.15.153:3300/0 conn(0x7f7fec0caa70 msgr2=0x7f7fec0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:51.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.327+0000 7f7fe9e00640 1 --2- 172.21.15.123:0/3072928721 >> v2:172.21.15.153:3300/0 conn(0x7f7fec0caa70 0x7f7fec0c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:51.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.328+0000 7f7fe9e00640 1 -- 172.21.15.123:0/3072928721 >> v1:172.21.15.123:6789/0 conn(0x7f7fec074840 legacy=0x7f7fec15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:51.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.328+0000 7f7fe9e00640 1 -- 172.21.15.123:0/3072928721 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec06cce0 con 0x7f7fec15c420 2024-04-25T12:36:51.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.329+0000 7f7fe9e00640 1 --2- 172.21.15.123:0/3072928721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 0x7f7fec0c82e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0c3640 tx=0x7f7fdc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e45518838d7d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:51.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.329+0000 7f7fe8a00640 1 -- 172.21.15.123:0/3072928721 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc037070 con 0x7f7fec15c420 2024-04-25T12:36:51.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.329+0000 7f7fe8a00640 1 -- 172.21.15.123:0/3072928721 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc032440 con 0x7f7fec15c420 2024-04-25T12:36:51.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.329+0000 7f7fe8a00640 1 -- 172.21.15.123:0/3072928721 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc032810 con 0x7f7fec15c420 2024-04-25T12:36:51.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.330+0000 7f7feb200640 1 -- 172.21.15.123:0/3072928721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 msgr2=0x7f7fec0c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:51.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.330+0000 7f7feb200640 1 --2- 172.21.15.123:0/3072928721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 0x7f7fec0c82e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0c3640 tx=0x7f7fdc02a1a0 comp rx=0 tx=0).stop 2024-04-25T12:36:51.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.330+0000 7f7feb200640 1 -- 172.21.15.123:0/3072928721 shutdown_connections 2024-04-25T12:36:51.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.330+0000 7f7feb200640 1 --2- 172.21.15.123:0/3072928721 >> v2:172.21.15.153:3300/0 conn(0x7f7fec0caa70 0x7f7fec0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:51.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.330+0000 7f7feb200640 1 --2- 172.21.15.123:0/3072928721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 0x7f7fec0c82e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:51.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.330+0000 7f7feb200640 1 -- 172.21.15.123:0/3072928721 >> 172.21.15.123:0/3072928721 conn(0x7f7fec070b70 msgr2=0x7f7fec070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:51.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.330+0000 7f7feb200640 1 -- 172.21.15.123:0/3072928721 shutdown_connections 2024-04-25T12:36:51.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.331+0000 7f7feb200640 1 -- 172.21.15.123:0/3072928721 wait complete. 2024-04-25T12:36:51.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.334+0000 7f7feb200640 1 Processor -- start 2024-04-25T12:36:51.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.334+0000 7f7feb200640 1 -- start start 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.336+0000 7f7feb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fec074840 0x7f7fec1f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.336+0000 7f7feb200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fec0caa70 0x7f7fec14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.336+0000 7f7feb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 0x7f7fec0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.336+0000 7f7feb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0c8820 con 0x7f7fec15c420 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.336+0000 7f7feb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0c7dc0 con 0x7f7fec074840 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.336+0000 7f7feb200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec0c3f70 con 0x7f7fec0caa70 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fec074840 0x7f7fec1f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:51.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fec074840 0x7f7fec1f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:47072/0 (socket says 172.21.15.123:47072) 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 -- 172.21.15.123:0/2976725964 learned_addr learned my addr 172.21.15.123:0/2976725964 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fea800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 0x7f7fec0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9400640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fec0caa70 0x7f7fec14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 -- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fec0caa70 msgr2=0x7f7fec14ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fec0caa70 0x7f7fec14ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 -- 172.21.15.123:0/2976725964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 msgr2=0x7f7fec0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 0x7f7fec0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:51.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.337+0000 7f7fe9e00640 1 -- 172.21.15.123:0/2976725964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec1f64d0 con 0x7f7fec074840 2024-04-25T12:36:51.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.338+0000 7f7fe9e00640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fec074840 0x7f7fec1f5f90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009890 tx=0x7f7fdc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:51.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.338+0000 7f7fe2000640 1 -- 172.21.15.123:0/2976725964 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc037040 con 0x7f7fec074840 2024-04-25T12:36:51.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.338+0000 7f7fe2000640 1 -- 172.21.15.123:0/2976725964 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc042070 con 0x7f7fec074840 2024-04-25T12:36:51.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.338+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec155920 con 0x7f7fec074840 2024-04-25T12:36:51.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.338+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1f2170 con 0x7f7fec074840 2024-04-25T12:36:51.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.338+0000 7f7fe2000640 1 -- 172.21.15.123:0/2976725964 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc030960 con 0x7f7fec074840 2024-04-25T12:36:51.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.340+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc4005290 con 0x7f7fec074840 2024-04-25T12:36:51.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.341+0000 7f7fe2000640 1 -- 172.21.15.123:0/2976725964 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7fdc030b00 con 0x7f7fec074840 2024-04-25T12:36:51.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.343+0000 7f7fe2000640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7fcc0744b0 0x7f7fcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:51.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.343+0000 7f7fe2000640 1 -- 172.21.15.123:0/2976725964 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6685+0+0 (secure 0 0 0) 0x7f7fdc0b5f00 con 0x7f7fec074840 2024-04-25T12:36:51.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.343+0000 7f7fe9400640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7fcc0744b0 0x7f7fcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:51.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.344+0000 7f7fe9400640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7fcc0744b0 0x7f7fcc0769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0c7c90 tx=0x7f7fd8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:51.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.347+0000 7f7fe2000640 1 -- 172.21.15.123:0/2976725964 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fdc07fbd0 con 0x7f7fec074840 2024-04-25T12:36:51.350 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:51.351+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:51.451 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:51.452+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:51.531 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:51.531 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:51.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T12:36:51.552 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:51.553+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:51.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:51.560+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7f7fc4005580 con 0x7f7fec074840 2024-04-25T12:36:51.653 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:51.654+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:51.754 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:51.755+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:51.795 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:51.795 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:51.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T12:36:51.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T12:36:51.857 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:51.858+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:51.957 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T12:36:51.958+0000 7f86fd400640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33454) UID: 0 2024-04-25T12:36:52.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.054+0000 7fefc9600640 1 Processor -- start 2024-04-25T12:36:52.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.054+0000 7fefc9600640 1 -- start start 2024-04-25T12:36:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.056+0000 7fefc9600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefc40c8fd0 0x7fefc4153640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.056+0000 7fefc9600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc4074400 con 0x7fefc40c8170 2024-04-25T12:36:52.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.056+0000 7fefc9600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc4153b80 con 0x7fefc41426a0 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.056+0000 7fefc9600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fefc41595e0 con 0x7fefc40c8fd0 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefc40c8fd0 0x7fefc4153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc3400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fefc40c8170 0x7fefc40ca200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56106/0 (socket says 172.21.15.123:56106) 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefc40c8fd0 0x7fefc4153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48732/0 (socket says 172.21.15.123:48732) 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc3400640 1 -- 172.21.15.123:0/1888417803 learned_addr learned my addr 172.21.15.123:0/1888417803 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc1600640 1 -- 172.21.15.123:0/1888417803 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 662492927 0 0) 0x7fefb0001110 con 0x7fefc41426a0 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc1600640 1 -- 172.21.15.123:0/1888417803 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefa8003520 con 0x7fefc41426a0 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc1600640 1 -- 172.21.15.123:0/1888417803 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2779391552 0 0) 0x7fefc4074400 con 0x7fefc40c8170 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc1600640 1 -- 172.21.15.123:0/1888417803 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefb0001110 con 0x7fefc40c8170 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc2a00640 1 -- 172.21.15.123:0/1888417803 >> v1:172.21.15.153:6789/0 conn(0x7fefc41426a0 legacy=0x7fefc4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc2a00640 1 -- 172.21.15.123:0/1888417803 >> v1:172.21.15.123:6789/0 conn(0x7fefc40c8170 legacy=0x7fefc40ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.057+0000 7fefc2a00640 1 -- 172.21.15.123:0/1888417803 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc4074670 con 0x7fefc40c8fd0 2024-04-25T12:36:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.058+0000 7fefc2a00640 1 --2- 172.21.15.123:0/1888417803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8fd0 0x7fefc4153640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fefb40097a0 tx=0x7fefb402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9ff3cc15fedf13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.059+0000 7fefc1600640 1 -- 172.21.15.123:0/1888417803 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4037070 con 0x7fefc40c8fd0 2024-04-25T12:36:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.059+0000 7fefc1600640 1 -- 172.21.15.123:0/1888417803 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb4032440 con 0x7fefc40c8fd0 2024-04-25T12:36:52.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.059+0000 7fefc1600640 1 -- 172.21.15.123:0/1888417803 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4032810 con 0x7fefc40c8fd0 2024-04-25T12:36:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.060+0000 7fefc9600640 1 -- 172.21.15.123:0/1888417803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8fd0 msgr2=0x7fefc4153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.060+0000 7fefc9600640 1 --2- 172.21.15.123:0/1888417803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8fd0 0x7fefc4153640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fefb40097a0 tx=0x7fefb402a230 comp rx=0 tx=0).stop 2024-04-25T12:36:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.060+0000 7fefc9600640 1 -- 172.21.15.123:0/1888417803 shutdown_connections 2024-04-25T12:36:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.060+0000 7fefc9600640 1 --2- 172.21.15.123:0/1888417803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8fd0 0x7fefc4153640 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.060+0000 7fefc9600640 1 -- 172.21.15.123:0/1888417803 >> 172.21.15.123:0/1888417803 conn(0x7fefc406f440 msgr2=0x7fefc406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.060+0000 7fefc9600640 1 -- 172.21.15.123:0/1888417803 shutdown_connections 2024-04-25T12:36:52.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.060+0000 7fefc9600640 1 -- 172.21.15.123:0/1888417803 wait complete. 2024-04-25T12:36:52.060 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:52.059+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:52.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.062+0000 7fefc9600640 1 Processor -- start 2024-04-25T12:36:52.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.062+0000 7fefc9600640 1 -- start start 2024-04-25T12:36:52.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8170 0x7fefc41e0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc40c8fd0 0x7fefc41f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc9600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefc41426a0 0x7fefc40c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc9600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40c4880 con 0x7fefc40c8170 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc9600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41598a0 con 0x7fefc40c8fd0 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc9600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fefc40ca910 con 0x7fefc41426a0 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8170 0x7fefc41e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc40c8fd0 0x7fefc41f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc3400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefc41426a0 0x7fefc40c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8170 0x7fefc41e0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48748/0 (socket says 172.21.15.123:48748) 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.064+0000 7fefc2a00640 1 -- 172.21.15.123:0/4072009698 learned_addr learned my addr 172.21.15.123:0/4072009698 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc2a00640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefc41426a0 msgr2=0x7fefc40c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefc41426a0 0x7fefc40c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc2a00640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc40c8fd0 msgr2=0x7fefc41f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc40c8fd0 0x7fefc41f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc2a00640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc41f0ba0 con 0x7fefc40c8170 2024-04-25T12:36:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc2000640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc40c8fd0 0x7fefc41f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc3400640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefc41426a0 0x7fefc40c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8170 0x7fefc41e0520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fefb40098c0 tx=0x7fefb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4041030 con 0x7fefc40c8170 2024-04-25T12:36:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb403d4e0 con 0x7fefc40c8170 2024-04-25T12:36:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.065+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc40c75b0 con 0x7fefc40c8170 2024-04-25T12:36:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.066+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4037040 con 0x7fefc40c8170 2024-04-25T12:36:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.066+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41df610 con 0x7fefc40c8170 2024-04-25T12:36:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.067+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fefc4076700 con 0x7fefc40c8170 2024-04-25T12:36:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.068+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fefb4044070 con 0x7fefc40c8170 2024-04-25T12:36:52.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.070+0000 7fefbaa00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefa40744e0 0x7fefa40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.070+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6685+0+0 (secure 0 0 0) 0x7fefb40b5ee0 con 0x7fefc40c8170 2024-04-25T12:36:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.070+0000 7fefc2000640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefa40744e0 0x7fefa40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.070+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v68) ==== 54+0+16 (secure 0 0 0) 0x7fefb404b020 con 0x7fefc40c8170 2024-04-25T12:36:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.071+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fefc40c57a0 0x7fefc4075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.071+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefc40c7ad0 con 0x7fefc40c57a0 2024-04-25T12:36:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.071+0000 7fefc2000640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefa40744e0 0x7fefa40769f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefc41ee0b0 tx=0x7fefb0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.071+0000 7fefc3400640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fefc40c57a0 0x7fefc4075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.072+0000 7fefc3400640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fefc40c57a0 0x7fefc4075cd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.072+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc40c7ad0 con 0x7fefc40c57a0 2024-04-25T12:36:52.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.088+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc40ca410 con 0x7fefc40c57a0 2024-04-25T12:36:52.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.090+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc40ca410 con 0x7fefc40c57a0 2024-04-25T12:36:52.089 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0: {} 2024-04-25T12:36:52.090 INFO:teuthology.orchestra.run.smithi123.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T12:36:52.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.091+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fefc41f0db0 0x7fefc41e3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.091+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fefc40cdc80 con 0x7fefc41f0db0 2024-04-25T12:36:52.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.091+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fefc41f0db0 0x7fefc41e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.091+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fefc41f0db0 0x7fefc41e3900 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.093+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc40cdc80 con 0x7fefc41f0db0 2024-04-25T12:36:52.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.103+0000 7f7fe2000640 1 -- 172.21.15.123:0/2976725964 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v69) ==== 113+0+0 (secure 0 0 0) 0x7f7fdc0882a0 con 0x7f7fec074840 2024-04-25T12:36:52.102 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.0 to 1 (10000) 2024-04-25T12:36:52.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7fcc0744b0 msgr2=0x7f7fcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7fcc0744b0 0x7f7fcc0769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0c7c90 tx=0x7f7fd8009190 comp rx=0 tx=0).stop 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fec074840 msgr2=0x7f7fec1f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fec074840 0x7f7fec1f5f90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009890 tx=0x7f7fdc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 shutdown_connections 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7fcc0744b0 0x7f7fcc0769c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fec15c420 0x7f7fec0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fec0caa70 0x7f7fec14ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 --2- 172.21.15.123:0/2976725964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fec074840 0x7f7fec1f5f90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 >> 172.21.15.123:0/2976725964 conn(0x7f7fec070b70 msgr2=0x7f7fec070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.106+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 shutdown_connections 2024-04-25T12:36:52.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.107+0000 7f7feb200640 1 -- 172.21.15.123:0/2976725964 wait complete. 2024-04-25T12:36:52.116 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-25T12:36:52.116 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T12:36:52.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T12:36:52.127 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1: {} 2024-04-25T12:36:52.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.125+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc41df980 con 0x7fefc41f0db0 2024-04-25T12:36:52.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.127+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc41df980 con 0x7fefc41f0db0 2024-04-25T12:36:52.127 INFO:teuthology.orchestra.run.smithi123.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T12:36:52.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.127+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fefc41f4100 0x7fefc4062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.127+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fefc41f4510 con 0x7fefc41f4100 2024-04-25T12:36:52.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.128+0000 7fefc2000640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fefc41f4100 0x7fefc4062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.128+0000 7fefc2000640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fefc41f4100 0x7fefc4062fe0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.130+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc41f4510 con 0x7fefc41f4100 2024-04-25T12:36:52.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-25T12:36:52.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T12:36:52.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.155+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc41f4510 con 0x7fefc41f4100 2024-04-25T12:36:52.159 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2: {} 2024-04-25T12:36:52.160 INFO:teuthology.orchestra.run.smithi123.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T12:36:52.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.157+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc41f4510 con 0x7fefc41f4100 2024-04-25T12:36:52.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.157+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fefc4140200 0x7fefc4140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.157+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fefc41f4510 con 0x7fefc4140200 2024-04-25T12:36:52.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.157+0000 7fefc3400640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fefc4140200 0x7fefc4140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.158+0000 7fefc3400640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fefc4140200 0x7fefc4140610 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.159+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc41f4510 con 0x7fefc4140200 2024-04-25T12:36:52.163 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T12:36:52.165+0000 7f3e07000640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33439) UID: 0 2024-04-25T12:36:52.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.174+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc41f4510 con 0x7fefc4140200 2024-04-25T12:36:52.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.177+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc41f4510 con 0x7fefc4140200 2024-04-25T12:36:52.176 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3: {} 2024-04-25T12:36:52.177 INFO:teuthology.orchestra.run.smithi123.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T12:36:52.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.178+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fefc402c9d0 0x7fefc402cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.178+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fefc41f4510 con 0x7fefc402c9d0 2024-04-25T12:36:52.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.178+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fefc402c9d0 0x7fefc402cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.179+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fefc402c9d0 0x7fefc402cde0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.180+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc41f4510 con 0x7fefc402c9d0 2024-04-25T12:36:52.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.201+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc41f4510 con 0x7fefc402c9d0 2024-04-25T12:36:52.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.203+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc41f4510 con 0x7fefc402c9d0 2024-04-25T12:36:52.202 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4: {} 2024-04-25T12:36:52.203 INFO:teuthology.orchestra.run.smithi123.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T12:36:52.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.204+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fefc402d830 0x7fefc4206d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.204+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fefc41411a0 con 0x7fefc402d830 2024-04-25T12:36:52.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.204+0000 7fefc2000640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fefc402d830 0x7fefc4206d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.205+0000 7fefc2000640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fefc402d830 0x7fefc4206d80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.206+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc41411a0 con 0x7fefc402d830 2024-04-25T12:36:52.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.227+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc402dca0 con 0x7fefc402d830 2024-04-25T12:36:52.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.230+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc402dca0 con 0x7fefc402d830 2024-04-25T12:36:52.229 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5: {} 2024-04-25T12:36:52.229 INFO:teuthology.orchestra.run.smithi123.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T12:36:52.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.230+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fefc402dc40 0x7fefc42097e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.230+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fefc420c810 con 0x7fefc402dc40 2024-04-25T12:36:52.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.231+0000 7fefc3400640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fefc402dc40 0x7fefc42097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.231+0000 7fefc3400640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fefc402dc40 0x7fefc42097e0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.232+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc420c810 con 0x7fefc402dc40 2024-04-25T12:36:52.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.252+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc420b550 con 0x7fefc402dc40 2024-04-25T12:36:52.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.255+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc420b550 con 0x7fefc402dc40 2024-04-25T12:36:52.254 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6: {} 2024-04-25T12:36:52.254 INFO:teuthology.orchestra.run.smithi123.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T12:36:52.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.255+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fefc420bc70 0x7fefc420c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.255+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fefc420fbe0 con 0x7fefc420bc70 2024-04-25T12:36:52.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.255+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fefc420bc70 0x7fefc420c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.256+0000 7fefc2a00640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fefc420bc70 0x7fefc420c080 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.257+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc420fbe0 con 0x7fefc420bc70 2024-04-25T12:36:52.260 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T12:36:52.262+0000 7f8b6fe00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33447) UID: 0 2024-04-25T12:36:52.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.288+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fefc420ea60 con 0x7fefc420bc70 2024-04-25T12:36:52.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.290+0000 7fefbaa00640 1 -- 172.21.15.123:0/4072009698 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fefc420ea60 con 0x7fefc420bc70 2024-04-25T12:36:52.289 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7: {} 2024-04-25T12:36:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T12:36:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fefc40c57a0 msgr2=0x7fefc4075cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fefc40c57a0 0x7fefc4075cd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fefc41f0db0 msgr2=0x7fefc41e3900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7fefc41f0db0 0x7fefc41e3900 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fefc41f4100 msgr2=0x7fefc4062fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fefc41f4100 0x7fefc4062fe0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fefc4140200 msgr2=0x7fefc4140610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fefc4140200 0x7fefc4140610 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fefc402c9d0 msgr2=0x7fefc402cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fefc402c9d0 0x7fefc402cde0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fefc402d830 msgr2=0x7fefc4206d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fefc402d830 0x7fefc4206d80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fefc402dc40 msgr2=0x7fefc42097e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fefc402dc40 0x7fefc42097e0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fefc420bc70 msgr2=0x7fefc420c080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fefc420bc70 0x7fefc420c080 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc3400640 1 -- 172.21.15.123:0/4072009698 reap_dead start 2024-04-25T12:36:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc3400640 1 -- 172.21.15.123:0/4072009698 reap_dead start 2024-04-25T12:36:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc3400640 1 -- 172.21.15.123:0/4072009698 reap_dead start 2024-04-25T12:36:52.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc3400640 1 -- 172.21.15.123:0/4072009698 reap_dead start 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc3400640 1 -- 172.21.15.123:0/4072009698 reap_dead start 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefa40744e0 msgr2=0x7fefa40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefa40744e0 0x7fefa40769f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefc41ee0b0 tx=0x7fefb0009040 comp rx=0 tx=0).stop 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8170 msgr2=0x7fefc41e0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.291+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8170 0x7fefc41e0520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fefb40098c0 tx=0x7fefb40046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.292+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 shutdown_connections 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.292+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fefa40744e0 0x7fefa40769f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.292+0000 7fefc9600640 1 --2- 172.21.15.123:0/4072009698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefc40c8170 0x7fefc41e0520 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.292+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 >> 172.21.15.123:0/4072009698 conn(0x7fefc406f440 msgr2=0x7fefc4158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.292+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 shutdown_connections 2024-04-25T12:36:52.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.292+0000 7fefc9600640 1 -- 172.21.15.123:0/4072009698 wait complete. 2024-04-25T12:36:52.361 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-04-25T12:36:52.362+0000 7f2d6dd10640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 42227) UID: 0 2024-04-25T12:36:52.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.390+0000 7f62c7c00640 1 Processor -- start 2024-04-25T12:36:52.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.391+0000 7f62c7c00640 1 -- start start 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.392+0000 7f62c7c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f62c00b71c0 0x7f62c00cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c7c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b7680 0x7f62c00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c7c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c00c7c70 con 0x7f62c0156b90 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c7c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f62c00c6cc0 con 0x7f62c00b71c0 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c7c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00c69e0 con 0x7f62c00b7680 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c5c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b7680 0x7f62c00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c5200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f62c00b71c0 0x7f62c00cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62bfe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f62c0156b90 0x7f62c00c70f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56108/0 (socket says 172.21.15.123:56108) 2024-04-25T12:36:52.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62bfe00640 1 -- 172.21.15.123:0/111702181 learned_addr learned my addr 172.21.15.123:0/111702181 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:52.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c5200640 1 -- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b7680 msgr2=0x7f62c00b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:52.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.393+0000 7f62c5200640 1 --2- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b7680 0x7f62c00b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.394+0000 7f62c5200640 1 -- 172.21.15.123:0/111702181 >> v1:172.21.15.123:6789/0 conn(0x7f62c0156b90 legacy=0x7f62c00c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.394+0000 7f62c5200640 1 -- 172.21.15.123:0/111702181 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c00b4720 con 0x7f62c00b71c0 2024-04-25T12:36:52.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.394+0000 7f62bf400640 1 -- 172.21.15.123:0/111702181 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 558936089 0 0) 0x7f62c00c7c70 con 0x7f62c0156b90 2024-04-25T12:36:52.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.394+0000 7f62c5c00640 1 --2- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b7680 0x7f62c00b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:52.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.395+0000 7f62c5200640 1 --2- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b71c0 0x7f62c00cde00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f62c00c7e40 tx=0x7f62b402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d25b6d9b979cf29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.396+0000 7f62bf400640 1 -- 172.21.15.123:0/111702181 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4037070 con 0x7f62c00b71c0 2024-04-25T12:36:52.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.396+0000 7f62bf400640 1 -- 172.21.15.123:0/111702181 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b402adb0 con 0x7f62c00b71c0 2024-04-25T12:36:52.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.396+0000 7f62bf400640 1 -- 172.21.15.123:0/111702181 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4032660 con 0x7f62c00b71c0 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.396+0000 7f62c7c00640 1 -- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b71c0 msgr2=0x7f62c00cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.396+0000 7f62c7c00640 1 --2- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b71c0 0x7f62c00cde00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f62c00c7e40 tx=0x7f62b402a060 comp rx=0 tx=0).stop 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.397+0000 7f62c7c00640 1 -- 172.21.15.123:0/111702181 shutdown_connections 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.397+0000 7f62c7c00640 1 --2- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b7680 0x7f62c00b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.397+0000 7f62c7c00640 1 --2- 172.21.15.123:0/111702181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b71c0 0x7f62c00cde00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.397+0000 7f62c7c00640 1 -- 172.21.15.123:0/111702181 >> 172.21.15.123:0/111702181 conn(0x7f62c00d1970 msgr2=0x7f62c00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.397+0000 7f62c7c00640 1 -- 172.21.15.123:0/111702181 shutdown_connections 2024-04-25T12:36:52.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.397+0000 7f62c7c00640 1 -- 172.21.15.123:0/111702181 wait complete. 2024-04-25T12:36:52.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.399+0000 7f62c7c00640 1 Processor -- start 2024-04-25T12:36:52.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.399+0000 7f62c7c00640 1 -- start start 2024-04-25T12:36:52.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c7c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b71c0 0x7f62c01e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c7c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b7680 0x7f62c01f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c7c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c0156b90 0x7f62c00cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c7c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62c00c7800 con 0x7f62c0156b90 2024-04-25T12:36:52.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c7c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62c00c7f10 con 0x7f62c00b7680 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c7c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00c7980 con 0x7f62c00b71c0 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c0156b90 0x7f62c00cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b7680 0x7f62c01f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c5200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b71c0 0x7f62c01e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b7680 0x7f62c01f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:47098/0 (socket says 172.21.15.123:47098) 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c0156b90 0x7f62c00cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48754/0 (socket says 172.21.15.123:48754) 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62bfe00640 1 -- 172.21.15.123:0/4146277651 learned_addr learned my addr 172.21.15.123:0/4146277651 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c5200640 1 -- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b7680 msgr2=0x7f62c01f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.401+0000 7f62c5200640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b7680 0x7f62c01f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62c5200640 1 -- 172.21.15.123:0/4146277651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c0156b90 msgr2=0x7f62c00cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62c5200640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c0156b90 0x7f62c00cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62c5200640 1 -- 172.21.15.123:0/4146277651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c01e8410 con 0x7f62c00b71c0 2024-04-25T12:36:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62c5200640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b71c0 0x7f62c01e6a90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f62c008f980 tx=0x7f62b40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62bcc00640 1 -- 172.21.15.123:0/4146277651 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4037040 con 0x7f62c00b71c0 2024-04-25T12:36:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62bcc00640 1 -- 172.21.15.123:0/4146277651 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b4004030 con 0x7f62c00b71c0 2024-04-25T12:36:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62bcc00640 1 -- 172.21.15.123:0/4146277651 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4004420 con 0x7f62c00b71c0 2024-04-25T12:36:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.402+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c01e86f0 con 0x7f62c00b71c0 2024-04-25T12:36:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.403+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c01e7060 con 0x7f62c00b71c0 2024-04-25T12:36:52.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.404+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c0076700 con 0x7f62c00b71c0 2024-04-25T12:36:52.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.406+0000 7f62bcc00640 1 -- 172.21.15.123:0/4146277651 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f62b4003610 con 0x7f62c00b71c0 2024-04-25T12:36:52.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.407+0000 7f62bcc00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f62a00744b0 0x7f62a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.407+0000 7f62bcc00640 1 -- 172.21.15.123:0/4146277651 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(69..69 src has 1..69) ==== 6685+0+0 (secure 0 0 0) 0x7f62b40b60f0 con 0x7f62c00b71c0 2024-04-25T12:36:52.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.407+0000 7f62bfe00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f62a00744b0 0x7f62a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.408+0000 7f62bfe00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f62a00744b0 0x7f62a00769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f62ac000f60 tx=0x7f62ac007ea0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.412+0000 7f62bcc00640 1 -- 172.21.15.123:0/4146277651 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62b407fef0 con 0x7f62c00b71c0 2024-04-25T12:36:52.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.432+0000 7f91e3c00640 1 Processor -- start 2024-04-25T12:36:52.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.432+0000 7f91e3c00640 1 -- start start 2024-04-25T12:36:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.435+0000 7f91e3c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.435+0000 7f91e3c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f91dc0cf730 0x7f91dc0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.435+0000 7f91e3c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91dc0cd000 con 0x7f91dc0ca7e0 2024-04-25T12:36:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.435+0000 7f91e3c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f91dc15d920 con 0x7f91dc0cf730 2024-04-25T12:36:52.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.435+0000 7f91e3c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0ccb70 con 0x7f91dc156e40 2024-04-25T12:36:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91dbe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f91dc0ca7e0 0x7f91dc0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56116/0 (socket says 172.21.15.123:56116) 2024-04-25T12:36:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91e1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91dbe00640 1 -- 172.21.15.123:0/992437014 learned_addr learned my addr 172.21.15.123:0/992437014 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91e1c00640 1 --2- 172.21.15.123:0/992437014 >> v2:172.21.15.123:3300/0 conn(0x7f91dc0cf730 0x7f91dc0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91e1200640 1 -- 172.21.15.123:0/992437014 >> v2:172.21.15.123:3300/0 conn(0x7f91dc0cf730 msgr2=0x7f91dc0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:52.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91e1200640 1 --2- 172.21.15.123:0/992437014 >> v2:172.21.15.123:3300/0 conn(0x7f91dc0cf730 0x7f91dc0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91e1200640 1 -- 172.21.15.123:0/992437014 >> v1:172.21.15.123:6789/0 conn(0x7f91dc0ca7e0 legacy=0x7f91dc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.436+0000 7f91e1200640 1 -- 172.21.15.123:0/992437014 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc0cbf60 con 0x7f91dc156e40 2024-04-25T12:36:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.437+0000 7f91e1200640 1 --2- 172.21.15.123:0/992437014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc15d3e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91d00097a0 tx=0x7f91d002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e721279546da6f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.437+0000 7f91db400640 1 -- 172.21.15.123:0/992437014 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0037070 con 0x7f91dc156e40 2024-04-25T12:36:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.438+0000 7f91db400640 1 -- 172.21.15.123:0/992437014 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d0032440 con 0x7f91dc156e40 2024-04-25T12:36:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.438+0000 7f91db400640 1 -- 172.21.15.123:0/992437014 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0032810 con 0x7f91dc156e40 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.438+0000 7f91e3c00640 1 -- 172.21.15.123:0/992437014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 msgr2=0x7f91dc15d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.438+0000 7f91e3c00640 1 --2- 172.21.15.123:0/992437014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc15d3e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91d00097a0 tx=0x7f91d002a250 comp rx=0 tx=0).stop 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.439+0000 7f91e3c00640 1 -- 172.21.15.123:0/992437014 shutdown_connections 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.439+0000 7f91e3c00640 1 --2- 172.21.15.123:0/992437014 >> v2:172.21.15.123:3300/0 conn(0x7f91dc0cf730 0x7f91dc0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.439+0000 7f91e3c00640 1 --2- 172.21.15.123:0/992437014 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc15d3e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.439+0000 7f91e3c00640 1 -- 172.21.15.123:0/992437014 >> 172.21.15.123:0/992437014 conn(0x7f91dc0715e0 msgr2=0x7f91dc068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.439+0000 7f91e3c00640 1 -- 172.21.15.123:0/992437014 shutdown_connections 2024-04-25T12:36:52.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.439+0000 7f91e3c00640 1 -- 172.21.15.123:0/992437014 wait complete. 2024-04-25T12:36:52.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.441+0000 7f91e3c00640 1 Processor -- start 2024-04-25T12:36:52.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.441+0000 7f91e3c00640 1 -- start start 2024-04-25T12:36:52.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.442+0000 7f91e3c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91dc0ca7e0 0x7f91dc151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91e3c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91dc0cf730 0x7f91dc15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91e3c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91e3c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0cc800 con 0x7f91dc0cf730 2024-04-25T12:36:52.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91e3c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0cc520 con 0x7f91dc0ca7e0 2024-04-25T12:36:52.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91e3c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0cb3f0 con 0x7f91dc156e40 2024-04-25T12:36:52.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91dc0cf730 0x7f91dc15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91dc0cf730 0x7f91dc15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48766/0 (socket says 172.21.15.123:48766) 2024-04-25T12:36:52.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91dbe00640 1 -- 172.21.15.123:0/355824047 learned_addr learned my addr 172.21.15.123:0/355824047 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:52.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91e1c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.443+0000 7f91e1200640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91dc0ca7e0 0x7f91dc151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91dbe00640 1 -- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 msgr2=0x7f91dc0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91dbe00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91dbe00640 1 -- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91dc0ca7e0 msgr2=0x7f91dc151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:52.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91dbe00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91dc0ca7e0 0x7f91dc151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:52.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91dbe00640 1 -- 172.21.15.123:0/355824047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc151840 con 0x7f91dc0cf730 2024-04-25T12:36:52.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91e1c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:52.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91e1200640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91dc0ca7e0 0x7f91dc151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:52.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91dbe00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91dc0cf730 0x7f91dc15cdf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f91dc074650 tx=0x7f91c800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91d8c00640 1 -- 172.21.15.123:0/355824047 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8017070 con 0x7f91dc0cf730 2024-04-25T12:36:52.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91d8c00640 1 -- 172.21.15.123:0/355824047 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c8005ce0 con 0x7f91dc0cf730 2024-04-25T12:36:52.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc1f3000 con 0x7f91dc0cf730 2024-04-25T12:36:52.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.444+0000 7f91d8c00640 1 -- 172.21.15.123:0/355824047 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80072d0 con 0x7f91dc0cf730 2024-04-25T12:36:52.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.445+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc1f35c0 con 0x7f91dc0cf730 2024-04-25T12:36:52.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.446+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc076700 con 0x7f91dc0cf730 2024-04-25T12:36:52.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.448+0000 7f91d8c00640 1 -- 172.21.15.123:0/355824047 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f91c801a070 con 0x7f91dc0cf730 2024-04-25T12:36:52.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.450+0000 7f91d8c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91bc074530 0x7f91bc076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:52.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.450+0000 7f91d8c00640 1 -- 172.21.15.123:0/355824047 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6685+0+0 (secure 0 0 0) 0x7f91c8094620 con 0x7f91dc0cf730 2024-04-25T12:36:52.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.450+0000 7f91e1200640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91bc074530 0x7f91bc076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:52.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.451+0000 7f91e1200640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91bc074530 0x7f91bc076a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f91d0034000 tx=0x7f91d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:52.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.455+0000 7f91d8c00640 1 -- 172.21.15.123:0/355824047 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91c805e400 con 0x7f91dc0cf730 2024-04-25T12:36:52.462 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T12:36:52.463+0000 7fbef1a00640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33448) UID: 0 2024-04-25T12:36:52.467 DEBUG:teuthology.parallel:result is None 2024-04-25T12:36:52.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T12:36:52.496 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:36:52.496 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-25T12:36:52.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-25T12:36:52.496 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:4 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'1, mlcod=0'0)", 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 153.545808913, 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010941543999999999, 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949974+0000", 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949977+0000", 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.463e-06 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.949995+0000", 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8749e-05 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950024+0000", 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8810999999999999e-05 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950046+0000", 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1631999999999999e-05 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950244+0000", 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019852300000000001 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950309+0000", 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4232000000000004e-05 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960803+0000", 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010494392999999999 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960916+0000", 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112741 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:5 2.1f e24/18 2:ff5b34d6:::1.00000000:head v 24'2, mlcod=0'0)", 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 153.54512473099999, 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010324425999999999, 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.501 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950658+0000", 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950661+0000", 2024-04-25T12:36:52.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950681+0000", 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.997e-05 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950713+0000", 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1906000000000003e-05 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.950739+0000", 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6126e-05 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951195+0000", 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045639299999999998 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.951253+0000", 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7571999999999997e-05 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960922+0000", 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096689129999999995 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960983+0000", 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1132999999999999e-05 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:26 2.1f e24/18 2:fe07bba1:::100.00000000.inode:head v 24'3, mlcod=0'0)", 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 153.54367418999999, 2024-04-25T12:36:52.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010302477000000001, 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952109+0000", 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952111+0000", 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.843e-06 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952131+0000", 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0417000000000001e-05 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952180+0000", 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8862000000000001e-05 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952204+0000", 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3516e-05 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952980+0000", 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00077692899999999999 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.953062+0000", 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1181999999999996e-05 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962340+0000", 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092781219999999998 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962411+0000", 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1606000000000005e-05 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1f epoch 24/18 rep_tid 8 entries 24'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 153.543413375, 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010105995, 2024-04-25T12:36:52.509 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952370+0000", 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952371+0000", 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3459999999999999e-06 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952395+0000", 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3773000000000001e-05 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952499+0000", 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104314 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.952552+0000", 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2506000000000002e-05 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.954700+0000", 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021487049999999999 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.962476+0000", 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077753509999999998 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 153.539562988, 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096684289999999992, 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956220+0000", 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956238+0000", 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.827e-05 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956269+0000", 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1038999999999999e-05 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956322+0000", 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2318999999999998e-05 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956352+0000", 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0346e-05 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958314+0000", 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019619389999999998 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.958683+0000", 2024-04-25T12:36:52.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000368854 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960009+0000", 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001326198 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965616+0000", 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056068089999999999 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965790+0000", 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017410400000000001 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965842+0000", 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1728999999999999e-05 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965889+0000", 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6822e-05 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:52.517 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 153.53934061999999, 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097817640000000001, 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956442+0000", 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956467+0000", 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4856000000000001e-05 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956480+0000", 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3037000000000001e-05 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956509+0000", 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8668999999999999e-05 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956533+0000", 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3903000000000001e-05 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961051+0000", 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045184650000000002 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961328+0000", 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027618899999999998 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.961709+0000", 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000381493 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.965628+0000", 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039191340000000003 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966052+0000", 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042342399999999999 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966159+0000", 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010765200000000001 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966224+0000", 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4942000000000003e-05 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 153.53898683099999, 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094695950000000008, 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956796+0000", 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956798+0000", 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.463e-06 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956805+0000", 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1770000000000001e-06 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956827+0000", 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1826000000000001e-05 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.956846+0000", 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8862999999999999e-05 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960103+0000", 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003257294 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.960888+0000", 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00078522499999999996 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:18.966266+0000", 2024-04-25T12:36:52.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053777470000000004 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:5 2.7 2:e455cf45:::60b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.378367883, 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013684428, 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117415+0000", 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117418+0000", 2024-04-25T12:36:52.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.163e-06 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117439+0000", 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0313e-05 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117518+0000", 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8862000000000005e-05 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117548+0000", 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0139000000000001e-05 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117995+0000", 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00044759699999999999 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118345+0000", 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00034927500000000002 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:52.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118700+0000", 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035548200000000001 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124853+0000", 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061533250000000003 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130994+0000", 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061404950000000002 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131057+0000", 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3544e-05 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131100+0000", 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2233e-05 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:52.532 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.37816039099999, 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014568739000000001, 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117623+0000", 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117624+0000", 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6959999999999999e-06 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117689+0000", 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4583000000000002e-05 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117715+0000", 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089e-05 2024-04-25T12:36:52.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117736+0000", 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0599999999999999e-05 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120547+0000", 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028112430000000002 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121439+0000", 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089183399999999999 2024-04-25T12:36:52.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121837+0000", 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039830099999999999 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.125934+0000", 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040972480000000004 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132084+0000", 2024-04-25T12:36:52.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061493989999999998 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132148+0000", 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4407999999999997e-05 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.132191+0000", 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3337999999999997e-05 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:7 2.b 2:d70a01c6:::60c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.378015901, 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012611218, 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117767+0000", 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117769+0000", 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5e-06 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117799+0000", 2024-04-25T12:36:52.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429e-05 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117823+0000", 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3543000000000001e-05 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117842+0000", 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9009999999999999e-05 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118299+0000", 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045691800000000001 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118443+0000", 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000144814 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118609+0000", 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165141 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124848+0000", 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062392970000000004 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130055+0000", 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052072860000000002 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130149+0000", 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3517000000000006e-05 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130378+0000", 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022976299999999999 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:8 2.b 2:d70a01c6:::60c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377884107, 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012700444, 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.543 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117899+0000", 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117900+0000", 2024-04-25T12:36:52.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.313e-06 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117911+0000", 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0823e-05 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117932+0000", 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0483e-05 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117949+0000", 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7873e-05 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118926+0000", 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097694800000000005 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119008+0000", 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1945000000000002e-05 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119158+0000", 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014997599999999999 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.124865+0000", 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005706554 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130487+0000", 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005621839 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130555+0000", 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8392000000000005e-05 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130599+0000", 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4298000000000002e-05 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:52.548 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377809812, 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092311500000000005, 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117973+0000", 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117974+0000", 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.133e-06 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.117984+0000", 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3959999999999995e-06 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118003+0000", 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9287e-05 2024-04-25T12:36:52.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118019+0000", 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6033e-05 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119813+0000", 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017934069999999999 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119940+0000", 2024-04-25T12:36:52.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127704 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120054+0000", 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113378 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.123715+0000", 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036615139999999998 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126910+0000", 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031949299999999999 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127077+0000", 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000166682 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127204+0000", 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012768600000000001 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377710071, 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010303596999999999, 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118073+0000", 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118074+0000", 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.136e-06 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118083+0000", 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5499999999999995e-06 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118103+0000", 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0557e-05 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118120+0000", 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7161999999999999e-05 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120350+0000", 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022293479999999999 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120427+0000", 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6909999999999994e-05 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120566+0000", 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013909 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.126873+0000", 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063073799999999996 2024-04-25T12:36:52.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.127225+0000", 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035195799999999998 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128289+0000", 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001063423 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128377+0000", 2024-04-25T12:36:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8083000000000004e-05 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377640609, 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012482241999999999, 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118142+0000", 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118144+0000", 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0130000000000001e-06 2024-04-25T12:36:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118180+0000", 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6715999999999999e-05 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118200+0000", 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9732999999999999e-05 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118219+0000", 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9136000000000001e-05 2024-04-25T12:36:52.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119124+0000", 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090532000000000004 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119422+0000", 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00029749000000000003 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.119872+0000", 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045020999999999999 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128141+0000", 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082689019999999998 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130484+0000", 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023428390000000002 2024-04-25T12:36:52.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130558+0000", 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3894999999999996e-05 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.130625+0000", 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6988000000000003e-05 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.37754049500001, 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011309397000000001, 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118243+0000", 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118244+0000", 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-06 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118252+0000", 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-04-25T12:36:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118271+0000", 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118287+0000", 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6566e-05 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121217+0000", 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029298219999999999 2024-04-25T12:36:52.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121319+0000", 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102117 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121459+0000", 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013999100000000001 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128114+0000", 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066549979999999996 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129390+0000", 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012760029999999999 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129481+0000", 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0282000000000001e-05 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129552+0000", 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1354999999999997e-05 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.569 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377443404, 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013121925, 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118340+0000", 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118341+0000", 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.35e-06 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118370+0000", 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8707000000000001e-05 2024-04-25T12:36:52.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118390+0000", 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0199000000000001e-05 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118407+0000", 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7393000000000001e-05 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122741+0000", 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043331940000000003 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122855+0000", 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011409 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122975+0000", 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120331 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128152+0000", 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051769709999999998 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131283+0000", 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031312179999999998 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131384+0000", 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101284 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.131462+0000", 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377352999, 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010388601000000001, 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.575 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118430+0000", 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118431+0000", 2024-04-25T12:36:52.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.206e-06 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118439+0000", 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2239999999999998e-06 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118477+0000", 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8509e-05 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118498+0000", 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0585999999999999e-05 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120792+0000", 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022948589999999998 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.120893+0000", 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100355 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128819+0000", 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079258620000000005 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377205026, 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010687432, 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118578+0000", 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118579+0000", 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.274e-06 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118587+0000", 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9030000000000004e-06 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118607+0000", 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0169e-05 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118631+0000", 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3280000000000001e-05 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121588+0000", 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029568340000000002 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121867+0000", 2024-04-25T12:36:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027932100000000002 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.129266+0000", 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007398651 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.1.9:33 2.0 2:0250a5a6:::201.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 122.377097621, 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009990868, 2024-04-25T12:36:52.583 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118685+0000", 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118687+0000", 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.156e-06 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118740+0000", 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3439000000000001e-05 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118764+0000", 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4332999999999999e-05 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.118783+0000", 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.121982+0000", 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031992610000000001 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.122104+0000", 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122182 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:50.128676+0000", 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065720409999999998 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 117.342846323, 2024-04-25T12:36:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092790649999999995, 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-25T12:36:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152937+0000", 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3300000000000003e-07 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152940+0000", 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8100000000000002e-06 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152947+0000", 2024-04-25T12:36:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0700000000000001e-06 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.152953+0000", 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6439999999999997e-06 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.153684+0000", 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000731531 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154017+0000", 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000333226 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.154591+0000", 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057304000000000003 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.160876+0000", 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062852309999999996 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162128+0000", 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00125176 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162179+0000", 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1329999999999998e-05 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-25T12:34:55.162216+0000", 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6990000000000003e-05 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T12:36:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T12:36:52.593 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:36:52.595 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T12:36:52.596 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T12:36:52.596 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-25T12:36:52.605 INFO:tasks.thrashosds:joining thrashosds 2024-04-25T12:36:52.606 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-04-25T12:36:52.595+0000 7ff8b88de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34701) UID: 0 2024-04-25T12:36:52.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.628+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f62c0153dd0 con 0x7f62c00b71c0 2024-04-25T12:36:52.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:52.669+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f91dc0cb990 con 0x7f91dc0cf730 2024-04-25T12:36:52.730 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:52.731 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:52.731 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:52.731 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:52.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T12:36:52.854 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:52.854 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:52.854 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:52.863 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T12:36:52.977 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02801836899999, 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015130565, 2024-04-25T12:36:52.978 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949275+0000", 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949277+0000", 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3769999999999999e-06 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949292+0000", 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5305e-05 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949310+0000", 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8181000000000001e-05 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949325+0000", 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.485e-05 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:52.981 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949450+0000", 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000124855 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949617+0000", 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000166976 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949757+0000", 2024-04-25T12:36:52.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013947800000000001 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952249+0000", 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024926129999999999 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964306+0000", 2024-04-25T12:36:52.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012056572 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964363+0000", 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964405+0000", 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2252999999999997e-05 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:52.984 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02793066000001, 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015187765000000001, 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:52.985 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949362+0000", 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949363+0000", 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5499999999999998e-07 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949369+0000", 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5910000000000003e-06 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949381+0000", 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2668e-05 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.987 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949387+0000", 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2269999999999998e-06 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950153+0000", 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000765692 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950248+0000", 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4796999999999994e-05 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.988 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950369+0000", 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012139899999999999 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954187+0000", 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003817165 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964466+0000", 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010279953 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.989 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964504+0000", 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7732999999999999e-05 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964550+0000", 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5884999999999999e-05 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.990 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02780473199999, 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0087587299999999993, 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:52.991 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949488+0000", 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949489+0000", 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7599999999999997e-07 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949498+0000", 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9649999999999997e-06 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.992 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949504+0000", 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1179999999999997e-06 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949533+0000", 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.8781000000000001e-05 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950951+0000", 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001418306 2024-04-25T12:36:52.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951103+0000", 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015218999999999999 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951238+0000", 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013452200000000001 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954075+0000", 2024-04-25T12:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028374580000000002 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958137+0000", 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040617270000000002 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958197+0000", 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9595000000000001e-05 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.995 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958247+0000", 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0392000000000001e-05 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02774307300001, 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088010950000000001, 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:52.996 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949550+0000", 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949551+0000", 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3399999999999997e-07 2024-04-25T12:36:52.997 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949558+0000", 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.52e-06 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949569+0000", 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.039e-05 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949590+0000", 2024-04-25T12:36:52.998 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0959e-05 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951587+0000", 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019974429999999998 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951671+0000", 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3944999999999997e-05 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951826+0000", 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000154437 2024-04-25T12:36:52.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954656+0000", 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028309369999999999 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958262+0000", 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036052079999999999 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958311+0000", 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8962999999999999e-05 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.000 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958351+0000", 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0358999999999998e-05 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02768631000001, 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088533829999999994, 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.001 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949607+0000", 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5700000000000002e-07 2024-04-25T12:36:53.002 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949613+0000", 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4049999999999999e-06 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949638+0000", 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5602999999999999e-05 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949654+0000", 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5231e-05 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.003 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952102+0000", 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024485399999999999 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952251+0000", 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014848899999999999 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952393+0000", 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000142566 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.004 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954713+0000", 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0023197840000000001 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958364+0000", 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036512060000000002 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958407+0000", 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2469999999999998e-05 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.005 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958460+0000", 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3331999999999997e-05 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.006 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02762283999999, 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0088886599999999996, 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949670+0000", 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.007 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949671+0000", 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7600000000000006e-07 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949677+0000", 2024-04-25T12:36:53.008 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2430000000000002e-06 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949687+0000", 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7690000000000008e-06 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949710+0000", 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2541e-05 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952707+0000", 2024-04-25T12:36:53.009 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029969789999999999 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952791+0000", 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4198999999999998e-05 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:53.010 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952956+0000", 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016465699999999999 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955838+0000", 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028822940000000001 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958471+0000", 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002632873 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.011 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958519+0000", 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8699000000000002e-05 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958559+0000", 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.943e-05 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.012 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02757089100001, 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094026309999999998, 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.013 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949722+0000", 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949723+0000", 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2699999999999999e-07 2024-04-25T12:36:53.014 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949745+0000", 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2770000000000001e-05 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949754+0000", 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9209999999999998e-06 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949760+0000", 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8259999999999998e-06 2024-04-25T12:36:53.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953256+0000", 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034953860000000001 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953385+0000", 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000129281 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953505+0000", 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012042800000000001 2024-04-25T12:36:53.016 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958569+0000", 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005063517 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959006+0000", 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00043726700000000003 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959061+0000", 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5152000000000003e-05 2024-04-25T12:36:53.017 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959125+0000", 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3455999999999997e-05 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02751894100001, 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094817040000000005, 2024-04-25T12:36:53.018 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.019 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949774+0000", 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4499999999999997e-07 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949778+0000", 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4800000000000001e-06 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949783+0000", 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0409999999999997e-06 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.020 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949787+0000", 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.0709999999999996e-06 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954117+0000", 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0043299590000000004 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954208+0000", 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0828999999999994e-05 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.021 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 7", 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954368+0000", 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015968800000000001 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959140+0000", 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0047725980000000003 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959177+0000", 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7162000000000001e-05 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.022 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959214+0000", 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6236999999999999e-05 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.959256+0000", 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2194000000000003e-05 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.023 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.02740183200001, 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014779841, 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949891+0000", 2024-04-25T12:36:53.024 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949892+0000", 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949898+0000", 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.144e-06 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949907+0000", 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7739999999999993e-06 2024-04-25T12:36:53.025 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949915+0000", 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2789999999999998e-06 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950682+0000", 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00076653599999999995 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.026 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950807+0000", 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012557700000000001 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.950936+0000", 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00012891199999999999 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954250+0000", 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003313543 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.027 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964608+0000", 2024-04-25T12:36:53.028 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010358488000000001 2024-04-25T12:36:53.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.029 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.029 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964641+0000", 2024-04-25T12:36:53.029 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2966999999999998e-05 2024-04-25T12:36:53.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.029 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964671+0000", 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0001e-05 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.027363385, 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.014840872999999999, 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.030 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949930+0000", 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999998e-07 2024-04-25T12:36:53.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949934+0000", 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5870000000000001e-06 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949941+0000", 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0550000000000002e-06 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.949946+0000", 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8389999999999997e-06 2024-04-25T12:36:53.032 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951369+0000", 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001422548 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951537+0000", 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000168033 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964770+0000", 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013233922 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 122.862228332, 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.013698139999999999, 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115065+0000", 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115066+0000", 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7900000000000007e-07 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115073+0000", 2024-04-25T12:36:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.9519999999999997e-06 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115085+0000", 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2084e-05 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115095+0000", 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8400000000000007e-06 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115171+0000", 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6070999999999997e-05 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115336+0000", 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000165539 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115485+0000", 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000149314 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119693+0000", 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042077319999999996 2024-04-25T12:36:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128645+0000", 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0089517739999999991 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128702+0000", 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6830000000000003e-05 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.128763+0000", 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1024999999999999e-05 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 122.862170281, 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015849164999999998, 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115123+0000", 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115124+0000", 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2900000000000002e-07 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115130+0000", 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115141+0000", 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0689999999999999e-05 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115151+0000", 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8879999999999992e-06 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115922+0000", 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077143199999999996 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116006+0000", 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3759999999999998e-05 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116183+0000", 2024-04-25T12:36:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017692200000000001 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120610+0000", 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0044272640000000002 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130881+0000", 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010271115000000001 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130935+0000", 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4156000000000001e-05 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.130972+0000", 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6718999999999999e-05 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 122.86211058799999, 2024-04-25T12:36:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.015951877999999999, 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115182+0000", 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115183+0000", 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7599999999999996e-07 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115189+0000", 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0979999999999999e-06 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115202+0000", 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3004e-05 2024-04-25T12:36:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115216+0000", 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3993e-05 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116488+0000", 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001271592 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116620+0000", 2024-04-25T12:36:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013221500000000001 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116777+0000", 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015700999999999999 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120690+0000", 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0039123329999999996 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131032+0000", 2024-04-25T12:36:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010342670999999999 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131089+0000", 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6675000000000002e-05 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.131134+0000", 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5411000000000003e-05 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.825190506, 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0093428629999999999, 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.050 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152102+0000", 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152105+0000", 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1890000000000001e-06 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152117+0000", 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2021000000000001e-05 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.051 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152137+0000", 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9919999999999999e-05 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152151+0000", 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4099e-05 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152238+0000", 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6866999999999993e-05 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.052 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152422+0000", 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018466899999999999 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00016619099999999999 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154557+0000", 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0019681719999999998 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161340+0000", 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0067835400000000002 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161398+0000", 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7367999999999997e-05 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7827000000000001e-05 2024-04-25T12:36:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.82510154000001, 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0094426330000000006, 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.055 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152191+0000", 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152192+0000", 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5300000000000002e-07 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.056 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152199+0000", 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5289999999999997e-06 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152209+0000", 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8279999999999998e-06 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152218+0000", 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6679999999999996e-06 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.057 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153233+0000", 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0010150210000000001 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153312+0000", 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8813000000000006e-05 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153449+0000", 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013696800000000001 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.058 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154568+0000", 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011191879999999999 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161516+0000", 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069472580000000004 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161577+0000", 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1302000000000004e-05 2024-04-25T12:36:53.059 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161634+0000", 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7105000000000003e-05 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.825057699, 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0092100570000000007, 2024-04-25T12:36:53.060 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152235+0000", 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152236+0000", 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4000000000000001e-07 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152242+0000", 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8440000000000003e-06 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152249+0000", 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1099999999999997e-06 2024-04-25T12:36:53.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152255+0000", 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4860000000000002e-06 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152326+0000", 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0885999999999999e-05 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152427+0000", 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0001009 2024-04-25T12:36:53.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152588+0000", 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000160347 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154343+0000", 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001755777 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161359+0000", 2024-04-25T12:36:53.064 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070158349999999998 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161403+0000", 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3909999999999998e-05 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161445+0000", 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2321999999999999e-05 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.82354379500001, 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0078805769999999997, 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153749+0000", 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153750+0000", 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6700000000000002e-07 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153761+0000", 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0892e-05 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.067 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153774+0000", 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2612999999999999e-05 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153785+0000", 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1029000000000001e-05 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153873+0000", 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8628000000000006e-05 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.068 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153974+0000", 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000100454 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154110+0000", 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013658799999999999 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156218+0000", 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002107439 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161514+0000", 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0052958520000000002 2024-04-25T12:36:53.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161582+0000", 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8047999999999996e-05 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161630+0000", 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8066999999999999e-05 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.070 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.82262457900001, 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0071498789999999996, 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154668+0000", 2024-04-25T12:36:53.071 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154669+0000", 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9500000000000002e-07 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154674+0000", 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6630000000000001e-06 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154683+0000", 2024-04-25T12:36:53.072 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7210000000000006e-06 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154692+0000", 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.1769999999999998e-06 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154748+0000", 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6008000000000003e-05 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154884+0000", 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135969 2024-04-25T12:36:53.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 3", 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155020+0000", 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136019 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156205+0000", 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0011851780000000001 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161705+0000", 2024-04-25T12:36:53.074 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00549995 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161766+0000", 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1051999999999998e-05 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161818+0000", 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.2546999999999997e-05 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.822169226, 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079792079999999998, 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.076 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155124+0000", 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155127+0000", 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0180000000000002e-06 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.077 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155135+0000", 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4259999999999997e-06 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155143+0000", 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9209999999999992e-06 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157157+0000", 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0020139310000000001 2024-04-25T12:36:53.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157292+0000", 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135795 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.163103+0000", 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005810567 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.079 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 117.822140248, 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.007132373, 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.080 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155153+0000", 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9999999999999998e-07 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155165+0000", 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1747e-05 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.081 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155173+0000", 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0090000000000007e-06 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155182+0000", 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1710000000000008e-06 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155714+0000", 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00053141500000000003 2024-04-25T12:36:53.082 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155857+0000", 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014304300000000001 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162285+0000", 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0064284880000000004 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.083 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:53.085 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T12:36:53.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.112+0000 7f91d8c00640 1 -- 172.21.15.123:0/355824047 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v70) ==== 113+0+0 (secure 0 0 0) 0x7f91c8062700 con 0x7f91dc0cf730 2024-04-25T12:36:53.111 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.1 to 1 (10000) 2024-04-25T12:36:53.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.116+0000 7f62bcc00640 1 -- 172.21.15.123:0/4146277651 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v70) ==== 97+0+0 (secure 0 0 0) 0x7f62b40841f0 con 0x7f62c00b71c0 2024-04-25T12:36:53.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.116+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91bc074530 msgr2=0x7f91bc076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.116+0000 7f91e3c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91bc074530 0x7f91bc076a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f91d0034000 tx=0x7f91d002e000 comp rx=0 tx=0).stop 2024-04-25T12:36:53.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.116+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91dc0cf730 msgr2=0x7f91dc15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.116+0000 7f91e3c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91dc0cf730 0x7f91dc15cdf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f91dc074650 tx=0x7f91c800c820 comp rx=0 tx=0).stop 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is set 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.116+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 shutdown_connections 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.116+0000 7f91e3c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91bc074530 0x7f91bc076a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.117+0000 7f91e3c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91dc156e40 0x7f91dc0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.117+0000 7f91e3c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91dc0cf730 0x7f91dc15cdf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.117+0000 7f91e3c00640 1 --2- 172.21.15.123:0/355824047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91dc0ca7e0 0x7f91dc151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.117+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 >> 172.21.15.123:0/355824047 conn(0x7f91dc0715e0 msgr2=0x7f91dc15a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.117+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 shutdown_connections 2024-04-25T12:36:53.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.117+0000 7f91e3c00640 1 -- 172.21.15.123:0/355824047 wait complete. 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.121+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f62a00744b0 msgr2=0x7f62a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.121+0000 7f62c7c00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f62a00744b0 0x7f62a00769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f62ac000f60 tx=0x7f62ac007ea0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.121+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b71c0 msgr2=0x7f62c01e6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.121+0000 7f62c7c00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b71c0 0x7f62c01e6a90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f62c008f980 tx=0x7f62b40046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.121+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 shutdown_connections 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.122+0000 7f62c7c00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f62a00744b0 0x7f62a00769c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.122+0000 7f62c7c00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c0156b90 0x7f62c00cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.122+0000 7f62c7c00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c00b7680 0x7f62c01f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.122+0000 7f62c7c00640 1 --2- 172.21.15.123:0/4146277651 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62c00b71c0 0x7f62c01e6a90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.122+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 >> 172.21.15.123:0/4146277651 conn(0x7f62c00d1970 msgr2=0x7f62c00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:53.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.122+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 shutdown_connections 2024-04-25T12:36:53.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.122+0000 7f62c7c00640 1 -- 172.21.15.123:0/4146277651 wait complete. 2024-04-25T12:36:53.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-25T12:36:53.204 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:53.205 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:53.205 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:53.205 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:53.214 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T12:36:53.326 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:53.326 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:53.326 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:53.326 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:53.327 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:53.335 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T12:36:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.446+0000 7ff7fa400640 1 Processor -- start 2024-04-25T12:36:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.446+0000 7ff7fa400640 1 -- start start 2024-04-25T12:36:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7fa400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7f41565f0 0x7ff7f406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f406ccf0 0x7ff7f40c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7fa400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f40c81d0 con 0x7ff7f40c7dc0 2024-04-25T12:36:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7fa400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f4156a00 con 0x7ff7f41565f0 2024-04-25T12:36:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7fa400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40c8450 con 0x7ff7f406ccf0 2024-04-25T12:36:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7f9000640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff7f40c7dc0 0x7ff7f40c2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:54056/0 (socket says 172.21.15.123:54056) 2024-04-25T12:36:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7f9000640 1 -- 172.21.15.123:0/1003090001 learned_addr learned my addr 172.21.15.123:0/1003090001 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7ebe00640 1 --2- 172.21.15.123:0/1003090001 >> v2:172.21.15.123:3300/0 conn(0x7ff7f41565f0 0x7ff7f406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.448+0000 7ff7f9a00640 1 --2- 172.21.15.123:0/1003090001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f406ccf0 0x7ff7f40c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.449+0000 7ff7f3e00640 1 -- 172.21.15.123:0/1003090001 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632615468 0 0) 0x7ff7dc001110 con 0x7ff7f40c7dc0 2024-04-25T12:36:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.449+0000 7ff7f3e00640 1 -- 172.21.15.123:0/1003090001 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d4003520 con 0x7ff7f40c7dc0 2024-04-25T12:36:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.449+0000 7ff7ebe00640 1 -- 172.21.15.123:0/1003090001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f406ccf0 msgr2=0x7ff7f40c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.449+0000 7ff7ebe00640 1 --2- 172.21.15.123:0/1003090001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f406ccf0 0x7ff7f40c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.449+0000 7ff7ebe00640 1 -- 172.21.15.123:0/1003090001 >> v1:172.21.15.153:6789/0 conn(0x7ff7f40c7dc0 legacy=0x7ff7f40c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.449+0000 7ff7ebe00640 1 -- 172.21.15.123:0/1003090001 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f4159310 con 0x7ff7f41565f0 2024-04-25T12:36:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.449+0000 7ff7f9a00640 1 --2- 172.21.15.123:0/1003090001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f406ccf0 0x7ff7f40c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:53.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.450+0000 7ff7ebe00640 1 --2- 172.21.15.123:0/1003090001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f41565f0 0x7ff7f406c7b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097a0 tx=0x7ff7e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee0219b5e002106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:53.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.450+0000 7ff7f3e00640 1 -- 172.21.15.123:0/1003090001 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4037070 con 0x7ff7f41565f0 2024-04-25T12:36:53.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.450+0000 7ff7f3e00640 1 -- 172.21.15.123:0/1003090001 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e4032440 con 0x7ff7f41565f0 2024-04-25T12:36:53.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.450+0000 7ff7f3e00640 1 -- 172.21.15.123:0/1003090001 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40327f0 con 0x7ff7f41565f0 2024-04-25T12:36:53.451 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:53.451 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:53.451 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:53.451 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:53.451 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.451 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 24/18 rep_tid 1 entries 24'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.494464792, 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00075768199999999997, 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.452 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954839+0000", 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954840+0000", 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6869999999999999e-06 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954864+0000", 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.3824000000000001e-05 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954902+0000", 2024-04-25T12:36:53.453 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7500999999999999e-05 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954919+0000", 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7618e-05 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955011+0000", 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.2250000000000006e-05 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.454 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955596+0000", 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058480200000000004 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:25 2.1b e25/18 2:dbae76b5:::mds1_inotable:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.33277971, 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036130300000000001, 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.455 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116524+0000", 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116525+0000", 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.455e-06 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.456 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116539+0000", 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4e-05 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116563+0000", 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4111999999999999e-05 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116577+0000", 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4059999999999999e-05 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.457 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116689+0000", 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011209000000000001 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116719+0000", 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.989e-05 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120082+0000", 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0033628130000000001 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.458 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120137+0000", 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4610000000000001e-05 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 4 entries 25'3 (0'0) error 2:db457a8a:::201.00000001:head by mds.1.9:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.331723169, 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030622480000000001, 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.459 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117580+0000", 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117581+0000", 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0279999999999999e-06 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.460 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117596+0000", 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5109999999999999e-05 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117631+0000", 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.4230000000000003e-05 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117655+0000", 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4987e-05 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.461 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117751+0000", 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5311999999999998e-05 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120642+0000", 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028915809999999998 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1b epoch 25/18 rep_tid 5 entries 25'4 (0'0) error 2:d80f8093:::201.00000002:head by mds.1.9:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:53.462 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.331221938, 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0025905469999999999, 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.463 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118081+0000", 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118082+0000", 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4000000000000001e-07 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118091+0000", 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9919999999999996e-06 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118108+0000", 2024-04-25T12:36:53.464 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7207999999999999e-05 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118124+0000", 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5378000000000001e-05 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.118203+0000", 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8929000000000003e-05 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120672+0000", 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024692999999999998 2024-04-25T12:36:53.465 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:34 2.1b e25/18 2:db457a8a:::201.00000001:head v 25'5, mlcod=25'3)", 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.327747604, 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016516759999999999, 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.466 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121556+0000", 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121557+0000", 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.423e-06 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121568+0000", 2024-04-25T12:36:53.467 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0499000000000001e-05 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121585+0000", 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7283e-05 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121597+0000", 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1821000000000001e-05 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121653+0000", 2024-04-25T12:36:53.468 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6533000000000002e-05 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121671+0000", 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7285000000000001e-05 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123158+0000", 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0014869519999999999 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.123207+0000", 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9879999999999997e-05 2024-04-25T12:36:53.469 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:35 2.a 2:54ba6923:::mds1_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.305635287, 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016436459999999999, 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.470 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143668+0000", 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143670+0000", 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4040000000000002e-06 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143685+0000", 2024-04-25T12:36:53.471 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4768e-05 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143703+0000", 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7424000000000001e-05 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143721+0000", 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8685000000000002e-05 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.143824+0000", 2024-04-25T12:36:53.472 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010286800000000001 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144032+0000", 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020812700000000001 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144205+0000", 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017276999999999999 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.144982+0000", 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077719300000000002 2024-04-25T12:36:53.473 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145196+0000", 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000213308 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145254+0000", 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8619000000000002e-05 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145312+0000", 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7479999999999999e-05 2024-04-25T12:36:53.474 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.1.9:35 2.a e25/18 ondisk, result = 0)", 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.30420872000001, 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00022686700000000001, 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:53.475 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145095+0000", 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145096+0000", 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2309999999999999e-06 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145102+0000", 2024-04-25T12:36:53.476 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1639999999999998e-06 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145115+0000", 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2809000000000001e-05 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145124+0000", 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9989999999999997e-06 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145178+0000", 2024-04-25T12:36:53.477 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4287999999999999e-05 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145189+0000", 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.084e-05 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.145321+0000", 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013253599999999999 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.478 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:11 2.c 2:36924187:::618.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.296959454, 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070050249999999998, 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152344+0000", 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152345+0000", 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6270000000000001e-06 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152355+0000", 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4069999999999999e-06 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.480 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152384+0000", 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.9479e-05 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152399+0000", 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4323e-05 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152478+0000", 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9702000000000004e-05 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.481 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152615+0000", 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013665500000000001 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152730+0000", 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00011504499999999999 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156394+0000", 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036643259999999999 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.482 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028370299999999999 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159289+0000", 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7268999999999997e-05 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159349+0000", 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.0161999999999998e-05 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.483 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.29687288, 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0070895239999999998, 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.484 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152430+0000", 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152431+0000", 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6899999999999996e-07 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152440+0000", 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3259999999999996e-06 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.485 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152450+0000", 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0874e-05 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152470+0000", 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9658999999999999e-05 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153265+0000", 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079454099999999995 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.486 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153349+0000", 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4381000000000004e-05 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153485+0000", 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000136365 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156400+0000", 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029144990000000001 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.487 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159418+0000", 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030182299999999998 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159477+0000", 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8653000000000002e-05 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159520+0000", 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.3127000000000002e-05 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.488 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.296809266, 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0068584939999999997, 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.489 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152494+0000", 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152495+0000", 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.16e-07 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152501+0000", 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7819999999999999e-06 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.490 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152510+0000", 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9199999999999993e-06 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152518+0000", 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9090000000000006e-06 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152651+0000", 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000132349 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.491 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152795+0000", 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014392800000000001 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152930+0000", 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013561399999999999 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156365+0000", 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0034348830000000001 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.492 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159266+0000", 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0029006959999999999 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159296+0000", 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0026e-05 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159353+0000", 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6570999999999998e-05 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.493 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.29672330299999, 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0069239999999999996, 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.494 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152580+0000", 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152581+0000", 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3399999999999998e-07 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152587+0000", 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4049999999999997e-06 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.495 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152598+0000", 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1043000000000001e-05 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152607+0000", 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9409999999999996e-06 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153448+0000", 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00084054699999999997 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.496 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153563+0000", 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000115584 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153719+0000", 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015566199999999999 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156373+0000", 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002653617 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.497 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159423+0000", 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030508060000000001 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159472+0000", 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9042000000000002e-05 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159504+0000", 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.1519000000000001e-05 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.498 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.29664042, 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0086667419999999998, 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.499 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152663+0000", 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152664+0000", 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0100000000000003e-07 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152671+0000", 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8870000000000003e-06 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.500 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152679+0000", 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2730000000000002e-06 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152696+0000", 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7373000000000001e-05 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153747+0000", 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001050647 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.501 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153967+0000", 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00021982699999999999 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154122+0000", 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015472399999999999 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156405+0000", 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022838110000000002 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.502 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161248+0000", 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048425669999999999 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161283+0000", 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5166000000000002e-05 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161330+0000", 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6566000000000001e-05 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.503 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.2 epoch 25/18 rep_tid 20 entries 25'1 (0'0) error 2:40dbbc66:::202.00000003:head by mds.2.13:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.295034539, 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0021562719999999999, 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.504 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154269+0000", 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154270+0000", 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.638e-06 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154290+0000", 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392e-05 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.505 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154315+0000", 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4858000000000002e-05 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154335+0000", 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0044e-05 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154711+0000", 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037622099999999999 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.506 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156425+0000", 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017141190000000001 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:11 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.290178645, 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00023418899999999999, 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.507 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159125+0000", 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159126+0000", 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7699999999999992e-07 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.508 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159131+0000", 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8109999999999999e-06 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159143+0000", 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1745e-05 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159153+0000", 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6520000000000001e-06 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.509 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159211+0000", 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8582999999999998e-05 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159225+0000", 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3088000000000001e-05 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159359+0000", 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013433299999999999 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.510 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:15 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.29012824900001, 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000189507, 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.511 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159175+0000", 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159176+0000", 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6899999999999998e-07 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159180+0000", 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1320000000000004e-06 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.512 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159188+0000", 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9319999999999996e-06 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159199+0000", 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1165999999999999e-05 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5071000000000002e-05 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.513 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159262+0000", 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0160000000000008e-06 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159365+0000", 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00010272099999999999 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:12 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:53.514 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.290089319, 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000315354, 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:53.515 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159214+0000", 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2099999999999998e-07 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159217+0000", 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.875e-06 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159223+0000", 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2999999999999998e-06 2024-04-25T12:36:53.516 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159229+0000", 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6740000000000002e-06 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159401+0000", 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00017201599999999999 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159412+0000", 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0349e-05 2024-04-25T12:36:53.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159529+0000", 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000117819 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:16 2.19 e25/18 ondisk, result = 0)", 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.29006105000001, 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00026996799999999999, 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159242+0000", 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.519 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159243+0000", 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.0100000000000001e-07 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159246+0000", 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.687e-06 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159254+0000", 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.072e-06 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.520 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159261+0000", 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2169999999999999e-06 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159408+0000", 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000147637 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159417+0000", 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8899999999999996e-06 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159512+0000", 2024-04-25T12:36:53.521 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5163999999999999e-05 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop_reply(mds.2.13:23 2.c e25/18 ondisk, result = 0)", 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.288146895, 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000182072, 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.522 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161156+0000", 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161157+0000", 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7400000000000002e-07 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.523 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161164+0000", 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3899999999999998e-06 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161180+0000", 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6189999999999999e-05 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161191+0000", 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0981e-05 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.524 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161238+0000", 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.7509999999999997e-05 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161244+0000", 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.694e-06 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.161338+0000", 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4432999999999995e-05 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.525 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.526 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.526 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:53.529 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 -- 172.21.15.123:0/1003090001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f41565f0 msgr2=0x7ff7f406c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 --2- 172.21.15.123:0/1003090001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f41565f0 0x7ff7f406c7b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097a0 tx=0x7ff7e402a210 comp rx=0 tx=0).stop 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 -- 172.21.15.123:0/1003090001 shutdown_connections 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 --2- 172.21.15.123:0/1003090001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f406ccf0 0x7ff7f40c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 --2- 172.21.15.123:0/1003090001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f41565f0 0x7ff7f406c7b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 -- 172.21.15.123:0/1003090001 >> 172.21.15.123:0/1003090001 conn(0x7ff7f4070b70 msgr2=0x7ff7f4070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 -- 172.21.15.123:0/1003090001 shutdown_connections 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.451+0000 7ff7fa400640 1 -- 172.21.15.123:0/1003090001 wait complete. 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.453+0000 7ff7fa400640 1 Processor -- start 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.454+0000 7ff7fa400640 1 -- start start 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f406ccf0 0x7ff7f40c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f40c7dc0 0x7ff7f4152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:53.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f41565f0 0x7ff7f4152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7fa400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4159bb0 con 0x7ff7f406ccf0 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7fa400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41598d0 con 0x7ff7f41565f0 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7fa400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40c4cb0 con 0x7ff7f40c7dc0 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7f9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f406ccf0 0x7ff7f40c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f40c7dc0 0x7ff7f4152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7f9a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f41565f0 0x7ff7f4152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7f9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f406ccf0 0x7ff7f40c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44378/0 (socket says 172.21.15.123:44378) 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f40c7dc0 0x7ff7f4152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:39792/0 (socket says 172.21.15.123:39792) 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7f9a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f41565f0 0x7ff7f4152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:57306/0 (socket says 172.21.15.123:57306) 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.455+0000 7ff7f9000640 1 -- 172.21.15.123:0/3951730657 learned_addr learned my addr 172.21.15.123:0/3951730657 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f9a00640 1 -- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f40c7dc0 msgr2=0x7ff7f4152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f9a00640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f40c7dc0 0x7ff7f4152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f9a00640 1 -- 172.21.15.123:0/3951730657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f406ccf0 msgr2=0x7ff7f40c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:53.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f9a00640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f406ccf0 0x7ff7f40c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f9a00640 1 -- 172.21.15.123:0/3951730657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f41f5d50 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f9000640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f406ccf0 0x7ff7f40c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7ebe00640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f40c7dc0 0x7ff7f4152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f9a00640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f41565f0 0x7ff7f4152650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00ef90 tx=0x7ff7ec00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f1600640 1 -- 172.21.15.123:0/3951730657 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec018020 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f1600640 1 -- 172.21.15.123:0/3951730657 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec014480 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7f1600640 1 -- 172.21.15.123:0/3951730657 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec014870 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f41f6030 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.456+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41f65f0 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.457+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f41537a0 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.459+0000 7ff7f1600640 1 -- 172.21.15.123:0/3951730657 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff7ec005d10 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.460+0000 7ff7f1600640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7d00744e0 0x7ff7d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.460+0000 7ff7f1600640 1 -- 172.21.15.123:0/3951730657 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6685+0+0 (secure 0 0 0) 0x7ff7ec0961f0 con 0x7ff7f41565f0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.460+0000 7ff7f9000640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7d00744e0 0x7ff7d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:53.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.460+0000 7ff7f9000640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7d00744e0 0x7ff7d00769f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0013b0 tx=0x7ff7dc000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:53.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.464+0000 7ff7f1600640 1 -- 172.21.15.123:0/3951730657 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7ec022030 con 0x7ff7f41565f0 2024-04-25T12:36:53.644 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:53.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:53.645 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:53.645 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:53.653 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T12:36:53.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:53.659+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7ff7f4076da0 con 0x7ff7f41565f0 2024-04-25T12:36:53.766 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:53.766 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:53.766 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:53.766 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:53.766 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:53.775 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.936878226, 2024-04-25T12:36:53.888 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0075788410000000002, 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951005+0000", 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.889 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951008+0000", 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.21e-06 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951029+0000", 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1199000000000002e-05 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951042+0000", 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3689000000000001e-05 2024-04-25T12:36:53.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951053+0000", 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0560999999999999e-05 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951243+0000", 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000190392 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951388+0000", 2024-04-25T12:36:53.891 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014414699999999999 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951532+0000", 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014442400000000001 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953150+0000", 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0016183020000000001 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958465+0000", 2024-04-25T12:36:53.892 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005314928 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958526+0000", 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1122000000000002e-05 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958584+0000", 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7867e-05 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.893 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.93679057200001, 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0076957650000000002, 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.894 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951093+0000", 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951094+0000", 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4e-07 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951112+0000", 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8386e-05 2024-04-25T12:36:53.895 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951123+0000", 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0525e-05 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951145+0000", 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.2161999999999999e-05 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951888+0000", 2024-04-25T12:36:53.896 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00074297499999999999 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951999+0000", 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000110646 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952154+0000", 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015521 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.897 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953858+0000", 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0017036309999999999 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958680+0000", 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048224230000000002 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958742+0000", 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2187999999999995e-05 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.898 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958789+0000", 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6678999999999998e-05 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:31 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.936446004, 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010654041, 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.899 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951438+0000", 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951439+0000", 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.19e-06 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951456+0000", 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7295e-05 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951469+0000", 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2520999999999999e-05 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951480+0000", 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1742e-05 2024-04-25T12:36:53.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951557+0000", 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6829000000000006e-05 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951745+0000", 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00018825299999999999 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951950+0000", 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00020443999999999999 2024-04-25T12:36:53.902 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953833+0000", 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0018829980000000001 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.961975+0000", 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0081419049999999996 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962033+0000", 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8708000000000001e-05 2024-04-25T12:36:53.903 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.962092+0000", 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8159999999999999e-05 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.93637353400001, 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012888956, 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.904 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951510+0000", 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.905 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951511+0000", 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4900000000000005e-07 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951525+0000", 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4188999999999999e-05 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.906 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951535+0000", 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0027000000000001e-05 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951561+0000", 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5612000000000001e-05 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952340+0000", 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00077879900000000005 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.907 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952470+0000", 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000130202 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952641+0000", 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000171268 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954003+0000", 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001362222 2024-04-25T12:36:53.908 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964280+0000", 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.010276407 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964346+0000", 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6141000000000003e-05 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.964399+0000", 2024-04-25T12:36:53.909 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.3239999999999998e-05 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.93629955599999, 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0073799249999999999, 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.910 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951584+0000", 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951585+0000", 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1459999999999999e-06 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.911 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951591+0000", 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.6810000000000003e-06 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951604+0000", 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2764e-05 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.951614+0000", 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.022e-05 2024-04-25T12:36:53.912 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952409+0000", 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079497200000000002 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952564+0000", 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015539899999999999 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.952714+0000", 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014957600000000001 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.955553+0000", 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028394230000000002 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958857+0000", 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.003303388 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958909+0000", 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1872000000000001e-05 2024-04-25T12:36:53.914 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958964+0000", 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5483999999999999e-05 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:12 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'7, mlcod=0'0)", 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.934262184, 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0049951290000000001, 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.915 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953621+0000", 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.916 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953623+0000", 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.08e-06 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953631+0000", 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8619999999999992e-06 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953648+0000", 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7067000000000001e-05 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.917 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953660+0000", 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1347e-05 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953722+0000", 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1947999999999997e-05 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.953744+0000", 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1868000000000001e-05 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.918 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958568+0000", 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048244610000000004 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958617+0000", 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8495999999999998e-05 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:13 2.10 e24/18 2:0d89b25e:::603.00000000:head v 24'8, mlcod=0'0)", 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 154.933402048, 2024-04-25T12:36:53.919 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004189842, 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954482+0000", 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.920 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954483+0000", 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.201e-06 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954492+0000", 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.5030000000000003e-06 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954511+0000", 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9069999999999999e-05 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.921 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954522+0000", 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0519999999999999e-05 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954584+0000", 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1944000000000002e-05 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.954606+0000", 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1732000000000001e-05 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.922 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958620+0000", 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0040144270000000001 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:18.958671+0000", 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.1445e-05 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.923 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.772716393, 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0054723469999999998, 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.924 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115167+0000", 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115170+0000", 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.982e-06 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115181+0000", 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0922e-05 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.925 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115200+0000", 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8436000000000001e-05 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115214+0000", 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4345000000000001e-05 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115292+0000", 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.7694999999999994e-05 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.926 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115493+0000", 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000201058 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115633+0000", 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014031800000000001 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117391+0000", 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001758017 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120529+0000", 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0031381059999999999 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120591+0000", 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2259000000000001e-05 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120640+0000", 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8208999999999999e-05 2024-04-25T12:36:53.928 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.772640279, 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.012668021999999999, 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.929 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115243+0000", 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115244+0000", 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.7899999999999997e-07 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115250+0000", 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.536e-06 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115259+0000", 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.6949999999999995e-06 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115268+0000", 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9879999999999993e-06 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.931 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116553+0000", 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001284451 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116643+0000", 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9944000000000005e-05 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.116792+0000", 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014905100000000001 2024-04-25T12:36:53.932 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119863+0000", 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0030709219999999998 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127790+0000", 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0079269560000000006 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.933 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127869+0000", 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.8839999999999997e-05 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.127911+0000", 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.2759999999999997e-05 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:53.934 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.77259461, 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0056396570000000002, 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.935 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115289+0000", 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115290+0000", 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.7999999999999995e-07 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115297+0000", 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3660000000000003e-06 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.936 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115306+0000", 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.799e-06 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115314+0000", 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0639999999999994e-06 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115381+0000", 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6839000000000002e-05 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.937 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115540+0000", 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015888499999999999 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115692+0000", 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000152082 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.117333+0000", 2024-04-25T12:36:53.938 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001641359 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120838+0000", 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035047799999999999 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120892+0000", 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.4410000000000003e-05 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.939 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.120929+0000", 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6393000000000002e-05 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.1.9:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.77254983500001, 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.004562455, 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "started", 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.940 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115334+0000", 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0299999999999998e-07 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115338+0000", 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.7220000000000001e-06 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115355+0000", 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6949e-05 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115364+0000", 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6889999999999993e-06 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.942 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115454+0000", 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.0298999999999997e-05 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.115600+0000", 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014599 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.119896+0000", 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042961029999999999 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.943 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.1.9:12 2.12 e25/18 2:48cd7578:::60e.00000000:head v 25'2, mlcod=0'0)", 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.76593341100001, 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042358630000000003, 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.944 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121950+0000", 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121951+0000", 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5199999999999997e-07 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.945 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121962+0000", 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.098e-05 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121971+0000", 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.1949999999999992e-06 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.121990+0000", 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9392999999999999e-05 2024-04-25T12:36:53.946 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122330+0000", 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00033970499999999999 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122356+0000", 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.5865000000000001e-05 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126137+0000", 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037806599999999999 2024-04-25T12:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126186+0000", 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.9512999999999998e-05 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.12 epoch 25/18 rep_tid 11 entries 25'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 123.76587392099999, 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0042052929999999997, 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.948 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.949 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122010+0000", 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.5000000000000003e-07 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122020+0000", 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.346e-06 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122040+0000", 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0174999999999999e-05 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.950 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122067+0000", 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7698000000000001e-05 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.122569+0000", 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000501301 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:50.126215+0000", 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0036462230000000001 2024-04-25T12:36:53.951 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.734903629, 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0037807079999999998, 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.952 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152980+0000", 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152982+0000", 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1509999999999998e-06 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.152993+0000", 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0444e-05 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153010+0000", 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7657999999999999e-05 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153025+0000", 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5014999999999999e-05 2024-04-25T12:36:53.954 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153106+0000", 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.0317999999999999e-05 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153267+0000", 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000161094 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153402+0000", 2024-04-25T12:36:53.955 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000135406 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.155675+0000", 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0022733509999999998 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156660+0000", 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00098492499999999999 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.956 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156719+0000", 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8687000000000002e-05 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156761+0000", 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1659000000000002e-05 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:53.957 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.734826118, 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0062839580000000001, 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153058+0000", 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153073+0000", 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4601e-05 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153083+0000", 2024-04-25T12:36:53.959 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0169e-05 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153092+0000", 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.9730000000000003e-06 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153812+0000", 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00072030800000000002 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.960 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153898+0000", 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6205999999999994e-05 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 0", 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154045+0000", 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014624899999999999 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156442+0000", 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.002397185 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.961 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159231+0000", 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0027891679999999999 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159297+0000", 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.5761999999999998e-05 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.159341+0000", 2024-04-25T12:36:53.962 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.4617999999999999e-05 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.734736379, 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0048832199999999997, 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.963 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153147+0000", 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153148+0000", 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.1900000000000002e-07 2024-04-25T12:36:53.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153152+0000", 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.9210000000000002e-06 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153160+0000", 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3899999999999993e-06 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153169+0000", 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.8969999999999997e-06 2024-04-25T12:36:53.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153844+0000", 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00067528300000000005 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153929+0000", 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4448000000000002e-05 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154069+0000", 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00013969399999999999 2024-04-25T12:36:53.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.156483+0000", 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024146580000000001 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157904+0000", 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001421124 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157973+0000", 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.8887000000000006e-05 2024-04-25T12:36:53.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158030+0000", 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.7198999999999999e-05 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e25)", 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.734702207, 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.005012814, 2024-04-25T12:36:53.968 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153181+0000", 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.969 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153182+0000", 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9299999999999998e-07 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153187+0000", 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.5070000000000002e-06 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153193+0000", 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.353e-06 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.970 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153200+0000", 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.2250000000000003e-06 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154331+0000", 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.001131171 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154482+0000", 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00015075700000000001 2024-04-25T12:36:53.971 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 5", 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154703+0000", 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000220633 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157508+0000", 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028056330000000001 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158088+0000", 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00057988400000000002 2024-04-25T12:36:53.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158148+0000", 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9546999999999998e-05 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.158194+0000", 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.6511000000000002e-05 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.973 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1e epoch 25/18 rep_tid 7 entries 25'2 (0'0) error 2:7d92db23:::202.00000002:head by mds.2.13:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.734419465, 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0035954120000000001, 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.974 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153464+0000", 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153466+0000", 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4470000000000001e-06 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153481+0000", 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4939e-05 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.975 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153502+0000", 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.141e-05 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.153522+0000", 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0262e-05 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.154211+0000", 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00068920199999999998 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.976 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157060+0000", 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0028481520000000001 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.10 epoch 25/18 rep_tid 31 entries 25'15 (0'0) error 2:0eee8b97:::202.00000004:head by mds.2.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'9)", 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 118.73027903400001, 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0053148750000000002, 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.977 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.978 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157605+0000", 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6400000000000001e-07 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157615+0000", 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4970000000000007e-06 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157632+0000", 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.7268000000000001e-05 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.979 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157648+0000", 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.5758999999999999e-05 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.157732+0000", 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.4053000000000003e-05 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:34:55.162919+0000", 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0051875339999999997 2024-04-25T12:36:53.980 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_op(mds.0.4:65 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:53.164195+0000", 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 0.72368906799999999, 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0045161910000000001, 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:53.981 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164195+0000", 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164195+0000", 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164196+0000", 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.11e-06 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.982 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164204+0000", 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.9340000000000006e-06 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164217+0000", 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3485999999999999e-05 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164227+0000", 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.4669999999999993e-06 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.983 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164290+0000", 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.347e-05 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op must wait for map", 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.164297+0000", 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.0999999999999998e-06 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166707+0000", 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.0024097609999999998 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.984 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166771+0000", 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.3632000000000003e-05 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "waiting for subops from 4", 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166914+0000", 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00014372500000000001 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "op_commit", 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.167808+0000", 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00089372599999999996 2024-04-25T12:36:53.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_commit_rec", 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.168602+0000", 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00079401899999999997 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.168662+0000", 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.9855000000000002e-05 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.168711+0000", 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8906000000000002e-05 2024-04-25T12:36:53.986 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.987 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.987 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.987 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:53.987 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:53.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T12:36:54.101 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:54.101 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:54.101 INFO:teuthology.orchestra.run.smithi153.stdout: "num_ops": 0 2024-04-25T12:36:54.101 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:54.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T12:36:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.117+0000 7ff7f1600640 1 -- 172.21.15.123:0/3951730657 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v71) ==== 113+0+0 (secure 0 0 0) 0x7ff7ec010040 con 0x7ff7f41565f0 2024-04-25T12:36:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.2 to 1 (10000) 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7d00744e0 msgr2=0x7ff7d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7d00744e0 0x7ff7d00769f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0013b0 tx=0x7ff7dc000ed0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f41565f0 msgr2=0x7ff7f4152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f41565f0 0x7ff7f4152650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00ef90 tx=0x7ff7ec00c490 comp rx=0 tx=0).stop 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 shutdown_connections 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff7d00744e0 0x7ff7d00769f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f41565f0 0x7ff7f4152650 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f40c7dc0 0x7ff7f4152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 --2- 172.21.15.123:0/3951730657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f406ccf0 0x7ff7f40c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 >> 172.21.15.123:0/3951730657 conn(0x7ff7f4070b70 msgr2=0x7ff7f40ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 shutdown_connections 2024-04-25T12:36:54.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.122+0000 7ff7fa400640 1 -- 172.21.15.123:0/3951730657 wait complete. 2024-04-25T12:36:54.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-25T12:36:54.225 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:54.225 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [], 2024-04-25T12:36:54.225 INFO:teuthology.orchestra.run.smithi153.stdout: "complaint_time": 180, 2024-04-25T12:36:54.225 INFO:teuthology.orchestra.run.smithi153.stdout: "num_blocked_ops": 0 2024-04-25T12:36:54.225 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:54.234 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T12:36:54.346 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-04-25T12:36:54.346 INFO:teuthology.orchestra.run.smithi153.stdout: "size": 20, 2024-04-25T12:36:54.346 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 600, 2024-04-25T12:36:54.346 INFO:teuthology.orchestra.run.smithi153.stdout: "ops": [ 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "MOSDECSubOpWrite(4.12s2 67/47 ECSubWrite(tid=35, reqid=client.4742.0:2, at_version=67'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:49.053569+0000", 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 5.2936641160000004, 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00063699000000000004, 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053569+0000", 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.347 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053569+0000", 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053570+0000", 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8899999999999999e-06 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053603+0000", 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2410999999999999e-05 2024-04-25T12:36:54.348 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053621+0000", 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.8372999999999998e-05 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053634+0000", 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.2435999999999999e-05 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053692+0000", 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.8430000000000001e-05 2024-04-25T12:36:54.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_started", 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.053701+0000", 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.6829999999999997e-06 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_committed", 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.054157+0000", 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00045579000000000003 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.350 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.054206+0000", 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8977000000000002e-05 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "MOSDECSubOpWrite(4.12s2 67/47 ECSubWrite(tid=36, reqid=client.4742.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=67'1))", 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:49.055351+0000", 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 5.2918812180000003, 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00040015899999999997, 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.351 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055351+0000", 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055351+0000", 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055352+0000", 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.2799999999999996e-07 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.352 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055362+0000", 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.8959999999999998e-06 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055372+0000", 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.048e-05 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055380+0000", 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.6020000000000002e-06 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.353 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055428+0000", 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8353000000000003e-05 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_started", 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055438+0000", 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3190000000000001e-06 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_committed", 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055725+0000", 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00028730299999999998 2024-04-25T12:36:54.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.055752+0000", 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.6577999999999999e-05 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "MOSDECSubOpWrite(4.12s2 67/47 ECSubWrite(tid=37, reqid=client.4742.0:3, at_version=67'2, trim_to=0'0, roll_forward_to=67'1))", 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:49.080033+0000", 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 5.267199733, 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00058533900000000004, 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080033+0000", 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080033+0000", 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.356 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080034+0000", 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.1999999999999999e-07 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080053+0000", 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.9701e-05 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080064+0000", 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1136e-05 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080072+0000", 2024-04-25T12:36:54.357 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.3420000000000002e-06 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080120+0000", 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.8599e-05 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_started", 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080127+0000", 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.6429999999999998e-06 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_committed", 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080585+0000", 2024-04-25T12:36:54.358 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000458429 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.080618+0000", 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.2869e-05 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "MOSDECSubOpWrite(4.12s2 67/47 ECSubWrite(tid=38, reqid=client.4742.0:3, at_version=0'0, trim_to=0'0, roll_forward_to=67'2))", 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:49.084563+0000", 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 5.262670118, 2024-04-25T12:36:54.359 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00049459999999999999, 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084563+0000", 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084563+0000", 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.360 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084564+0000", 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.9800000000000002e-07 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084577+0000", 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3005e-05 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084590+0000", 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3206e-05 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.361 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084603+0000", 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3614e-05 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084668+0000", 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 6.4228999999999998e-05 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_started", 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.084678+0000", 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0378e-05 2024-04-25T12:36:54.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_committed", 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.085024+0000", 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000345762 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:49.085057+0000", 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.3408000000000003e-05 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "pg_update_log_missing(2.1 epoch 70/47 rep_tid 14 entries 70'5 (0'0) error 2:86ee9bce:::500.00000001:head by mds.0.4:59 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:53.166544+0000", 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 1.180688344, 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00052618900000000004, 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166544+0000", 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166544+0000", 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166545+0000", 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.53e-07 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166555+0000", 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0084999999999999e-05 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166580+0000", 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.4882999999999999e-05 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166601+0000", 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.1630000000000001e-05 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.166699+0000", 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.7318999999999994e-05 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.366 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.167070+0000", 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00037191900000000001 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "osd_repop(mds.0.4:66 2.1 e70/47 2:86ee9bce:::500.00000001:head v 70'6, mlcod=25'4)", 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:53.176184+0000", 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 1.171048243, 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000556767, 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.367 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176184+0000", 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176184+0000", 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.368 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176185+0000", 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 9.3200000000000003e-07 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176196+0000", 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0373999999999999e-05 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176207+0000", 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.1267000000000001e-05 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.369 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176215+0000", 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.3110000000000006e-06 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176266+0000", 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 5.0590000000000002e-05 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "started", 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176282+0000", 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.6320999999999998e-05 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.370 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "commit_sent", 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176706+0000", 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.000423779 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.176741+0000", 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.5193e-05 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "MOSDECSubOpWrite(4.12s2 70/47 ECSubWrite(tid=43, reqid=mds.0.4:69, at_version=70'3, trim_to=0'0, roll_forward_to=67'2))", 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:53.179460+0000", 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 1.167772313, 2024-04-25T12:36:54.371 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00097680399999999995, 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179460+0000", 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179460+0000", 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.372 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179461+0000", 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 4.1300000000000001e-07 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179482+0000", 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.0766000000000001e-05 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179495+0000", 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3019e-05 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.373 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179505+0000", 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.031e-05 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179580+0000", 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.5301000000000003e-05 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_started", 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.179590+0000", 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0355e-05 2024-04-25T12:36:54.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_committed", 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.180410+0000", 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00081932000000000001 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.180437+0000", 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 2.7319999999999999e-05 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.375 INFO:teuthology.orchestra.run.smithi153.stdout: "description": "MOSDECSubOpWrite(4.12s2 70/47 ECSubWrite(tid=44, reqid=mds.0.4:69, at_version=0'0, trim_to=0'0, roll_forward_to=70'3))", 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "initiated_at": "2024-04-25T12:36:53.182562+0000", 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "age": 1.1646701799999999, 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00054150699999999999, 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "continuous": false, 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "type_data": { 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "flag_point": "reached pg", 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "events": [ 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "initiated", 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182562+0000", 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.376 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "header_read", 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182562+0000", 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "throttled", 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182563+0000", 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 8.2399999999999997e-07 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "all_read", 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182577+0000", 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.3885e-05 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.377 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "dispatched", 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182592+0000", 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.4899000000000001e-05 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "queued_for_pg", 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182603+0000", 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0750999999999999e-05 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "reached_pg", 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182677+0000", 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 7.4086999999999997e-05 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_started", 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.182687+0000", 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 1.0356e-05 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "sub_op_committed", 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.183067+0000", 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 0.00038011300000000001 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: { 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "event": "done", 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "time": "2024-04-25T12:36:53.183104+0000", 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: "duration": 3.6591999999999998e-05 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.379 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.380 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.380 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-04-25T12:36:54.380 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-04-25T12:36:54.380 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-04-25T12:36:54.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.418+0000 7fe603200640 1 Processor -- start 2024-04-25T12:36:54.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.418+0000 7fe603200640 1 -- start start 2024-04-25T12:36:54.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.420+0000 7fe603200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe6040c7e30 0x7fe604154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:54.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.421+0000 7fe603200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe604074290 con 0x7fe604072cc0 2024-04-25T12:36:54.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.421+0000 7fe603200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe604074570 con 0x7fe6040730d0 2024-04-25T12:36:54.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.421+0000 7fe603200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe604074850 con 0x7fe6040c7e30 2024-04-25T12:36:54.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.421+0000 7fe601400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe604072cc0 0x7fe60406c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55506/0 (socket says 172.21.15.123:55506) 2024-04-25T12:36:54.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.421+0000 7fe601400640 1 -- 172.21.15.123:0/2734763513 learned_addr learned my addr 172.21.15.123:0/2734763513 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:54.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.421+0000 7fe601e00640 1 --2- 172.21.15.123:0/2734763513 >> v2:172.21.15.153:3300/0 conn(0x7fe6040c7e30 0x7fe604154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.421+0000 7fe600a00640 1 -- 172.21.15.123:0/2734763513 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244441469 0 0) 0x7fe604074290 con 0x7fe604072cc0 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.422+0000 7fe600a00640 1 -- 172.21.15.123:0/2734763513 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5e8003520 con 0x7fe604072cc0 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.422+0000 7fe600a00640 1 -- 172.21.15.123:0/2734763513 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710697976 0 0) 0x7fe604074570 con 0x7fe6040730d0 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.422+0000 7fe600a00640 1 -- 172.21.15.123:0/2734763513 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe604074290 con 0x7fe6040730d0 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.422+0000 7fe601e00640 1 -- 172.21.15.123:0/2734763513 >> v1:172.21.15.153:6789/0 conn(0x7fe6040730d0 legacy=0x7fe60415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.422+0000 7fe601e00640 1 -- 172.21.15.123:0/2734763513 >> v1:172.21.15.123:6789/0 conn(0x7fe604072cc0 legacy=0x7fe60406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.422+0000 7fe601e00640 1 -- 172.21.15.123:0/2734763513 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe604074b30 con 0x7fe6040c7e30 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.423+0000 7fe601e00640 1 --2- 172.21.15.123:0/2734763513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 0x7fe604154280 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40097a0 tx=0x7fe5f402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b58c145d28f0df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.423+0000 7fe600a00640 1 -- 172.21.15.123:0/2734763513 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4037070 con 0x7fe6040c7e30 2024-04-25T12:36:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.423+0000 7fe600a00640 1 -- 172.21.15.123:0/2734763513 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f4032440 con 0x7fe6040c7e30 2024-04-25T12:36:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.423+0000 7fe600a00640 1 -- 172.21.15.123:0/2734763513 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4032810 con 0x7fe6040c7e30 2024-04-25T12:36:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.424+0000 7fe603200640 1 -- 172.21.15.123:0/2734763513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 msgr2=0x7fe604154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.424+0000 7fe603200640 1 --2- 172.21.15.123:0/2734763513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 0x7fe604154280 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40097a0 tx=0x7fe5f402a230 comp rx=0 tx=0).stop 2024-04-25T12:36:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.424+0000 7fe603200640 1 -- 172.21.15.123:0/2734763513 shutdown_connections 2024-04-25T12:36:54.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.424+0000 7fe603200640 1 --2- 172.21.15.123:0/2734763513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 0x7fe604154280 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.424+0000 7fe603200640 1 -- 172.21.15.123:0/2734763513 >> 172.21.15.123:0/2734763513 conn(0x7fe604070b70 msgr2=0x7fe604070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:54.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.424+0000 7fe603200640 1 -- 172.21.15.123:0/2734763513 shutdown_connections 2024-04-25T12:36:54.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.424+0000 7fe603200640 1 -- 172.21.15.123:0/2734763513 wait complete. 2024-04-25T12:36:54.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.427+0000 7fe603200640 1 Processor -- start 2024-04-25T12:36:54.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.427+0000 7fe603200640 1 -- start start 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe603200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe604072cc0 0x7fe6041f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe603200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6040730d0 0x7fe604142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe603200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 0x7fe6041495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe603200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6040c8240 con 0x7fe6040730d0 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe603200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041547c0 con 0x7fe6040c7e30 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe603200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe6040c83c0 con 0x7fe604072cc0 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe601e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe604072cc0 0x7fe6041f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe601400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6040730d0 0x7fe604142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe601e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe604072cc0 0x7fe6041f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:39794/0 (socket says 172.21.15.123:39794) 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe601e00640 1 -- 172.21.15.123:0/2219437469 learned_addr learned my addr 172.21.15.123:0/2219437469 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.429+0000 7fe602800640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 0x7fe6041495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe601e00640 1 -- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 msgr2=0x7fe6041495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe601e00640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 0x7fe6041495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe601e00640 1 -- 172.21.15.123:0/2219437469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6040730d0 msgr2=0x7fe604142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe601e00640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6040730d0 0x7fe604142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe601e00640 1 -- 172.21.15.123:0/2219437469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6041e4dc0 con 0x7fe604072cc0 2024-04-25T12:36:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe601e00640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe604072cc0 0x7fe6041f1f40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe5f402a820 tx=0x7fe5f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe5fa000640 1 -- 172.21.15.123:0/2219437469 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4041030 con 0x7fe604072cc0 2024-04-25T12:36:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe5fa000640 1 -- 172.21.15.123:0/2219437469 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f403d4e0 con 0x7fe604072cc0 2024-04-25T12:36:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.430+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6041e5060 con 0x7fe604072cc0 2024-04-25T12:36:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.431+0000 7fe5fa000640 1 -- 172.21.15.123:0/2219437469 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4037040 con 0x7fe604072cc0 2024-04-25T12:36:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.431+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe604146e30 con 0x7fe604072cc0 2024-04-25T12:36:54.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.432+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe604076da0 con 0x7fe604072cc0 2024-04-25T12:36:54.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.433+0000 7fe5fa000640 1 -- 172.21.15.123:0/2219437469 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe5f4044070 con 0x7fe604072cc0 2024-04-25T12:36:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.434+0000 7fe5fa000640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe5e40744b0 0x7fe5e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.434+0000 7fe5fa000640 1 -- 172.21.15.123:0/2219437469 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 6685+0+0 (secure 0 0 0) 0x7fe5f40b5960 con 0x7fe604072cc0 2024-04-25T12:36:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.434+0000 7fe601400640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe5e40744b0 0x7fe5e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:54.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.435+0000 7fe601400640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe5e40744b0 0x7fe5e40769c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0000f60 tx=0x7fe5f0007d70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:54.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.438+0000 7fe5fa000640 1 -- 172.21.15.123:0/2219437469 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5f404b070 con 0x7fe604072cc0 2024-04-25T12:36:54.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:54.632+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7fe604073690 con 0x7fe604072cc0 2024-04-25T12:36:55.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.121+0000 7fe5fa000640 1 -- 172.21.15.123:0/2219437469 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v72) ==== 113+0+0 (secure 0 0 0) 0x7fe5f407f6d0 con 0x7fe604072cc0 2024-04-25T12:36:55.121 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.3 to 1 (10000) 2024-04-25T12:36:55.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.125+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe5e40744b0 msgr2=0x7fe5e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.125+0000 7fe603200640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe5e40744b0 0x7fe5e40769c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0000f60 tx=0x7fe5f0007d70 comp rx=0 tx=0).stop 2024-04-25T12:36:55.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe604072cc0 msgr2=0x7fe6041f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe604072cc0 0x7fe6041f1f40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe5f402a820 tx=0x7fe5f40046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 shutdown_connections 2024-04-25T12:36:55.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe5e40744b0 0x7fe5e40769c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6040c7e30 0x7fe6041495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6040730d0 0x7fe604142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 --2- 172.21.15.123:0/2219437469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe604072cc0 0x7fe6041f1f40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 >> 172.21.15.123:0/2219437469 conn(0x7fe604070b70 msgr2=0x7fe6040c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:55.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 shutdown_connections 2024-04-25T12:36:55.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.126+0000 7fe603200640 1 -- 172.21.15.123:0/2219437469 wait complete. 2024-04-25T12:36:55.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T12:36:55.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-25T12:36:55.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.412+0000 7fa04ac00640 1 Processor -- start 2024-04-25T12:36:55.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.413+0000 7fa04ac00640 1 -- start start 2024-04-25T12:36:55.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.414+0000 7fa04ac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa04415d180 0x7fa0440c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.414+0000 7fa04ac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0441569e0 0x7fa0440c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.414+0000 7fa04ac00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0441600e0 con 0x7fa04415fcd0 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.414+0000 7fa04ac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa04415d590 con 0x7fa0441569e0 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.414+0000 7fa04ac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa0440c3880 con 0x7fa04415d180 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa048c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0441569e0 0x7fa0440c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa03be00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa04415fcd0 0x7fa044070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:54074/0 (socket says 172.21.15.123:54074) 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa03be00640 1 -- 172.21.15.123:0/649688738 learned_addr learned my addr 172.21.15.123:0/649688738 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa043e00640 1 --2- 172.21.15.123:0/649688738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa04415d180 0x7fa0440c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa043400640 1 -- 172.21.15.123:0/649688738 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139420048 0 0) 0x7fa02c001110 con 0x7fa04415fcd0 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa043400640 1 -- 172.21.15.123:0/649688738 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa028003520 con 0x7fa04415fcd0 2024-04-25T12:36:55.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa043e00640 1 -- 172.21.15.123:0/649688738 >> v2:172.21.15.123:3300/0 conn(0x7fa0441569e0 msgr2=0x7fa0440c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:55.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa043e00640 1 --2- 172.21.15.123:0/649688738 >> v2:172.21.15.123:3300/0 conn(0x7fa0441569e0 0x7fa0440c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa043e00640 1 -- 172.21.15.123:0/649688738 >> v1:172.21.15.153:6789/0 conn(0x7fa04415fcd0 legacy=0x7fa044070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa043e00640 1 -- 172.21.15.123:0/649688738 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0440c54d0 con 0x7fa04415d180 2024-04-25T12:36:55.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.415+0000 7fa048c00640 1 --2- 172.21.15.123:0/649688738 >> v2:172.21.15.123:3300/0 conn(0x7fa0441569e0 0x7fa0440c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:55.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.416+0000 7fa043e00640 1 --2- 172.21.15.123:0/649688738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa04415d180 0x7fa0440c95d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0340097a0 tx=0x7fa03402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af26ca87fadce42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:55.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.417+0000 7fa043400640 1 -- 172.21.15.123:0/649688738 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034037070 con 0x7fa04415d180 2024-04-25T12:36:55.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.417+0000 7fa043400640 1 -- 172.21.15.123:0/649688738 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03402ae50 con 0x7fa04415d180 2024-04-25T12:36:55.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.417+0000 7fa043400640 1 -- 172.21.15.123:0/649688738 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034032660 con 0x7fa04415d180 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.417+0000 7fa04ac00640 1 -- 172.21.15.123:0/649688738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa04415d180 msgr2=0x7fa0440c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.417+0000 7fa04ac00640 1 --2- 172.21.15.123:0/649688738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa04415d180 0x7fa0440c95d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0340097a0 tx=0x7fa03402a100 comp rx=0 tx=0).stop 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.418+0000 7fa04ac00640 1 -- 172.21.15.123:0/649688738 shutdown_connections 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.418+0000 7fa04ac00640 1 --2- 172.21.15.123:0/649688738 >> v2:172.21.15.123:3300/0 conn(0x7fa0441569e0 0x7fa0440c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.418+0000 7fa04ac00640 1 --2- 172.21.15.123:0/649688738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa04415d180 0x7fa0440c95d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.418+0000 7fa04ac00640 1 -- 172.21.15.123:0/649688738 >> 172.21.15.123:0/649688738 conn(0x7fa044071b90 msgr2=0x7fa044071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.418+0000 7fa04ac00640 1 -- 172.21.15.123:0/649688738 shutdown_connections 2024-04-25T12:36:55.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.418+0000 7fa04ac00640 1 -- 172.21.15.123:0/649688738 wait complete. 2024-04-25T12:36:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.420+0000 7fa04ac00640 1 Processor -- start 2024-04-25T12:36:55.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.420+0000 7fa04ac00640 1 -- start start 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.421+0000 7fa04ac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0441569e0 0x7fa0441f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.421+0000 7fa04ac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa04415d180 0x7fa04415f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.421+0000 7fa04ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04415fcd0 0x7fa044151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.421+0000 7fa04ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0440c5770 con 0x7fa04415fcd0 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.421+0000 7fa04ac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0440c5a50 con 0x7fa04415d180 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.421+0000 7fa04ac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa044156df0 con 0x7fa0441569e0 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0441569e0 0x7fa0441f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0441569e0 0x7fa0441f65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:39806/0 (socket says 172.21.15.123:39806) 2024-04-25T12:36:55.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 -- 172.21.15.123:0/1341103586 learned_addr learned my addr 172.21.15.123:0/1341103586 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa03be00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa04415d180 0x7fa04415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa048c00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04415fcd0 0x7fa044151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 -- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa04415d180 msgr2=0x7fa04415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa04415d180 0x7fa04415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 -- 172.21.15.123:0/1341103586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04415fcd0 msgr2=0x7fa044151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04415fcd0 0x7fa044151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.422+0000 7fa043e00640 1 -- 172.21.15.123:0/1341103586 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0441f2560 con 0x7fa0441569e0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.423+0000 7fa043e00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0441569e0 0x7fa0441f65b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa034035040 tx=0x7fa0340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.423+0000 7fa040c00640 1 -- 172.21.15.123:0/1341103586 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034041070 con 0x7fa0441569e0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.423+0000 7fa040c00640 1 -- 172.21.15.123:0/1341103586 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03403c460 con 0x7fa0441569e0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.423+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04414e880 con 0x7fa0441569e0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.423+0000 7fa040c00640 1 -- 172.21.15.123:0/1341103586 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034037040 con 0x7fa0441569e0 2024-04-25T12:36:55.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.423+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0440cb130 con 0x7fa0441569e0 2024-04-25T12:36:55.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.424+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa044076700 con 0x7fa0441569e0 2024-04-25T12:36:55.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.425+0000 7fa040c00640 1 -- 172.21.15.123:0/1341103586 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa03403a070 con 0x7fa0441569e0 2024-04-25T12:36:55.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.426+0000 7fa040c00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa0200744b0 0x7fa0200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.426+0000 7fa040c00640 1 -- 172.21.15.123:0/1341103586 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(72..72 src has 1..72) ==== 6685+0+0 (secure 0 0 0) 0x7fa0340b5180 con 0x7fa0441569e0 2024-04-25T12:36:55.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.426+0000 7fa03be00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa0200744b0 0x7fa0200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.427+0000 7fa03be00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa0200744b0 0x7fa0200769c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005880 tx=0x7fa02c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:55.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.431+0000 7fa040c00640 1 -- 172.21.15.123:0/1341103586 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa03407ef70 con 0x7fa0441569e0 2024-04-25T12:36:55.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.456+0000 7fde45600640 1 Processor -- start 2024-04-25T12:36:55.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.456+0000 7fde45600640 1 -- start start 2024-04-25T12:36:55.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.457+0000 7fde45600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde40154650 0x7fde4015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde45600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fde400c9f00 0x7fde400c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde45600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4015b380 con 0x7fde400730e0 2024-04-25T12:36:55.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde45600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fde40074b20 con 0x7fde400c9f00 2024-04-25T12:36:55.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde45600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fde400ca780 con 0x7fde40154650 2024-04-25T12:36:55.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde3f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fde400730e0 0x7fde400708f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55512/0 (socket says 172.21.15.123:55512) 2024-04-25T12:36:55.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde3f400640 1 -- 172.21.15.123:0/1696348664 learned_addr learned my addr 172.21.15.123:0/1696348664 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:55.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde3ea00640 1 --2- 172.21.15.123:0/1696348664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde40154650 0x7fde4015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde3e000640 1 --2- 172.21.15.123:0/1696348664 >> v2:172.21.15.153:3300/0 conn(0x7fde400c9f00 0x7fde400c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde3d600640 1 -- 172.21.15.123:0/1696348664 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693451013 0 0) 0x7fde38001490 con 0x7fde400730e0 2024-04-25T12:36:55.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.458+0000 7fde3d600640 1 -- 172.21.15.123:0/1696348664 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde24003520 con 0x7fde400730e0 2024-04-25T12:36:55.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.459+0000 7fde3ea00640 1 -- 172.21.15.123:0/1696348664 >> v2:172.21.15.153:3300/0 conn(0x7fde400c9f00 msgr2=0x7fde400c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:55.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.459+0000 7fde3ea00640 1 --2- 172.21.15.123:0/1696348664 >> v2:172.21.15.153:3300/0 conn(0x7fde400c9f00 0x7fde400c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.459+0000 7fde3ea00640 1 -- 172.21.15.123:0/1696348664 >> v1:172.21.15.123:6789/0 conn(0x7fde400730e0 legacy=0x7fde400708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.459+0000 7fde3ea00640 1 -- 172.21.15.123:0/1696348664 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde400caa60 con 0x7fde40154650 2024-04-25T12:36:55.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.459+0000 7fde3e000640 1 --2- 172.21.15.123:0/1696348664 >> v2:172.21.15.153:3300/0 conn(0x7fde400c9f00 0x7fde400c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:55.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.460+0000 7fde3ea00640 1 --2- 172.21.15.123:0/1696348664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde40154650 0x7fde4015ae40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde300097a0 tx=0x7fde3002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ba24776a786860 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:55.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.460+0000 7fde3d600640 1 -- 172.21.15.123:0/1696348664 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30037070 con 0x7fde40154650 2024-04-25T12:36:55.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.460+0000 7fde3d600640 1 -- 172.21.15.123:0/1696348664 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde30032440 con 0x7fde40154650 2024-04-25T12:36:55.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.460+0000 7fde3d600640 1 -- 172.21.15.123:0/1696348664 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300327f0 con 0x7fde40154650 2024-04-25T12:36:55.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 -- 172.21.15.123:0/1696348664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde40154650 msgr2=0x7fde4015ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 --2- 172.21.15.123:0/1696348664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde40154650 0x7fde4015ae40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde300097a0 tx=0x7fde3002a210 comp rx=0 tx=0).stop 2024-04-25T12:36:55.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 -- 172.21.15.123:0/1696348664 shutdown_connections 2024-04-25T12:36:55.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 --2- 172.21.15.123:0/1696348664 >> v2:172.21.15.153:3300/0 conn(0x7fde400c9f00 0x7fde400c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 --2- 172.21.15.123:0/1696348664 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde40154650 0x7fde4015ae40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 -- 172.21.15.123:0/1696348664 >> 172.21.15.123:0/1696348664 conn(0x7fde4006f440 msgr2=0x7fde4006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:55.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 -- 172.21.15.123:0/1696348664 shutdown_connections 2024-04-25T12:36:55.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.461+0000 7fde45600640 1 -- 172.21.15.123:0/1696348664 wait complete. 2024-04-25T12:36:55.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.464+0000 7fde45600640 1 Processor -- start 2024-04-25T12:36:55.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.464+0000 7fde45600640 1 -- start start 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.465+0000 7fde45600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde400730e0 0x7fde401f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.465+0000 7fde45600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde400c9f00 0x7fde40146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.465+0000 7fde45600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde40154650 0x7fde401592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.465+0000 7fde45600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde4015e010 con 0x7fde400730e0 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.465+0000 7fde45600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde4015dd30 con 0x7fde40154650 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.465+0000 7fde45600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fde4015e2f0 con 0x7fde400c9f00 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde400730e0 0x7fde401f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde400c9f00 0x7fde40146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde400c9f00 0x7fde40146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:39824/0 (socket says 172.21.15.123:39824) 2024-04-25T12:36:55.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde400730e0 0x7fde401f6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44412/0 (socket says 172.21.15.123:44412) 2024-04-25T12:36:55.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 -- 172.21.15.123:0/608172158 learned_addr learned my addr 172.21.15.123:0/608172158 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:55.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3f400640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde40154650 0x7fde401592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 -- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde40154650 msgr2=0x7fde401592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde40154650 0x7fde401592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 -- 172.21.15.123:0/608172158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde400730e0 msgr2=0x7fde401f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:55.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde400730e0 0x7fde401f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:55.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3e000640 1 -- 172.21.15.123:0/608172158 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde40146850 con 0x7fde400c9f00 2024-04-25T12:36:55.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.466+0000 7fde3f400640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde40154650 0x7fde401592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:55.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.467+0000 7fde3e000640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde400c9f00 0x7fde40146310 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00e900 tx=0x7fde2c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:55.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.467+0000 7fde36a00640 1 -- 172.21.15.123:0/608172158 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c017070 con 0x7fde400c9f00 2024-04-25T12:36:55.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.467+0000 7fde36a00640 1 -- 172.21.15.123:0/608172158 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2c00ca00 con 0x7fde400c9f00 2024-04-25T12:36:55.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.467+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde401f33f0 con 0x7fde400c9f00 2024-04-25T12:36:55.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.467+0000 7fde36a00640 1 -- 172.21.15.123:0/608172158 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00cdf0 con 0x7fde400c9f00 2024-04-25T12:36:55.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.467+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fde401f3c50 con 0x7fde400c9f00 2024-04-25T12:36:55.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.468+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde40076420 con 0x7fde400c9f00 2024-04-25T12:36:55.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.470+0000 7fde36a00640 1 -- 172.21.15.123:0/608172158 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde2c01a070 con 0x7fde400c9f00 2024-04-25T12:36:55.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.471+0000 7fde36a00640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744b0 0x7fde200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:55.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.471+0000 7fde36a00640 1 -- 172.21.15.123:0/608172158 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(72..72 src has 1..72) ==== 6685+0+0 (secure 0 0 0) 0x7fde2c094f90 con 0x7fde400c9f00 2024-04-25T12:36:55.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.472+0000 7fde3ea00640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744b0 0x7fde200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:55.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.472+0000 7fde3ea00640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744b0 0x7fde200769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde30034000 tx=0x7fde3002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:55.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.476+0000 7fde36a00640 1 -- 172.21.15.123:0/608172158 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde40076420 con 0x7fde400c9f00 2024-04-25T12:36:55.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.636+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fa0440711a0 con 0x7fa0441569e0 2024-04-25T12:36:55.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:55.686+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7fde40076700 con 0x7fde400c9f00 2024-04-25T12:36:56.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.141+0000 7fa040c00640 1 -- 172.21.15.123:0/1341103586 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v73) ==== 91+0+0 (secure 0 0 0) 0x7fa034083270 con 0x7fa0441569e0 2024-04-25T12:36:56.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.141+0000 7fde36a00640 1 -- 172.21.15.123:0/608172158 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v73) ==== 113+0+0 (secure 0 0 0) 0x7fde2c05ee00 con 0x7fde400c9f00 2024-04-25T12:36:56.142 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.4 to 1 (10000) 2024-04-25T12:36:56.143 INFO:teuthology.orchestra.run.smithi123.stderr:noscrub is unset 2024-04-25T12:36:56.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744b0 msgr2=0x7fde200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744b0 0x7fde200769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde30034000 tx=0x7fde3002e000 comp rx=0 tx=0).stop 2024-04-25T12:36:56.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde400c9f00 msgr2=0x7fde40146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde400c9f00 0x7fde40146310 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00e900 tx=0x7fde2c00edc0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 shutdown_connections 2024-04-25T12:36:56.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fde200744b0 0x7fde200769c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde40154650 0x7fde401592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde400c9f00 0x7fde40146310 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 --2- 172.21.15.123:0/608172158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde400730e0 0x7fde401f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 >> 172.21.15.123:0/608172158 conn(0x7fde4006f440 msgr2=0x7fde400c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:56.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 shutdown_connections 2024-04-25T12:36:56.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.147+0000 7fde45600640 1 -- 172.21.15.123:0/608172158 wait complete. 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa0200744b0 msgr2=0x7fa0200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa0200744b0 0x7fa0200769c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005880 tx=0x7fa02c009040 comp rx=0 tx=0).stop 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0441569e0 msgr2=0x7fa0441f65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0441569e0 0x7fa0441f65b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa034035040 tx=0x7fa0340046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 shutdown_connections 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa0200744b0 0x7fa0200769c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04415fcd0 0x7fa044151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa04415d180 0x7fa04415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 --2- 172.21.15.123:0/1341103586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0441569e0 0x7fa0441f65b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 >> 172.21.15.123:0/1341103586 conn(0x7fa044071b90 msgr2=0x7fa044155280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 shutdown_connections 2024-04-25T12:36:56.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.148+0000 7fa04ac00640 1 -- 172.21.15.123:0/1341103586 wait complete. 2024-04-25T12:36:56.177 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-25T12:36:56.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T12:36:56.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.469+0000 7f2a1a600640 1 Processor -- start 2024-04-25T12:36:56.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.469+0000 7f2a1a600640 1 -- start start 2024-04-25T12:36:56.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a1a600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a140c6750 0x7f2a14156850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a1a600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2a1406e0c0 0x7f2a140cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a1a600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a1406e5b0 con 0x7f2a1415fce0 2024-04-25T12:36:56.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a1a600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2a140cb010 con 0x7f2a1406e0c0 2024-04-25T12:36:56.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a1a600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2a140cb2f0 con 0x7f2a140c6750 2024-04-25T12:36:56.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a140c6750 0x7f2a14156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a13e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2a1406e0c0 0x7f2a140cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a13400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2a1415fce0 0x7f2a1406f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:54086/0 (socket says 172.21.15.123:54086) 2024-04-25T12:36:56.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.471+0000 7f2a13400640 1 -- 172.21.15.123:0/2984664704 learned_addr learned my addr 172.21.15.123:0/2984664704 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:56.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.472+0000 7f2a12000640 1 -- 172.21.15.123:0/2984664704 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093296626 0 0) 0x7f2a1406e5b0 con 0x7f2a1415fce0 2024-04-25T12:36:56.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.472+0000 7f2a13e00640 1 -- 172.21.15.123:0/2984664704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a140c6750 msgr2=0x7f2a14156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:56.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.472+0000 7f2a13e00640 1 --2- 172.21.15.123:0/2984664704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a140c6750 0x7f2a14156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.472+0000 7f2a13e00640 1 -- 172.21.15.123:0/2984664704 >> v1:172.21.15.153:6789/0 conn(0x7f2a1415fce0 legacy=0x7f2a1406f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.472+0000 7f2a13e00640 1 -- 172.21.15.123:0/2984664704 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1406e920 con 0x7f2a1406e0c0 2024-04-25T12:36:56.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.472+0000 7f2a12a00640 1 --2- 172.21.15.123:0/2984664704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a140c6750 0x7f2a14156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.473+0000 7f2a13e00640 1 --2- 172.21.15.123:0/2984664704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1406e0c0 0x7f2a140cfb70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009d90 tx=0x7f2a0c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8539843af14d834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.473+0000 7f2a12000640 1 -- 172.21.15.123:0/2984664704 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c007740 con 0x7f2a1406e0c0 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.473+0000 7f2a12000640 1 -- 172.21.15.123:0/2984664704 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0c0078e0 con 0x7f2a1406e0c0 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.473+0000 7f2a12000640 1 -- 172.21.15.123:0/2984664704 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c007cb0 con 0x7f2a1406e0c0 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 -- 172.21.15.123:0/2984664704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1406e0c0 msgr2=0x7f2a140cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2984664704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1406e0c0 0x7f2a140cfb70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009d90 tx=0x7f2a0c02a660 comp rx=0 tx=0).stop 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 -- 172.21.15.123:0/2984664704 shutdown_connections 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2984664704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1406e0c0 0x7f2a140cfb70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2984664704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a140c6750 0x7f2a14156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 -- 172.21.15.123:0/2984664704 >> 172.21.15.123:0/2984664704 conn(0x7f2a14068600 msgr2=0x7f2a14068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 -- 172.21.15.123:0/2984664704 shutdown_connections 2024-04-25T12:36:56.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.474+0000 7f2a1a600640 1 -- 172.21.15.123:0/2984664704 wait complete. 2024-04-25T12:36:56.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.475+0000 7f313e600640 1 Processor -- start 2024-04-25T12:36:56.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.475+0000 7f313e600640 1 -- start start 2024-04-25T12:36:56.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.476+0000 7f2a1a600640 1 Processor -- start 2024-04-25T12:36:56.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.476+0000 7f2a1a600640 1 -- start start 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.477+0000 7f313e600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3138156670 0x7f31380c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.477+0000 7f313e600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313806d850 con 0x7f31381606e0 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.477+0000 7f313e600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3138156a80 con 0x7f313806d440 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.477+0000 7f313e600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3138160af0 con 0x7f3138156670 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.477+0000 7f3137400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f31381606e0 0x7f313815d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55516/0 (socket says 172.21.15.123:55516) 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.477+0000 7f3137400640 1 -- 172.21.15.123:0/3287359933 learned_addr learned my addr 172.21.15.123:0/3287359933 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.477+0000 7f3137e00640 1 --2- 172.21.15.123:0/3287359933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3138156670 0x7f31380c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f3136000640 1 -- 172.21.15.123:0/3287359933 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3534596011 0 0) 0x7f3138156a80 con 0x7f313806d440 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f3136000640 1 -- 172.21.15.123:0/3287359933 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f311c003520 con 0x7f313806d440 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f3136000640 1 -- 172.21.15.123:0/3287359933 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4012368519 0 0) 0x7f3128001110 con 0x7f31381606e0 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f3137e00640 1 -- 172.21.15.123:0/3287359933 >> v1:172.21.15.153:6789/0 conn(0x7f313806d440 legacy=0x7f31380c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f3137e00640 1 -- 172.21.15.123:0/3287359933 >> v1:172.21.15.123:6789/0 conn(0x7f31381606e0 legacy=0x7f313815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f3137e00640 1 -- 172.21.15.123:0/3287359933 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313815fc40 con 0x7f3138156670 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a1a600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a1406e0c0 0x7f2a1414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a1a600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a140c6750 0x7f2a1414e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a1a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1415fce0 0x7f2a1415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a1a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a140cb7e0 con 0x7f2a1415fce0 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a1a600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14156d90 con 0x7f2a140c6750 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a1a600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1406d680 con 0x7f2a1406e0c0 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a13400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a1406e0c0 0x7f2a1414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a13400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a1406e0c0 0x7f2a1414e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:39840/0 (socket says 172.21.15.123:39840) 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a13400640 1 -- 172.21.15.123:0/2192207621 learned_addr learned my addr 172.21.15.123:0/2192207621 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a12a00640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a140c6750 0x7f2a1414e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.478+0000 7f2a13e00640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1415fce0 0x7f2a1415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a13400640 1 -- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a140c6750 msgr2=0x7f2a1414e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a13400640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a140c6750 0x7f2a1414e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a13400640 1 -- 172.21.15.123:0/2192207621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1415fce0 msgr2=0x7f2a1415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a13400640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1415fce0 0x7f2a1415ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a13400640 1 -- 172.21.15.123:0/2192207621 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a141e8620 con 0x7f2a1406e0c0 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a13400640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a1406e0c0 0x7f2a1414e340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400c430 tx=0x7f2a0400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:56.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f3137e00640 1 --2- 172.21.15.123:0/3287359933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3138156670 0x7f31380c79d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3130009dc0 tx=0x7f313002a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a254e667de74441e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:56.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a0b400640 1 -- 172.21.15.123:0/2192207621 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a040074e0 con 0x7f2a1406e0c0 2024-04-25T12:36:56.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a0b400640 1 -- 172.21.15.123:0/2192207621 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a040047e0 con 0x7f2a1406e0c0 2024-04-25T12:36:56.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.479+0000 7f2a0b400640 1 -- 172.21.15.123:0/2192207621 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04004bd0 con 0x7f2a1406e0c0 2024-04-25T12:36:56.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a14154a80 con 0x7f2a1406e0c0 2024-04-25T12:36:56.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f3136000640 1 -- 172.21.15.123:0/3287359933 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130007740 con 0x7f3138156670 2024-04-25T12:36:56.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f3136000640 1 -- 172.21.15.123:0/3287359933 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31300078e0 con 0x7f3138156670 2024-04-25T12:36:56.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f3136000640 1 -- 172.21.15.123:0/3287359933 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130007cb0 con 0x7f3138156670 2024-04-25T12:36:56.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a140cf150 con 0x7f2a1406e0c0 2024-04-25T12:36:56.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f313e600640 1 -- 172.21.15.123:0/3287359933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3138156670 msgr2=0x7f31380c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f313e600640 1 --2- 172.21.15.123:0/3287359933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3138156670 0x7f31380c79d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3130009dc0 tx=0x7f313002a760 comp rx=0 tx=0).stop 2024-04-25T12:36:56.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f313e600640 1 -- 172.21.15.123:0/3287359933 shutdown_connections 2024-04-25T12:36:56.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.480+0000 7f313e600640 1 --2- 172.21.15.123:0/3287359933 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3138156670 0x7f31380c79d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.481+0000 7f313e600640 1 -- 172.21.15.123:0/3287359933 >> 172.21.15.123:0/3287359933 conn(0x7f3138068a50 msgr2=0x7f31380d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:56.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.481+0000 7f313e600640 1 -- 172.21.15.123:0/3287359933 shutdown_connections 2024-04-25T12:36:56.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.481+0000 7f313e600640 1 -- 172.21.15.123:0/3287359933 wait complete. 2024-04-25T12:36:56.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.481+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a14076700 con 0x7f2a1406e0c0 2024-04-25T12:36:56.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.482+0000 7f2a0b400640 1 -- 172.21.15.123:0/2192207621 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a04011070 con 0x7f2a1406e0c0 2024-04-25T12:36:56.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.483+0000 7f313e600640 1 Processor -- start 2024-04-25T12:36:56.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.483+0000 7f2a0b400640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29f40744b0 0x7f29f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.483+0000 7f313e600640 1 -- start start 2024-04-25T12:36:56.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.483+0000 7f2a0b400640 1 -- 172.21.15.123:0/2192207621 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 6685+0+0 (secure 0 0 0) 0x7f2a040951f0 con 0x7f2a1406e0c0 2024-04-25T12:36:56.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.483+0000 7f2a12a00640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29f40744b0 0x7f29f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.484+0000 7f2a12a00640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29f40744b0 0x7f29f40769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a141f3770 tx=0x7f2a00009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:56.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f313e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806d440 0x7f3138150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f313e600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3138156670 0x7f313815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f313e600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31381606e0 0x7f313814b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f313e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f313815fee0 con 0x7f313806d440 2024-04-25T12:36:56.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f313e600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31380cccc0 con 0x7f3138156670 2024-04-25T12:36:56.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f313e600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f31381601c0 con 0x7f31381606e0 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3137400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806d440 0x7f3138150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3138156670 0x7f313815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3137e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31381606e0 0x7f313814b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3138156670 0x7f313815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:57366/0 (socket says 172.21.15.123:57366) 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 -- 172.21.15.123:0/3623363866 learned_addr learned my addr 172.21.15.123:0/3623363866 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 -- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31381606e0 msgr2=0x7f313814b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31381606e0 0x7f313814b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 -- 172.21.15.123:0/3623363866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806d440 msgr2=0x7f3138150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806d440 0x7f3138150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3136a00640 1 -- 172.21.15.123:0/3623363866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31381510b0 con 0x7f3138156670 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.485+0000 7f3137400640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806d440 0x7f3138150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.486+0000 7f3136a00640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3138156670 0x7f313815ece0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f312400c430 tx=0x7f312400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.486+0000 7f312f400640 1 -- 172.21.15.123:0/3623363866 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124017070 con 0x7f3138156670 2024-04-25T12:36:56.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.486+0000 7f312f400640 1 -- 172.21.15.123:0/3623363866 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31240052f0 con 0x7f3138156670 2024-04-25T12:36:56.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.486+0000 7f312f400640 1 -- 172.21.15.123:0/3623363866 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124003830 con 0x7f3138156670 2024-04-25T12:36:56.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.486+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3138151370 con 0x7f3138156670 2024-04-25T12:36:56.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.486+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31381f5da0 con 0x7f3138156670 2024-04-25T12:36:56.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.487+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3138076700 con 0x7f3138156670 2024-04-25T12:36:56.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.488+0000 7f2a0b400640 1 -- 172.21.15.123:0/2192207621 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a0405eed0 con 0x7f2a1406e0c0 2024-04-25T12:36:56.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.488+0000 7f312f400640 1 -- 172.21.15.123:0/3623363866 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f312401a070 con 0x7f3138156670 2024-04-25T12:36:56.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.489+0000 7f312f400640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31180744e0 0x7f31180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:56.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.489+0000 7f312f400640 1 -- 172.21.15.123:0/3623363866 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6685+0+0 (secure 0 0 0) 0x7f3124094aa0 con 0x7f3138156670 2024-04-25T12:36:56.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.490+0000 7f3137400640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31180744e0 0x7f31180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:56.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.490+0000 7f3137400640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31180744e0 0x7f31180769f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3128000f60 tx=0x7f3128009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:56.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.494+0000 7f312f400640 1 -- 172.21.15.123:0/3623363866 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f312405e6f0 con 0x7f3138156670 2024-04-25T12:36:56.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.688+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f2a1415a110 con 0x7f2a1406e0c0 2024-04-25T12:36:56.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:56.708+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7f31380cbf40 con 0x7f3138156670 2024-04-25T12:36:57.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.153+0000 7f312f400640 1 -- 172.21.15.123:0/3623363866 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v74) ==== 113+0+0 (secure 0 0 0) 0x7f31240629f0 con 0x7f3138156670 2024-04-25T12:36:57.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.154+0000 7f2a0b400640 1 -- 172.21.15.123:0/2192207621 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v74) ==== 101+0+0 (secure 0 0 0) 0x7f2a040631d0 con 0x7f2a1406e0c0 2024-04-25T12:36:57.154 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.5 to 1 (10000) 2024-04-25T12:36:57.155 INFO:teuthology.orchestra.run.smithi123.stderr:nodeep-scrub is unset 2024-04-25T12:36:57.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.157+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29f40744b0 msgr2=0x7f29f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.157+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29f40744b0 0x7f29f40769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a141f3770 tx=0x7f2a00009190 comp rx=0 tx=0).stop 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.157+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a1406e0c0 msgr2=0x7f2a1414e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.157+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a1406e0c0 0x7f2a1414e340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400c430 tx=0x7f2a0400c8f0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 shutdown_connections 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f29f40744b0 0x7f29f40769c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a1415fce0 0x7f2a1415ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a140c6750 0x7f2a1414e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 --2- 172.21.15.123:0/2192207621 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2a1406e0c0 0x7f2a1414e340 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 >> 172.21.15.123:0/2192207621 conn(0x7f2a14068600 msgr2=0x7f2a14075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 shutdown_connections 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31180744e0 msgr2=0x7f31180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31180744e0 0x7f31180769f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3128000f60 tx=0x7f3128009040 comp rx=0 tx=0).stop 2024-04-25T12:36:57.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f2a1a600640 1 -- 172.21.15.123:0/2192207621 wait complete. 2024-04-25T12:36:57.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3138156670 msgr2=0x7f313815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3138156670 0x7f313815ece0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f312400c430 tx=0x7f312400c8f0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 shutdown_connections 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31180744e0 0x7f31180769f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31381606e0 0x7f313814b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3138156670 0x7f313815ece0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 --2- 172.21.15.123:0/3623363866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806d440 0x7f3138150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 >> 172.21.15.123:0/3623363866 conn(0x7f3138068a50 msgr2=0x7f31380b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 shutdown_connections 2024-04-25T12:36:57.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.158+0000 7f313e600640 1 -- 172.21.15.123:0/3623363866 wait complete. 2024-04-25T12:36:57.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-25T12:36:57.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.513+0000 7f769b200640 1 Processor -- start 2024-04-25T12:36:57.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.513+0000 7f769b200640 1 -- start start 2024-04-25T12:36:57.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.515+0000 7f769b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0c7e30 0x7f769c154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:57.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.515+0000 7f769b200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f769c072cc0 0x7f769c06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:57.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.515+0000 7f769b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769c074290 con 0x7f769c0730d0 2024-04-25T12:36:57.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.515+0000 7f769b200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f769c074570 con 0x7f769c072cc0 2024-04-25T12:36:57.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.515+0000 7f769b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f769c074850 con 0x7f769c0c7e30 2024-04-25T12:36:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f769a800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f769c0730d0 0x7f769c15dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55520/0 (socket says 172.21.15.123:55520) 2024-04-25T12:36:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f769a800640 1 -- 172.21.15.123:0/1518825770 learned_addr learned my addr 172.21.15.123:0/1518825770 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f7699e00640 1 --2- 172.21.15.123:0/1518825770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0c7e30 0x7f769c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f7691400640 1 --2- 172.21.15.123:0/1518825770 >> v2:172.21.15.153:3300/0 conn(0x7f769c072cc0 0x7f769c06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f7699e00640 1 -- 172.21.15.123:0/1518825770 >> v2:172.21.15.153:3300/0 conn(0x7f769c072cc0 msgr2=0x7f769c06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f7699e00640 1 --2- 172.21.15.123:0/1518825770 >> v2:172.21.15.153:3300/0 conn(0x7f769c072cc0 0x7f769c06c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f7699400640 1 -- 172.21.15.123:0/1518825770 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 110443915 0 0) 0x7f769c074290 con 0x7f769c0730d0 2024-04-25T12:36:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f7699e00640 1 -- 172.21.15.123:0/1518825770 >> v1:172.21.15.123:6789/0 conn(0x7f769c0730d0 legacy=0x7f769c15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.516+0000 7f7699e00640 1 -- 172.21.15.123:0/1518825770 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c074b30 con 0x7f769c0c7e30 2024-04-25T12:36:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.517+0000 7f7699e00640 1 --2- 172.21.15.123:0/1518825770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0c7e30 0x7f769c154280 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097a0 tx=0x7f768c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45eab97b52c666d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:57.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.517+0000 7f7699400640 1 -- 172.21.15.123:0/1518825770 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c037070 con 0x7f769c0c7e30 2024-04-25T12:36:57.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.517+0000 7f7699400640 1 -- 172.21.15.123:0/1518825770 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c032440 con 0x7f769c0c7e30 2024-04-25T12:36:57.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.517+0000 7f7699400640 1 -- 172.21.15.123:0/1518825770 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c032810 con 0x7f769c0c7e30 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 -- 172.21.15.123:0/1518825770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0c7e30 msgr2=0x7f769c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 --2- 172.21.15.123:0/1518825770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0c7e30 0x7f769c154280 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097a0 tx=0x7f768c02a180 comp rx=0 tx=0).stop 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 -- 172.21.15.123:0/1518825770 shutdown_connections 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 --2- 172.21.15.123:0/1518825770 >> v2:172.21.15.153:3300/0 conn(0x7f769c072cc0 0x7f769c06c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 --2- 172.21.15.123:0/1518825770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0c7e30 0x7f769c154280 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 -- 172.21.15.123:0/1518825770 >> 172.21.15.123:0/1518825770 conn(0x7f769c070b70 msgr2=0x7f769c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 -- 172.21.15.123:0/1518825770 shutdown_connections 2024-04-25T12:36:57.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.518+0000 7f769b200640 1 -- 172.21.15.123:0/1518825770 wait complete. 2024-04-25T12:36:57.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.521+0000 7f769b200640 1 Processor -- start 2024-04-25T12:36:57.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.521+0000 7f769b200640 1 -- start start 2024-04-25T12:36:57.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f769b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f769c072cc0 0x7f769c1e5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:57.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f769b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0730d0 0x7f769c14d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:57.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f769b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f769c0c7e30 0x7f769c14dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:57.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f769b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f769c0c8240 con 0x7f769c0c7e30 2024-04-25T12:36:57.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f769b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f769c1547c0 con 0x7f769c072cc0 2024-04-25T12:36:57.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f769b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f769c0c83c0 con 0x7f769c0730d0 2024-04-25T12:36:57.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f7691400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0730d0 0x7f769c14d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:57.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.522+0000 7f7699e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f769c072cc0 0x7f769c1e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:57.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7691400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0730d0 0x7f769c14d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:39876/0 (socket says 172.21.15.123:39876) 2024-04-25T12:36:57.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f769a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f769c0c7e30 0x7f769c14dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:57.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7699e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f769c072cc0 0x7f769c1e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:57380/0 (socket says 172.21.15.123:57380) 2024-04-25T12:36:57.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7691400640 1 -- 172.21.15.123:0/2216532694 learned_addr learned my addr 172.21.15.123:0/2216532694 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7699e00640 1 -- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0730d0 msgr2=0x7f769c14d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7699e00640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0730d0 0x7f769c14d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7699e00640 1 -- 172.21.15.123:0/2216532694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f769c0c7e30 msgr2=0x7f769c14dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7699e00640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f769c0c7e30 0x7f769c14dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7699e00640 1 -- 172.21.15.123:0/2216532694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c1e5660 con 0x7f769c072cc0 2024-04-25T12:36:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7699e00640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f769c072cc0 0x7f769c1e5120 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f768c031000 tx=0x7f768c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:57.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.523+0000 7f7692a00640 1 -- 172.21.15.123:0/2216532694 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c004220 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.524+0000 7f7692a00640 1 -- 172.21.15.123:0/2216532694 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c0043c0 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.524+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c14b9c0 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.524+0000 7f7692a00640 1 -- 172.21.15.123:0/2216532694 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c03aa70 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.524+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c1e3eb0 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.525+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c076da0 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.526+0000 7f7692a00640 1 -- 172.21.15.123:0/2216532694 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f768c03ac10 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.527+0000 7f7692a00640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76780744b0 0x7f76780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.527+0000 7f7692a00640 1 -- 172.21.15.123:0/2216532694 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6685+0+0 (secure 0 0 0) 0x7f768c0b9ac0 con 0x7f769c072cc0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.527+0000 7f7691400640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76780744b0 0x7f76780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:57.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.528+0000 7f7691400640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76780744b0 0x7f76780769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f769c1f1010 tx=0x7f7684009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:57.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.532+0000 7f7692a00640 1 -- 172.21.15.123:0/2216532694 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f768c040070 con 0x7f769c072cc0 2024-04-25T12:36:57.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:57.743+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7f769c073690 con 0x7f769c072cc0 2024-04-25T12:36:58.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.157+0000 7f7692a00640 1 -- 172.21.15.123:0/2216532694 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v75) ==== 113+0+0 (secure 0 0 0) 0x7f768c083790 con 0x7f769c072cc0 2024-04-25T12:36:58.157 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.7 to 1 (10000) 2024-04-25T12:36:58.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.160+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76780744b0 msgr2=0x7f76780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.160+0000 7f769b200640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76780744b0 0x7f76780769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f769c1f1010 tx=0x7f7684009190 comp rx=0 tx=0).stop 2024-04-25T12:36:58.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.160+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f769c072cc0 msgr2=0x7f769c1e5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.160+0000 7f769b200640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f769c072cc0 0x7f769c1e5120 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f768c031000 tx=0x7f768c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 shutdown_connections 2024-04-25T12:36:58.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76780744b0 0x7f76780769c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f769c0c7e30 0x7f769c14dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f769c0730d0 0x7f769c14d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 --2- 172.21.15.123:0/2216532694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f769c072cc0 0x7f769c1e5120 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 >> 172.21.15.123:0/2216532694 conn(0x7f769c070b70 msgr2=0x7f769c153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:58.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 shutdown_connections 2024-04-25T12:36:58.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.161+0000 7f769b200640 1 -- 172.21.15.123:0/2216532694 wait complete. 2024-04-25T12:36:58.190 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-25T12:36:58.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:36:58.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.461+0000 7f53fba00640 1 Processor -- start 2024-04-25T12:36:58.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.462+0000 7f53fba00640 1 -- start start 2024-04-25T12:36:58.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53fba00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53f40c4d50 0x7f53f4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53fba00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f53f4074870 0x7f53f415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53fba00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f4073ce0 con 0x7f53f40c8c60 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53fba00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f53f40c61f0 con 0x7f53f40c4d50 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53fba00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f53f40c87d0 con 0x7f53f4074870 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53f9000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53f40c4d50 0x7f53f4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53f9a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f53f40c8c60 0x7f53f40cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55522/0 (socket says 172.21.15.123:55522) 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53f9a00640 1 -- 172.21.15.123:0/1467454410 learned_addr learned my addr 172.21.15.123:0/1467454410 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.464+0000 7f53f3e00640 1 --2- 172.21.15.123:0/1467454410 >> v2:172.21.15.153:3300/0 conn(0x7f53f4074870 0x7f53f415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.465+0000 7f53f9000640 1 -- 172.21.15.123:0/1467454410 >> v2:172.21.15.153:3300/0 conn(0x7f53f4074870 msgr2=0x7f53f415afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.465+0000 7f53f9000640 1 --2- 172.21.15.123:0/1467454410 >> v2:172.21.15.153:3300/0 conn(0x7f53f4074870 0x7f53f415afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.465+0000 7f53f9000640 1 -- 172.21.15.123:0/1467454410 >> v1:172.21.15.123:6789/0 conn(0x7f53f40c8c60 legacy=0x7f53f40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.465+0000 7f53f3400640 1 -- 172.21.15.123:0/1467454410 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347174685 0 0) 0x7f53f4073ce0 con 0x7f53f40c8c60 2024-04-25T12:36:58.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.465+0000 7f53f9000640 1 -- 172.21.15.123:0/1467454410 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f4073fc0 con 0x7f53f40c4d50 2024-04-25T12:36:58.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.466+0000 7f53f9000640 1 --2- 172.21.15.123:0/1467454410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f40c4d50 0x7f53f4154890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53f4072a70 tx=0x7f53e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247056c884e25822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:58.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.466+0000 7f53f3400640 1 -- 172.21.15.123:0/1467454410 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4037070 con 0x7f53f40c4d50 2024-04-25T12:36:58.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.466+0000 7f53f3400640 1 -- 172.21.15.123:0/1467454410 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e4032440 con 0x7f53f40c4d50 2024-04-25T12:36:58.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.466+0000 7f53f3400640 1 -- 172.21.15.123:0/1467454410 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4032810 con 0x7f53f40c4d50 2024-04-25T12:36:58.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 -- 172.21.15.123:0/1467454410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f40c4d50 msgr2=0x7f53f4154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 --2- 172.21.15.123:0/1467454410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f40c4d50 0x7f53f4154890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53f4072a70 tx=0x7f53e402a230 comp rx=0 tx=0).stop 2024-04-25T12:36:58.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 -- 172.21.15.123:0/1467454410 shutdown_connections 2024-04-25T12:36:58.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 --2- 172.21.15.123:0/1467454410 >> v2:172.21.15.153:3300/0 conn(0x7f53f4074870 0x7f53f415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 --2- 172.21.15.123:0/1467454410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f40c4d50 0x7f53f4154890 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 -- 172.21.15.123:0/1467454410 >> 172.21.15.123:0/1467454410 conn(0x7f53f406c750 msgr2=0x7f53f406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 -- 172.21.15.123:0/1467454410 shutdown_connections 2024-04-25T12:36:58.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.467+0000 7f53fba00640 1 -- 172.21.15.123:0/1467454410 wait complete. 2024-04-25T12:36:58.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.469+0000 7f53fba00640 1 Processor -- start 2024-04-25T12:36:58.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.469+0000 7f53fba00640 1 -- start start 2024-04-25T12:36:58.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53fba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f4074870 0x7f53f4149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53fba00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53f40c4d50 0x7f53f414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53fba00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f40c8c60 0x7f53f4144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53fba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f40ce1a0 con 0x7f53f4074870 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53fba00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53f40c8460 con 0x7f53f40c8c60 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53fba00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f53f40ce320 con 0x7f53f40c4d50 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53f9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f4074870 0x7f53f4149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53f9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f4074870 0x7f53f4149090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44458/0 (socket says 172.21.15.123:44458) 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53f9000640 1 -- 172.21.15.123:0/3213807602 learned_addr learned my addr 172.21.15.123:0/3213807602 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.471+0000 7f53f3e00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53f40c4d50 0x7f53f414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.472+0000 7f53f9a00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f40c8c60 0x7f53f4144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.472+0000 7f53f9000640 1 -- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53f40c4d50 msgr2=0x7f53f414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.472+0000 7f53f9000640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53f40c4d50 0x7f53f414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.472+0000 7f53f9000640 1 -- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f40c8c60 msgr2=0x7f53f4144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.472+0000 7f53f9000640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f40c8c60 0x7f53f4144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.472+0000 7f53f9000640 1 -- 172.21.15.123:0/3213807602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f41495d0 con 0x7f53f4074870 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.472+0000 7f53f9000640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f4074870 0x7f53f4149090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009890 tx=0x7f53e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.473+0000 7f53f0c00640 1 -- 172.21.15.123:0/3213807602 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4037040 con 0x7f53f4074870 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.473+0000 7f53f0c00640 1 -- 172.21.15.123:0/3213807602 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e4042070 con 0x7f53f4074870 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.473+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f414c670 con 0x7f53f4074870 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.473+0000 7f53f0c00640 1 -- 172.21.15.123:0/3213807602 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4030920 con 0x7f53f4074870 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.473+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f414cd90 con 0x7f53f4074870 2024-04-25T12:36:58.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.474+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f4076700 con 0x7f53f4074870 2024-04-25T12:36:58.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.475+0000 7f53f0c00640 1 -- 172.21.15.123:0/3213807602 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f53e4030ac0 con 0x7f53f4074870 2024-04-25T12:36:58.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.476+0000 7f53f0c00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53d40744b0 0x7f53d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:58.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.476+0000 7f53f0c00640 1 -- 172.21.15.123:0/3213807602 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6685+0+0 (secure 0 0 0) 0x7f53e40b5d20 con 0x7f53f4074870 2024-04-25T12:36:58.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.477+0000 7f53f3e00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53d40744b0 0x7f53d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:58.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.477+0000 7f53f3e00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53d40744b0 0x7f53d40769c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f53f414aa10 tx=0x7f53e000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:58.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.481+0000 7f53f0c00640 1 -- 172.21.15.123:0/3213807602 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53e404c020 con 0x7f53f4074870 2024-04-25T12:36:58.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.677+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53f40b3610 con 0x7f53d40744b0 2024-04-25T12:36:58.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.688+0000 7f53f0c00640 1 -- 172.21.15.123:0/3213807602 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333057 (secure 0 0 0) 0x7f53f40b3610 con 0x7f53d40744b0 2024-04-25T12:36:58.693 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:36:58.696 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:36:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53d40744b0 msgr2=0x7f53d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53d40744b0 0x7f53d40769c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f53f414aa10 tx=0x7f53e000b040 comp rx=0 tx=0).stop 2024-04-25T12:36:58.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f4074870 msgr2=0x7f53f4149090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f4074870 0x7f53f4149090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009890 tx=0x7f53e40046c0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 shutdown_connections 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f53d40744b0 0x7f53d40769c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f40c8c60 0x7f53f4144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53f40c4d50 0x7f53f414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 --2- 172.21.15.123:0/3213807602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f4074870 0x7f53f4149090 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 >> 172.21.15.123:0/3213807602 conn(0x7f53f406c750 msgr2=0x7f53f4071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 shutdown_connections 2024-04-25T12:36:58.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:58.695+0000 7f53fba00640 1 -- 172.21.15.123:0/3213807602 wait complete. 2024-04-25T12:36:58.722 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-25T12:36:58.155831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":627931,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":67,"num_read_kb":56,"num_write":262,"num_write_kb":1718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":591572,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":200580,"kb_used_data":6036,"kb_used_omap":18,"kb_used_meta":194413,"kb_avail":660402300,"statfs":{"total":676457349120,"available":676251955200,"internally_reserved":0,"allocated":6180864,"data_stored":4254739,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":18931,"internal_metadata":199079437},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8374,"num_objects":0,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039506"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":125,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121732+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:36:55.121732+0000","last_peered":"2024-04-25T12:36:55.121732+0000","last_clean":"2024-04-25T12:36:55.121732+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:36:55.121732+0000","last_undegraded":"2024-04-25T12:36:55.121732+0000","last_fullsized":"2024-04-25T12:36:55.121732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168709+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:36:53.168709+0000","last_peered":"2024-04-25T12:36:53.168709+0000","last_clean":"2024-04-25T12:36:53.168709+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:36:53.168709+0000","last_undegraded":"2024-04-25T12:36:53.168709+0000","last_fullsized":"2024-04-25T12:36:53.168709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":118,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.459883+0000","last_change":"2024-04-25T12:36:47.042028+0000","last_active":"2024-04-25T12:36:56.459883+0000","last_peered":"2024-04-25T12:36:56.459883+0000","last_clean":"2024-04-25T12:36:56.459883+0000","last_became_active":"2024-04-25T12:36:47.041474+0000","last_became_peered":"2024-04-25T12:36:47.041474+0000","last_unstale":"2024-04-25T12:36:56.459883+0000","last_undegraded":"2024-04-25T12:36:56.459883+0000","last_fullsized":"2024-04-25T12:36:56.459883+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:09.865004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,1],"acting":[7,5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":80,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168401+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:36:53.168401+0000","last_peered":"2024-04-25T12:36:53.168401+0000","last_clean":"2024-04-25T12:36:53.168401+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:36:53.168401+0000","last_undegraded":"2024-04-25T12:36:53.168401+0000","last_fullsized":"2024-04-25T12:36:53.168401+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168653+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:36:53.168653+0000","last_peered":"2024-04-25T12:36:53.168653+0000","last_clean":"2024-04-25T12:36:53.168653+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:36:53.168653+0000","last_undegraded":"2024-04-25T12:36:53.168653+0000","last_fullsized":"2024-04-25T12:36:53.168653+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146217+0000","last_change":"2024-04-25T12:36:47.048696+0000","last_active":"2024-04-25T12:36:56.146217+0000","last_peered":"2024-04-25T12:36:56.146217+0000","last_clean":"2024-04-25T12:36:56.146217+0000","last_became_active":"2024-04-25T12:36:47.048209+0000","last_became_peered":"2024-04-25T12:36:47.048209+0000","last_unstale":"2024-04-25T12:36:56.146217+0000","last_undegraded":"2024-04-25T12:36:56.146217+0000","last_fullsized":"2024-04-25T12:36:56.146217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:06.823061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,1],"acting":[2,7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":123,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164994+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:36:57.164994+0000","last_peered":"2024-04-25T12:36:57.164994+0000","last_clean":"2024-04-25T12:36:57.164994+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:36:57.164994+0000","last_undegraded":"2024-04-25T12:36:57.164994+0000","last_fullsized":"2024-04-25T12:36:57.164994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147838+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:36:56.147838+0000","last_peered":"2024-04-25T12:36:56.147838+0000","last_clean":"2024-04-25T12:36:56.147838+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:36:56.147838+0000","last_undegraded":"2024-04-25T12:36:56.147838+0000","last_fullsized":"2024-04-25T12:36:56.147838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":128,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146846+0000","last_change":"2024-04-25T12:36:47.060381+0000","last_active":"2024-04-25T12:36:56.146846+0000","last_peered":"2024-04-25T12:36:56.146846+0000","last_clean":"2024-04-25T12:36:56.146846+0000","last_became_active":"2024-04-25T12:36:47.059933+0000","last_became_peered":"2024-04-25T12:36:47.059933+0000","last_unstale":"2024-04-25T12:36:56.146846+0000","last_undegraded":"2024-04-25T12:36:56.146846+0000","last_fullsized":"2024-04-25T12:36:56.146846+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:42.613333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,0],"acting":[2,5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":120,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165373+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:36:57.165373+0000","last_peered":"2024-04-25T12:36:57.165373+0000","last_clean":"2024-04-25T12:36:57.165373+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:36:57.165373+0000","last_undegraded":"2024-04-25T12:36:57.165373+0000","last_fullsized":"2024-04-25T12:36:57.165373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350833+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:36:57.350833+0000","last_peered":"2024-04-25T12:36:57.350833+0000","last_clean":"2024-04-25T12:36:57.350833+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:36:57.350833+0000","last_undegraded":"2024-04-25T12:36:57.350833+0000","last_fullsized":"2024-04-25T12:36:57.350833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":128,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146424+0000","last_change":"2024-04-25T12:36:47.061434+0000","last_active":"2024-04-25T12:36:56.146424+0000","last_peered":"2024-04-25T12:36:56.146424+0000","last_clean":"2024-04-25T12:36:56.146424+0000","last_became_active":"2024-04-25T12:36:47.060986+0000","last_became_peered":"2024-04-25T12:36:47.060986+0000","last_unstale":"2024-04-25T12:36:56.146424+0000","last_undegraded":"2024-04-25T12:36:56.146424+0000","last_fullsized":"2024-04-25T12:36:56.146424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:54.714311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,7],"acting":[2,0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351025+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:36:57.351025+0000","last_peered":"2024-04-25T12:36:57.351025+0000","last_clean":"2024-04-25T12:36:57.351025+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:36:57.351025+0000","last_undegraded":"2024-04-25T12:36:57.351025+0000","last_fullsized":"2024-04-25T12:36:57.351025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350562+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:36:57.350562+0000","last_peered":"2024-04-25T12:36:57.350562+0000","last_clean":"2024-04-25T12:36:57.350562+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:36:57.350562+0000","last_undegraded":"2024-04-25T12:36:57.350562+0000","last_fullsized":"2024-04-25T12:36:57.350562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163514+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:36:57.163514+0000","last_peered":"2024-04-25T12:36:57.163514+0000","last_clean":"2024-04-25T12:36:57.163514+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:36:57.163514+0000","last_undegraded":"2024-04-25T12:36:57.163514+0000","last_fullsized":"2024-04-25T12:36:57.163514+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":121,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460155+0000","last_change":"2024-04-25T12:36:47.046901+0000","last_active":"2024-04-25T12:36:56.460155+0000","last_peered":"2024-04-25T12:36:56.460155+0000","last_clean":"2024-04-25T12:36:56.460155+0000","last_became_active":"2024-04-25T12:36:47.046621+0000","last_became_peered":"2024-04-25T12:36:47.046621+0000","last_unstale":"2024-04-25T12:36:56.460155+0000","last_undegraded":"2024-04-25T12:36:56.460155+0000","last_fullsized":"2024-04-25T12:36:56.460155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:30.345307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.167991+0000","last_change":"2024-04-25T12:36:47.066385+0000","last_active":"2024-04-25T12:36:53.167991+0000","last_peered":"2024-04-25T12:36:53.167991+0000","last_clean":"2024-04-25T12:36:53.167991+0000","last_became_active":"2024-04-25T12:36:47.046898+0000","last_became_peered":"2024-04-25T12:36:47.046898+0000","last_unstale":"2024-04-25T12:36:53.167991+0000","last_undegraded":"2024-04-25T12:36:53.167991+0000","last_fullsized":"2024-04-25T12:36:53.167991+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.673850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":118,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164308+0000","last_change":"2024-04-25T12:36:47.051974+0000","last_active":"2024-04-25T12:36:57.164308+0000","last_peered":"2024-04-25T12:36:57.164308+0000","last_clean":"2024-04-25T12:36:57.164308+0000","last_became_active":"2024-04-25T12:36:47.051537+0000","last_became_peered":"2024-04-25T12:36:47.051537+0000","last_unstale":"2024-04-25T12:36:57.164308+0000","last_undegraded":"2024-04-25T12:36:57.164308+0000","last_fullsized":"2024-04-25T12:36:57.164308+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.244233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.120969+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:36:55.120969+0000","last_peered":"2024-04-25T12:36:55.120969+0000","last_clean":"2024-04-25T12:36:55.120969+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:36:55.120969+0000","last_undegraded":"2024-04-25T12:36:55.120969+0000","last_fullsized":"2024-04-25T12:36:55.120969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":119,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121559+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:36:55.121559+0000","last_peered":"2024-04-25T12:36:55.121559+0000","last_clean":"2024-04-25T12:36:55.121559+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:36:55.121559+0000","last_undegraded":"2024-04-25T12:36:55.121559+0000","last_fullsized":"2024-04-25T12:36:55.121559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":111,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460524+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:36:56.460524+0000","last_peered":"2024-04-25T12:36:56.460524+0000","last_clean":"2024-04-25T12:36:56.460524+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:36:56.460524+0000","last_undegraded":"2024-04-25T12:36:56.460524+0000","last_fullsized":"2024-04-25T12:36:56.460524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349878+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:36:57.349878+0000","last_peered":"2024-04-25T12:36:57.349878+0000","last_clean":"2024-04-25T12:36:57.349878+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:36:57.349878+0000","last_undegraded":"2024-04-25T12:36:57.349878+0000","last_fullsized":"2024-04-25T12:36:57.349878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.348869+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:36:57.348869+0000","last_peered":"2024-04-25T12:36:57.348869+0000","last_clean":"2024-04-25T12:36:57.348869+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:36:57.348869+0000","last_undegraded":"2024-04-25T12:36:57.348869+0000","last_fullsized":"2024-04-25T12:36:57.348869+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":110,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148228+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:36:56.148228+0000","last_peered":"2024-04-25T12:36:56.148228+0000","last_clean":"2024-04-25T12:36:56.148228+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:36:56.148228+0000","last_undegraded":"2024-04-25T12:36:56.148228+0000","last_fullsized":"2024-04-25T12:36:56.148228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121355+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:36:55.121355+0000","last_peered":"2024-04-25T12:36:55.121355+0000","last_clean":"2024-04-25T12:36:55.121355+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:36:55.121355+0000","last_undegraded":"2024-04-25T12:36:55.121355+0000","last_fullsized":"2024-04-25T12:36:55.121355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352342+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:36:57.352342+0000","last_peered":"2024-04-25T12:36:57.352342+0000","last_clean":"2024-04-25T12:36:57.352342+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:36:57.352342+0000","last_undegraded":"2024-04-25T12:36:57.352342+0000","last_fullsized":"2024-04-25T12:36:57.352342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146333+0000","last_change":"2024-04-25T12:36:47.058724+0000","last_active":"2024-04-25T12:36:56.146333+0000","last_peered":"2024-04-25T12:36:56.146333+0000","last_clean":"2024-04-25T12:36:56.146333+0000","last_became_active":"2024-04-25T12:36:47.058280+0000","last_became_peered":"2024-04-25T12:36:47.058280+0000","last_unstale":"2024-04-25T12:36:56.146333+0000","last_undegraded":"2024-04-25T12:36:56.146333+0000","last_fullsized":"2024-04-25T12:36:56.146333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:02.910553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,1],"acting":[2,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":119,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121725+0000","last_change":"2024-04-25T12:36:47.041012+0000","last_active":"2024-04-25T12:36:55.121725+0000","last_peered":"2024-04-25T12:36:55.121725+0000","last_clean":"2024-04-25T12:36:55.121725+0000","last_became_active":"2024-04-25T12:36:47.040684+0000","last_became_peered":"2024-04-25T12:36:47.040684+0000","last_unstale":"2024-04-25T12:36:55.121725+0000","last_undegraded":"2024-04-25T12:36:55.121725+0000","last_fullsized":"2024-04-25T12:36:55.121725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"70'11","reported_seq":127,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121558+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:36:55.121558+0000","last_peered":"2024-04-25T12:36:55.121558+0000","last_clean":"2024-04-25T12:36:55.121558+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:36:55.121558+0000","last_undegraded":"2024-04-25T12:36:55.121558+0000","last_fullsized":"2024-04-25T12:36:55.121558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.120937+0000","last_change":"2024-04-25T12:36:47.044258+0000","last_active":"2024-04-25T12:36:55.120937+0000","last_peered":"2024-04-25T12:36:55.120937+0000","last_clean":"2024-04-25T12:36:55.120937+0000","last_became_active":"2024-04-25T12:36:47.043910+0000","last_became_peered":"2024-04-25T12:36:47.043910+0000","last_unstale":"2024-04-25T12:36:55.120937+0000","last_undegraded":"2024-04-25T12:36:55.120937+0000","last_fullsized":"2024-04-25T12:36:55.120937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,3],"acting":[4,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":80,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168474+0000","last_change":"2024-04-25T12:36:47.043858+0000","last_active":"2024-04-25T12:36:53.168474+0000","last_peered":"2024-04-25T12:36:53.168474+0000","last_clean":"2024-04-25T12:36:53.168474+0000","last_became_active":"2024-04-25T12:36:47.043265+0000","last_became_peered":"2024-04-25T12:36:47.043265+0000","last_unstale":"2024-04-25T12:36:53.168474+0000","last_undegraded":"2024-04-25T12:36:53.168474+0000","last_fullsized":"2024-04-25T12:36:53.168474+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:31.461355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":117,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352198+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:36:57.352198+0000","last_peered":"2024-04-25T12:36:57.352198+0000","last_clean":"2024-04-25T12:36:57.352198+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:36:57.352198+0000","last_undegraded":"2024-04-25T12:36:57.352198+0000","last_fullsized":"2024-04-25T12:36:57.352198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168164+0000","last_change":"2024-04-25T12:36:47.047427+0000","last_active":"2024-04-25T12:36:53.168164+0000","last_peered":"2024-04-25T12:36:53.168164+0000","last_clean":"2024-04-25T12:36:53.168164+0000","last_became_active":"2024-04-25T12:36:47.046961+0000","last_became_peered":"2024-04-25T12:36:47.046961+0000","last_unstale":"2024-04-25T12:36:53.168164+0000","last_undegraded":"2024-04-25T12:36:53.168164+0000","last_fullsized":"2024-04-25T12:36:53.168164+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.321075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,7],"acting":[5,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148302+0000","last_change":"2024-04-25T12:36:47.043679+0000","last_active":"2024-04-25T12:36:56.148302+0000","last_peered":"2024-04-25T12:36:56.148302+0000","last_clean":"2024-04-25T12:36:56.148302+0000","last_became_active":"2024-04-25T12:36:47.043290+0000","last_became_peered":"2024-04-25T12:36:47.043290+0000","last_unstale":"2024-04-25T12:36:56.148302+0000","last_undegraded":"2024-04-25T12:36:56.148302+0000","last_fullsized":"2024-04-25T12:36:56.148302+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:56.140737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":69,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168239+0000","last_change":"2024-04-25T12:36:47.043694+0000","last_active":"2024-04-25T12:36:53.168239+0000","last_peered":"2024-04-25T12:36:53.168239+0000","last_clean":"2024-04-25T12:36:53.168239+0000","last_became_active":"2024-04-25T12:36:47.043238+0000","last_became_peered":"2024-04-25T12:36:47.043238+0000","last_unstale":"2024-04-25T12:36:53.168239+0000","last_undegraded":"2024-04-25T12:36:53.168239+0000","last_fullsized":"2024-04-25T12:36:53.168239+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:58.619827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121112+0000","last_change":"2024-04-25T12:36:47.048814+0000","last_active":"2024-04-25T12:36:55.121112+0000","last_peered":"2024-04-25T12:36:55.121112+0000","last_clean":"2024-04-25T12:36:55.121112+0000","last_became_active":"2024-04-25T12:36:47.048345+0000","last_became_peered":"2024-04-25T12:36:47.048345+0000","last_unstale":"2024-04-25T12:36:55.121112+0000","last_undegraded":"2024-04-25T12:36:55.121112+0000","last_fullsized":"2024-04-25T12:36:55.121112+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,7],"acting":[4,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164628+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:36:57.164628+0000","last_peered":"2024-04-25T12:36:57.164628+0000","last_clean":"2024-04-25T12:36:57.164628+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:36:57.164628+0000","last_undegraded":"2024-04-25T12:36:57.164628+0000","last_fullsized":"2024-04-25T12:36:57.164628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":122,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351311+0000","last_change":"2024-04-25T12:36:47.053780+0000","last_active":"2024-04-25T12:36:57.351311+0000","last_peered":"2024-04-25T12:36:57.351311+0000","last_clean":"2024-04-25T12:36:57.351311+0000","last_became_active":"2024-04-25T12:36:47.053245+0000","last_became_peered":"2024-04-25T12:36:47.053245+0000","last_unstale":"2024-04-25T12:36:57.351311+0000","last_undegraded":"2024-04-25T12:36:57.351311+0000","last_fullsized":"2024-04-25T12:36:57.351311+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:52.614962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147558+0000","last_change":"2024-04-25T12:36:47.055665+0000","last_active":"2024-04-25T12:36:56.147558+0000","last_peered":"2024-04-25T12:36:56.147558+0000","last_clean":"2024-04-25T12:36:56.147558+0000","last_became_active":"2024-04-25T12:36:47.055104+0000","last_became_peered":"2024-04-25T12:36:47.055104+0000","last_unstale":"2024-04-25T12:36:56.147558+0000","last_undegraded":"2024-04-25T12:36:56.147558+0000","last_fullsized":"2024-04-25T12:36:56.147558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0,3],"acting":[1,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.167909+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:36:53.167909+0000","last_peered":"2024-04-25T12:36:53.167909+0000","last_clean":"2024-04-25T12:36:53.167909+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:36:53.167909+0000","last_undegraded":"2024-04-25T12:36:53.167909+0000","last_fullsized":"2024-04-25T12:36:53.167909+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"70'6","reported_seq":132,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460030+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:36:56.460030+0000","last_peered":"2024-04-25T12:36:56.460030+0000","last_clean":"2024-04-25T12:36:56.460030+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:36:56.460030+0000","last_undegraded":"2024-04-25T12:36:56.460030+0000","last_fullsized":"2024-04-25T12:36:56.460030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352147+0000","last_change":"2024-04-25T12:36:47.048616+0000","last_active":"2024-04-25T12:36:57.352147+0000","last_peered":"2024-04-25T12:36:57.352147+0000","last_clean":"2024-04-25T12:36:57.352147+0000","last_became_active":"2024-04-25T12:36:47.048179+0000","last_became_peered":"2024-04-25T12:36:47.048179+0000","last_unstale":"2024-04-25T12:36:57.352147+0000","last_undegraded":"2024-04-25T12:36:57.352147+0000","last_fullsized":"2024-04-25T12:36:57.352147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:00.979187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,7],"acting":[3,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146342+0000","last_change":"2024-04-25T12:36:47.036913+0000","last_active":"2024-04-25T12:36:56.146342+0000","last_peered":"2024-04-25T12:36:56.146342+0000","last_clean":"2024-04-25T12:36:56.146342+0000","last_became_active":"2024-04-25T12:36:47.032924+0000","last_became_peered":"2024-04-25T12:36:47.032924+0000","last_unstale":"2024-04-25T12:36:56.146342+0000","last_undegraded":"2024-04-25T12:36:56.146342+0000","last_fullsized":"2024-04-25T12:36:56.146342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:05.790594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":126,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349785+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:36:57.349785+0000","last_peered":"2024-04-25T12:36:57.349785+0000","last_clean":"2024-04-25T12:36:57.349785+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:36:57.349785+0000","last_undegraded":"2024-04-25T12:36:57.349785+0000","last_fullsized":"2024-04-25T12:36:57.349785+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":75,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168866+0000","last_change":"2024-04-25T12:36:47.048722+0000","last_active":"2024-04-25T12:36:53.168866+0000","last_peered":"2024-04-25T12:36:53.168866+0000","last_clean":"2024-04-25T12:36:53.168866+0000","last_became_active":"2024-04-25T12:36:47.047551+0000","last_became_peered":"2024-04-25T12:36:47.047551+0000","last_unstale":"2024-04-25T12:36:53.168866+0000","last_undegraded":"2024-04-25T12:36:53.168866+0000","last_fullsized":"2024-04-25T12:36:53.168866+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:27.524785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0,1],"acting":[5,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121559+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:36:55.121559+0000","last_peered":"2024-04-25T12:36:55.121559+0000","last_clean":"2024-04-25T12:36:55.121559+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:36:55.121559+0000","last_undegraded":"2024-04-25T12:36:55.121559+0000","last_fullsized":"2024-04-25T12:36:55.121559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":119,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350590+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:36:57.350590+0000","last_peered":"2024-04-25T12:36:57.350590+0000","last_clean":"2024-04-25T12:36:57.350590+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:36:57.350590+0000","last_undegraded":"2024-04-25T12:36:57.350590+0000","last_fullsized":"2024-04-25T12:36:57.350590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":224,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.145925+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:36:56.145925+0000","last_peered":"2024-04-25T12:36:56.145925+0000","last_clean":"2024-04-25T12:36:56.145925+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:36:56.145925+0000","last_undegraded":"2024-04-25T12:36:56.145925+0000","last_fullsized":"2024-04-25T12:36:56.145925+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":110,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351272+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:36:57.351272+0000","last_peered":"2024-04-25T12:36:57.351272+0000","last_clean":"2024-04-25T12:36:57.351272+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:36:57.351272+0000","last_undegraded":"2024-04-25T12:36:57.351272+0000","last_fullsized":"2024-04-25T12:36:57.351272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351077+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:36:57.351077+0000","last_peered":"2024-04-25T12:36:57.351077+0000","last_clean":"2024-04-25T12:36:57.351077+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:36:57.351077+0000","last_undegraded":"2024-04-25T12:36:57.351077+0000","last_fullsized":"2024-04-25T12:36:57.351077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165273+0000","last_change":"2024-04-25T12:36:47.073082+0000","last_active":"2024-04-25T12:36:57.165273+0000","last_peered":"2024-04-25T12:36:57.165273+0000","last_clean":"2024-04-25T12:36:57.165273+0000","last_became_active":"2024-04-25T12:36:47.047092+0000","last_became_peered":"2024-04-25T12:36:47.047092+0000","last_unstale":"2024-04-25T12:36:57.165273+0000","last_undegraded":"2024-04-25T12:36:57.165273+0000","last_fullsized":"2024-04-25T12:36:57.165273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:06.488375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":110,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146510+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:36:56.146510+0000","last_peered":"2024-04-25T12:36:56.146510+0000","last_clean":"2024-04-25T12:36:56.146510+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:36:56.146510+0000","last_undegraded":"2024-04-25T12:36:56.146510+0000","last_fullsized":"2024-04-25T12:36:56.146510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165094+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:36:57.165094+0000","last_peered":"2024-04-25T12:36:57.165094+0000","last_clean":"2024-04-25T12:36:57.165094+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:36:57.165094+0000","last_undegraded":"2024-04-25T12:36:57.165094+0000","last_fullsized":"2024-04-25T12:36:57.165094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":77,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168592+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:36:53.168592+0000","last_peered":"2024-04-25T12:36:53.168592+0000","last_clean":"2024-04-25T12:36:53.168592+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:36:53.168592+0000","last_undegraded":"2024-04-25T12:36:53.168592+0000","last_fullsized":"2024-04-25T12:36:53.168592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":68,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168287+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:36:53.168287+0000","last_peered":"2024-04-25T12:36:53.168287+0000","last_clean":"2024-04-25T12:36:53.168287+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:36:53.168287+0000","last_undegraded":"2024-04-25T12:36:53.168287+0000","last_fullsized":"2024-04-25T12:36:53.168287+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164976+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:36:57.164976+0000","last_peered":"2024-04-25T12:36:57.164976+0000","last_clean":"2024-04-25T12:36:57.164976+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:36:57.164976+0000","last_undegraded":"2024-04-25T12:36:57.164976+0000","last_fullsized":"2024-04-25T12:36:57.164976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":125,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121442+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:36:55.121442+0000","last_peered":"2024-04-25T12:36:55.121442+0000","last_clean":"2024-04-25T12:36:55.121442+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:36:55.121442+0000","last_undegraded":"2024-04-25T12:36:55.121442+0000","last_fullsized":"2024-04-25T12:36:55.121442+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":110,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349113+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:36:57.349113+0000","last_peered":"2024-04-25T12:36:57.349113+0000","last_clean":"2024-04-25T12:36:57.349113+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:36:57.349113+0000","last_undegraded":"2024-04-25T12:36:57.349113+0000","last_fullsized":"2024-04-25T12:36:57.349113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148438+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:36:56.148438+0000","last_peered":"2024-04-25T12:36:56.148438+0000","last_clean":"2024-04-25T12:36:56.148438+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:36:56.148438+0000","last_undegraded":"2024-04-25T12:36:56.148438+0000","last_fullsized":"2024-04-25T12:36:56.148438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":127,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165188+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:36:57.165188+0000","last_peered":"2024-04-25T12:36:57.165188+0000","last_clean":"2024-04-25T12:36:57.165188+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:36:57.165188+0000","last_undegraded":"2024-04-25T12:36:57.165188+0000","last_fullsized":"2024-04-25T12:36:57.165188+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146611+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:36:56.146611+0000","last_peered":"2024-04-25T12:36:56.146611+0000","last_clean":"2024-04-25T12:36:56.146611+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:36:56.146611+0000","last_undegraded":"2024-04-25T12:36:56.146611+0000","last_fullsized":"2024-04-25T12:36:56.146611+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349560+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:36:57.349560+0000","last_peered":"2024-04-25T12:36:57.349560+0000","last_clean":"2024-04-25T12:36:57.349560+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:36:57.349560+0000","last_undegraded":"2024-04-25T12:36:57.349560+0000","last_fullsized":"2024-04-25T12:36:57.349560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":131,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121444+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:36:55.121444+0000","last_peered":"2024-04-25T12:36:55.121444+0000","last_clean":"2024-04-25T12:36:55.121444+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:36:55.121444+0000","last_undegraded":"2024-04-25T12:36:55.121444+0000","last_fullsized":"2024-04-25T12:36:55.121444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":75,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168794+0000","last_change":"2024-04-25T12:36:47.048991+0000","last_active":"2024-04-25T12:36:53.168794+0000","last_peered":"2024-04-25T12:36:53.168794+0000","last_clean":"2024-04-25T12:36:53.168794+0000","last_became_active":"2024-04-25T12:36:47.047195+0000","last_became_peered":"2024-04-25T12:36:47.047195+0000","last_unstale":"2024-04-25T12:36:53.168794+0000","last_undegraded":"2024-04-25T12:36:53.168794+0000","last_fullsized":"2024-04-25T12:36:53.168794+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:15.211552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0,1],"acting":[5,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.145745+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:36:56.145745+0000","last_peered":"2024-04-25T12:36:56.145745+0000","last_clean":"2024-04-25T12:36:56.145745+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:36:56.145745+0000","last_undegraded":"2024-04-25T12:36:56.145745+0000","last_fullsized":"2024-04-25T12:36:56.145745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163852+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:36:57.163852+0000","last_peered":"2024-04-25T12:36:57.163852+0000","last_clean":"2024-04-25T12:36:57.163852+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:36:57.163852+0000","last_undegraded":"2024-04-25T12:36:57.163852+0000","last_fullsized":"2024-04-25T12:36:57.163852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":75,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168723+0000","last_change":"2024-04-25T12:36:47.048302+0000","last_active":"2024-04-25T12:36:53.168723+0000","last_peered":"2024-04-25T12:36:53.168723+0000","last_clean":"2024-04-25T12:36:53.168723+0000","last_became_active":"2024-04-25T12:36:47.047884+0000","last_became_peered":"2024-04-25T12:36:47.047884+0000","last_unstale":"2024-04-25T12:36:53.168723+0000","last_undegraded":"2024-04-25T12:36:53.168723+0000","last_fullsized":"2024-04-25T12:36:53.168723+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:02.311326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,4],"acting":[5,1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350004+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:36:57.350004+0000","last_peered":"2024-04-25T12:36:57.350004+0000","last_clean":"2024-04-25T12:36:57.350004+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:36:57.350004+0000","last_undegraded":"2024-04-25T12:36:57.350004+0000","last_fullsized":"2024-04-25T12:36:57.350004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":120,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351096+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:36:57.351096+0000","last_peered":"2024-04-25T12:36:57.351096+0000","last_clean":"2024-04-25T12:36:57.351096+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:36:57.351096+0000","last_undegraded":"2024-04-25T12:36:57.351096+0000","last_fullsized":"2024-04-25T12:36:57.351096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350113+0000","last_change":"2024-04-25T12:36:47.049569+0000","last_active":"2024-04-25T12:36:57.350113+0000","last_peered":"2024-04-25T12:36:57.350113+0000","last_clean":"2024-04-25T12:36:57.350113+0000","last_became_active":"2024-04-25T12:36:47.049150+0000","last_became_peered":"2024-04-25T12:36:47.049150+0000","last_unstale":"2024-04-25T12:36:57.350113+0000","last_undegraded":"2024-04-25T12:36:57.350113+0000","last_fullsized":"2024-04-25T12:36:57.350113+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349952+0000","last_change":"2024-04-25T12:36:47.055946+0000","last_active":"2024-04-25T12:36:57.349952+0000","last_peered":"2024-04-25T12:36:57.349952+0000","last_clean":"2024-04-25T12:36:57.349952+0000","last_became_active":"2024-04-25T12:36:47.055621+0000","last_became_peered":"2024-04-25T12:36:47.055621+0000","last_unstale":"2024-04-25T12:36:57.349952+0000","last_undegraded":"2024-04-25T12:36:57.349952+0000","last_fullsized":"2024-04-25T12:36:57.349952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:25.650869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":124,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460297+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:36:56.460297+0000","last_peered":"2024-04-25T12:36:56.460297+0000","last_clean":"2024-04-25T12:36:56.460297+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:36:56.460297+0000","last_undegraded":"2024-04-25T12:36:56.460297+0000","last_fullsized":"2024-04-25T12:36:56.460297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146693+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:36:56.146693+0000","last_peered":"2024-04-25T12:36:56.146693+0000","last_clean":"2024-04-25T12:36:56.146693+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:36:56.146693+0000","last_undegraded":"2024-04-25T12:36:56.146693+0000","last_fullsized":"2024-04-25T12:36:56.146693+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:49.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":107,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460310+0000","last_change":"2024-04-25T12:36:47.045792+0000","last_active":"2024-04-25T12:36:56.460310+0000","last_peered":"2024-04-25T12:36:56.460310+0000","last_clean":"2024-04-25T12:36:56.460310+0000","last_became_active":"2024-04-25T12:36:47.045430+0000","last_became_peered":"2024-04-25T12:36:47.045430+0000","last_unstale":"2024-04-25T12:36:56.460310+0000","last_undegraded":"2024-04-25T12:36:56.460310+0000","last_fullsized":"2024-04-25T12:36:56.460310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:56.140737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"70'23","reported_seq":140,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148164+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:36:56.148164+0000","last_peered":"2024-04-25T12:36:56.148164+0000","last_clean":"2024-04-25T12:36:56.148164+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:36:56.148164+0000","last_undegraded":"2024-04-25T12:36:56.148164+0000","last_fullsized":"2024-04-25T12:36:56.148164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":122,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121327+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:36:55.121327+0000","last_peered":"2024-04-25T12:36:55.121327+0000","last_clean":"2024-04-25T12:36:55.121327+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:36:55.121327+0000","last_undegraded":"2024-04-25T12:36:55.121327+0000","last_fullsized":"2024-04-25T12:36:55.121327+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350391+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:36:57.350391+0000","last_peered":"2024-04-25T12:36:57.350391+0000","last_clean":"2024-04-25T12:36:57.350391+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:36:57.350391+0000","last_undegraded":"2024-04-25T12:36:57.350391+0000","last_fullsized":"2024-04-25T12:36:57.350391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"70'6","reported_seq":124,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460598+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:36:56.460598+0000","last_peered":"2024-04-25T12:36:56.460598+0000","last_clean":"2024-04-25T12:36:56.460598+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:36:56.460598+0000","last_undegraded":"2024-04-25T12:36:56.460598+0000","last_fullsized":"2024-04-25T12:36:56.460598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":110,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146980+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:36:56.146980+0000","last_peered":"2024-04-25T12:36:56.146980+0000","last_clean":"2024-04-25T12:36:56.146980+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:36:56.146980+0000","last_undegraded":"2024-04-25T12:36:56.146980+0000","last_fullsized":"2024-04-25T12:36:56.146980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168555+0000","last_change":"2024-04-25T12:36:00.636634+0000","last_active":"2024-04-25T12:36:53.168555+0000","last_peered":"2024-04-25T12:36:53.168555+0000","last_clean":"2024-04-25T12:36:53.168555+0000","last_became_active":"2024-04-25T12:36:00.636371+0000","last_became_peered":"2024-04-25T12:36:00.636371+0000","last_unstale":"2024-04-25T12:36:53.168555+0000","last_undegraded":"2024-04-25T12:36:53.168555+0000","last_fullsized":"2024-04-25T12:36:53.168555+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:36.185819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":120,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165416+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:36:57.165416+0000","last_peered":"2024-04-25T12:36:57.165416+0000","last_clean":"2024-04-25T12:36:57.165416+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:36:57.165416+0000","last_undegraded":"2024-04-25T12:36:57.165416+0000","last_fullsized":"2024-04-25T12:36:57.165416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":111,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.459748+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:36:56.459748+0000","last_peered":"2024-04-25T12:36:56.459748+0000","last_clean":"2024-04-25T12:36:56.459748+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:36:56.459748+0000","last_undegraded":"2024-04-25T12:36:56.459748+0000","last_fullsized":"2024-04-25T12:36:56.459748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148421+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:36:56.148421+0000","last_peered":"2024-04-25T12:36:56.148421+0000","last_clean":"2024-04-25T12:36:56.148421+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:36:56.148421+0000","last_undegraded":"2024-04-25T12:36:56.148421+0000","last_fullsized":"2024-04-25T12:36:56.148421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":121,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148133+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:36:56.148133+0000","last_peered":"2024-04-25T12:36:56.148133+0000","last_clean":"2024-04-25T12:36:56.148133+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:36:56.148133+0000","last_undegraded":"2024-04-25T12:36:56.148133+0000","last_fullsized":"2024-04-25T12:36:56.148133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":108,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121070+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:36:55.121070+0000","last_peered":"2024-04-25T12:36:55.121070+0000","last_clean":"2024-04-25T12:36:55.121070+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:36:55.121070+0000","last_undegraded":"2024-04-25T12:36:55.121070+0000","last_fullsized":"2024-04-25T12:36:55.121070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":119,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121239+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:36:55.121239+0000","last_peered":"2024-04-25T12:36:55.121239+0000","last_clean":"2024-04-25T12:36:55.121239+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:36:55.121239+0000","last_undegraded":"2024-04-25T12:36:55.121239+0000","last_fullsized":"2024-04-25T12:36:55.121239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121189+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:36:55.121189+0000","last_peered":"2024-04-25T12:36:55.121189+0000","last_clean":"2024-04-25T12:36:55.121189+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:36:55.121189+0000","last_undegraded":"2024-04-25T12:36:55.121189+0000","last_fullsized":"2024-04-25T12:36:55.121189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":117,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349591+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:36:57.349591+0000","last_peered":"2024-04-25T12:36:57.349591+0000","last_clean":"2024-04-25T12:36:57.349591+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:36:57.349591+0000","last_undegraded":"2024-04-25T12:36:57.349591+0000","last_fullsized":"2024-04-25T12:36:57.349591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.348823+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:36:57.348823+0000","last_peered":"2024-04-25T12:36:57.348823+0000","last_clean":"2024-04-25T12:36:57.348823+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:36:57.348823+0000","last_undegraded":"2024-04-25T12:36:57.348823+0000","last_fullsized":"2024-04-25T12:36:57.348823+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":69,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.167982+0000","last_change":"2024-04-25T12:36:47.039070+0000","last_active":"2024-04-25T12:36:53.167982+0000","last_peered":"2024-04-25T12:36:53.167982+0000","last_clean":"2024-04-25T12:36:53.167982+0000","last_became_active":"2024-04-25T12:36:47.037815+0000","last_became_peered":"2024-04-25T12:36:47.037815+0000","last_unstale":"2024-04-25T12:36:53.167982+0000","last_undegraded":"2024-04-25T12:36:53.167982+0000","last_fullsized":"2024-04-25T12:36:53.167982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:39.429293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163571+0000","last_change":"2024-04-25T12:36:47.057899+0000","last_active":"2024-04-25T12:36:57.163571+0000","last_peered":"2024-04-25T12:36:57.163571+0000","last_clean":"2024-04-25T12:36:57.163571+0000","last_became_active":"2024-04-25T12:36:47.047096+0000","last_became_peered":"2024-04-25T12:36:47.047096+0000","last_unstale":"2024-04-25T12:36:57.163571+0000","last_undegraded":"2024-04-25T12:36:57.163571+0000","last_fullsized":"2024-04-25T12:36:57.163571+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":119,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350790+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:36:57.350790+0000","last_peered":"2024-04-25T12:36:57.350790+0000","last_clean":"2024-04-25T12:36:57.350790+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:36:57.350790+0000","last_undegraded":"2024-04-25T12:36:57.350790+0000","last_fullsized":"2024-04-25T12:36:57.350790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121238+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:36:55.121238+0000","last_peered":"2024-04-25T12:36:55.121238+0000","last_clean":"2024-04-25T12:36:55.121238+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:36:55.121238+0000","last_undegraded":"2024-04-25T12:36:55.121238+0000","last_fullsized":"2024-04-25T12:36:55.121238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":126,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121269+0000","last_change":"2024-04-25T12:36:47.042017+0000","last_active":"2024-04-25T12:36:55.121269+0000","last_peered":"2024-04-25T12:36:55.121269+0000","last_clean":"2024-04-25T12:36:55.121269+0000","last_became_active":"2024-04-25T12:36:47.041560+0000","last_became_peered":"2024-04-25T12:36:47.041560+0000","last_unstale":"2024-04-25T12:36:55.121269+0000","last_undegraded":"2024-04-25T12:36:55.121269+0000","last_fullsized":"2024-04-25T12:36:55.121269+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:05.907270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,7],"acting":[4,1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":94,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168173+0000","last_change":"2024-04-25T12:36:47.074244+0000","last_active":"2024-04-25T12:36:53.168173+0000","last_peered":"2024-04-25T12:36:53.168173+0000","last_clean":"2024-04-25T12:36:53.168173+0000","last_became_active":"2024-04-25T12:36:47.050694+0000","last_became_peered":"2024-04-25T12:36:47.050694+0000","last_unstale":"2024-04-25T12:36:53.168173+0000","last_undegraded":"2024-04-25T12:36:53.168173+0000","last_fullsized":"2024-04-25T12:36:53.168173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:17.710097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147139+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:36:56.147139+0000","last_peered":"2024-04-25T12:36:56.147139+0000","last_clean":"2024-04-25T12:36:56.147139+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:36:56.147139+0000","last_undegraded":"2024-04-25T12:36:56.147139+0000","last_fullsized":"2024-04-25T12:36:56.147139+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460342+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:36:56.460342+0000","last_peered":"2024-04-25T12:36:56.460342+0000","last_clean":"2024-04-25T12:36:56.460342+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:36:56.460342+0000","last_undegraded":"2024-04-25T12:36:56.460342+0000","last_fullsized":"2024-04-25T12:36:56.460342+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146501+0000","last_change":"2024-04-25T12:36:47.053095+0000","last_active":"2024-04-25T12:36:56.146501+0000","last_peered":"2024-04-25T12:36:56.146501+0000","last_clean":"2024-04-25T12:36:56.146501+0000","last_became_active":"2024-04-25T12:36:47.052678+0000","last_became_peered":"2024-04-25T12:36:47.052678+0000","last_unstale":"2024-04-25T12:36:56.146501+0000","last_undegraded":"2024-04-25T12:36:56.146501+0000","last_fullsized":"2024-04-25T12:36:56.146501+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:30.345307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.459978+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:36:56.459978+0000","last_peered":"2024-04-25T12:36:56.459978+0000","last_clean":"2024-04-25T12:36:56.459978+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:36:56.459978+0000","last_undegraded":"2024-04-25T12:36:56.459978+0000","last_fullsized":"2024-04-25T12:36:56.459978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350411+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:36:57.350411+0000","last_peered":"2024-04-25T12:36:57.350411+0000","last_clean":"2024-04-25T12:36:57.350411+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:36:57.350411+0000","last_undegraded":"2024-04-25T12:36:57.350411+0000","last_fullsized":"2024-04-25T12:36:57.350411+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168293+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:36:53.168293+0000","last_peered":"2024-04-25T12:36:53.168293+0000","last_clean":"2024-04-25T12:36:53.168293+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:36:53.168293+0000","last_undegraded":"2024-04-25T12:36:53.168293+0000","last_fullsized":"2024-04-25T12:36:53.168293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164355+0000","last_change":"2024-04-25T12:36:47.051425+0000","last_active":"2024-04-25T12:36:57.164355+0000","last_peered":"2024-04-25T12:36:57.164355+0000","last_clean":"2024-04-25T12:36:57.164355+0000","last_became_active":"2024-04-25T12:36:47.051047+0000","last_became_peered":"2024-04-25T12:36:47.051047+0000","last_unstale":"2024-04-25T12:36:57.164355+0000","last_undegraded":"2024-04-25T12:36:57.164355+0000","last_fullsized":"2024-04-25T12:36:57.164355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460459+0000","last_change":"2024-04-25T12:36:47.046741+0000","last_active":"2024-04-25T12:36:56.460459+0000","last_peered":"2024-04-25T12:36:56.460459+0000","last_clean":"2024-04-25T12:36:56.460459+0000","last_became_active":"2024-04-25T12:36:47.046344+0000","last_became_peered":"2024-04-25T12:36:47.046344+0000","last_unstale":"2024-04-25T12:36:56.460459+0000","last_undegraded":"2024-04-25T12:36:56.460459+0000","last_fullsized":"2024-04-25T12:36:56.460459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:02.748651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148276+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:36:56.148276+0000","last_peered":"2024-04-25T12:36:56.148276+0000","last_clean":"2024-04-25T12:36:56.148276+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:36:56.148276+0000","last_undegraded":"2024-04-25T12:36:56.148276+0000","last_fullsized":"2024-04-25T12:36:56.148276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168623+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:36:53.168623+0000","last_peered":"2024-04-25T12:36:53.168623+0000","last_clean":"2024-04-25T12:36:53.168623+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:36:53.168623+0000","last_undegraded":"2024-04-25T12:36:53.168623+0000","last_fullsized":"2024-04-25T12:36:53.168623+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349358+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:36:57.349358+0000","last_peered":"2024-04-25T12:36:57.349358+0000","last_clean":"2024-04-25T12:36:57.349358+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:36:57.349358+0000","last_undegraded":"2024-04-25T12:36:57.349358+0000","last_fullsized":"2024-04-25T12:36:57.349358+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.167974+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:36:53.167974+0000","last_peered":"2024-04-25T12:36:53.167974+0000","last_clean":"2024-04-25T12:36:53.167974+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:36:53.167974+0000","last_undegraded":"2024-04-25T12:36:53.167974+0000","last_fullsized":"2024-04-25T12:36:53.167974+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147026+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:36:56.147026+0000","last_peered":"2024-04-25T12:36:56.147026+0000","last_clean":"2024-04-25T12:36:56.147026+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:36:56.147026+0000","last_undegraded":"2024-04-25T12:36:56.147026+0000","last_fullsized":"2024-04-25T12:36:56.147026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121072+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:36:55.121072+0000","last_peered":"2024-04-25T12:36:55.121072+0000","last_clean":"2024-04-25T12:36:55.121072+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:36:55.121072+0000","last_undegraded":"2024-04-25T12:36:55.121072+0000","last_fullsized":"2024-04-25T12:36:55.121072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164898+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:36:57.164898+0000","last_peered":"2024-04-25T12:36:57.164898+0000","last_clean":"2024-04-25T12:36:57.164898+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:36:57.164898+0000","last_undegraded":"2024-04-25T12:36:57.164898+0000","last_fullsized":"2024-04-25T12:36:57.164898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352181+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:36:57.352181+0000","last_peered":"2024-04-25T12:36:57.352181+0000","last_clean":"2024-04-25T12:36:57.352181+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:36:57.352181+0000","last_undegraded":"2024-04-25T12:36:57.352181+0000","last_fullsized":"2024-04-25T12:36:57.352181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460544+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:36:56.460544+0000","last_peered":"2024-04-25T12:36:56.460544+0000","last_clean":"2024-04-25T12:36:56.460544+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:36:56.460544+0000","last_undegraded":"2024-04-25T12:36:56.460544+0000","last_fullsized":"2024-04-25T12:36:56.460544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352144+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:36:57.352144+0000","last_peered":"2024-04-25T12:36:57.352144+0000","last_clean":"2024-04-25T12:36:57.352144+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:36:57.352144+0000","last_undegraded":"2024-04-25T12:36:57.352144+0000","last_fullsized":"2024-04-25T12:36:57.352144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349338+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:36:57.349338+0000","last_peered":"2024-04-25T12:36:57.349338+0000","last_clean":"2024-04-25T12:36:57.349338+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:36:57.349338+0000","last_undegraded":"2024-04-25T12:36:57.349338+0000","last_fullsized":"2024-04-25T12:36:57.349338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168020+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:36:53.168020+0000","last_peered":"2024-04-25T12:36:53.168020+0000","last_clean":"2024-04-25T12:36:53.168020+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:36:53.168020+0000","last_undegraded":"2024-04-25T12:36:53.168020+0000","last_fullsized":"2024-04-25T12:36:53.168020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"70'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460030+0000","last_change":"2024-04-25T12:36:47.022124+0000","last_active":"2024-04-25T12:36:56.460030+0000","last_peered":"2024-04-25T12:36:56.460030+0000","last_clean":"2024-04-25T12:36:56.460030+0000","last_became_active":"2024-04-25T12:36:47.021756+0000","last_became_peered":"2024-04-25T12:36:47.021756+0000","last_unstale":"2024-04-25T12:36:56.460030+0000","last_undegraded":"2024-04-25T12:36:56.460030+0000","last_fullsized":"2024-04-25T12:36:56.460030+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:52.833074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164288+0000","last_change":"2024-04-25T12:36:47.045979+0000","last_active":"2024-04-25T12:36:57.164288+0000","last_peered":"2024-04-25T12:36:57.164288+0000","last_clean":"2024-04-25T12:36:57.164288+0000","last_became_active":"2024-04-25T12:36:47.045157+0000","last_became_peered":"2024-04-25T12:36:47.045157+0000","last_unstale":"2024-04-25T12:36:57.164288+0000","last_undegraded":"2024-04-25T12:36:57.164288+0000","last_fullsized":"2024-04-25T12:36:57.164288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:51.221823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164714+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:36:57.164714+0000","last_peered":"2024-04-25T12:36:57.164714+0000","last_clean":"2024-04-25T12:36:57.164714+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:36:57.164714+0000","last_undegraded":"2024-04-25T12:36:57.164714+0000","last_fullsized":"2024-04-25T12:36:57.164714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164801+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:36:57.164801+0000","last_peered":"2024-04-25T12:36:57.164801+0000","last_clean":"2024-04-25T12:36:57.164801+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:36:57.164801+0000","last_undegraded":"2024-04-25T12:36:57.164801+0000","last_fullsized":"2024-04-25T12:36:57.164801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165285+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:36:57.165285+0000","last_peered":"2024-04-25T12:36:57.165285+0000","last_clean":"2024-04-25T12:36:57.165285+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:36:57.165285+0000","last_undegraded":"2024-04-25T12:36:57.165285+0000","last_fullsized":"2024-04-25T12:36:57.165285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148001+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:36:56.148001+0000","last_peered":"2024-04-25T12:36:56.148001+0000","last_clean":"2024-04-25T12:36:56.148001+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:36:56.148001+0000","last_undegraded":"2024-04-25T12:36:56.148001+0000","last_fullsized":"2024-04-25T12:36:56.148001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168009+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:36:53.168009+0000","last_peered":"2024-04-25T12:36:53.168009+0000","last_clean":"2024-04-25T12:36:53.168009+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:36:53.168009+0000","last_undegraded":"2024-04-25T12:36:53.168009+0000","last_fullsized":"2024-04-25T12:36:53.168009+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163777+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:36:57.163777+0000","last_peered":"2024-04-25T12:36:57.163777+0000","last_clean":"2024-04-25T12:36:57.163777+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:36:57.163777+0000","last_undegraded":"2024-04-25T12:36:57.163777+0000","last_fullsized":"2024-04-25T12:36:57.163777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146669+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:36:56.146669+0000","last_peered":"2024-04-25T12:36:56.146669+0000","last_clean":"2024-04-25T12:36:56.146669+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:36:56.146669+0000","last_undegraded":"2024-04-25T12:36:56.146669+0000","last_fullsized":"2024-04-25T12:36:56.146669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":123,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164545+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:36:57.164545+0000","last_peered":"2024-04-25T12:36:57.164545+0000","last_clean":"2024-04-25T12:36:57.164545+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:36:57.164545+0000","last_undegraded":"2024-04-25T12:36:57.164545+0000","last_fullsized":"2024-04-25T12:36:57.164545+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148034+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:36:56.148034+0000","last_peered":"2024-04-25T12:36:56.148034+0000","last_clean":"2024-04-25T12:36:56.148034+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:36:56.148034+0000","last_undegraded":"2024-04-25T12:36:56.148034+0000","last_fullsized":"2024-04-25T12:36:56.148034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168477+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:36:53.168477+0000","last_peered":"2024-04-25T12:36:53.168477+0000","last_clean":"2024-04-25T12:36:53.168477+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:36:53.168477+0000","last_undegraded":"2024-04-25T12:36:53.168477+0000","last_fullsized":"2024-04-25T12:36:53.168477+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":69,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T12:36:53.168608+0000","last_change":"2024-04-25T12:36:47.049286+0000","last_active":"2024-04-25T12:36:53.168608+0000","last_peered":"2024-04-25T12:36:53.168608+0000","last_clean":"2024-04-25T12:36:53.168608+0000","last_became_active":"2024-04-25T12:36:47.048252+0000","last_became_peered":"2024-04-25T12:36:47.048252+0000","last_unstale":"2024-04-25T12:36:53.168608+0000","last_undegraded":"2024-04-25T12:36:53.168608+0000","last_fullsized":"2024-04-25T12:36:53.168608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:38.216266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164734+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:36:57.164734+0000","last_peered":"2024-04-25T12:36:57.164734+0000","last_clean":"2024-04-25T12:36:57.164734+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:36:57.164734+0000","last_undegraded":"2024-04-25T12:36:57.164734+0000","last_fullsized":"2024-04-25T12:36:57.164734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":37563,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1204,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":76330,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":430081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":430793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":430074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":60129542181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462924,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":808,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":827392,"data_stored":413658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":1054433,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":892928,"data_stored":469424,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":1026276,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":33676,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":35210,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:36:58.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:36:59.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.041+0000 7f129a000640 1 Processor -- start 2024-04-25T12:36:59.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.041+0000 7f129a000640 1 -- start start 2024-04-25T12:36:59.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f129a000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12940cdc60 0x7f12940b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f129a000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f129a000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129415fdd0 con 0x7f12940d1ad0 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f129a000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f12941600b0 con 0x7f12940cdc60 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f129a000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1294160390 con 0x7f129415cf60 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f1298c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12940cdc60 0x7f12940b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f1298c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12940cdc60 0x7f12940b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44474/0 (socket says 172.21.15.123:44474) 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f1298c00640 1 -- 172.21.15.123:0/3512441279 learned_addr learned my addr 172.21.15.123:0/3512441279 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.043+0000 7f1299600640 1 --2- 172.21.15.123:0/3512441279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.044+0000 7f1293400640 1 -- 172.21.15.123:0/3512441279 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2665168982 0 0) 0x7f129415fdd0 con 0x7f12940d1ad0 2024-04-25T12:36:59.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.044+0000 7f1293400640 1 -- 172.21.15.123:0/3512441279 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1278003520 con 0x7f12940d1ad0 2024-04-25T12:36:59.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.044+0000 7f1298c00640 1 -- 172.21.15.123:0/3512441279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 msgr2=0x7f12940d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:59.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.044+0000 7f1298c00640 1 --2- 172.21.15.123:0/3512441279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.044+0000 7f1298c00640 1 -- 172.21.15.123:0/3512441279 >> v1:172.21.15.153:6789/0 conn(0x7f12940d1ad0 legacy=0x7f129415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.044+0000 7f1298c00640 1 -- 172.21.15.123:0/3512441279 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1294160670 con 0x7f12940cdc60 2024-04-25T12:36:59.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.045+0000 7f1298c00640 1 --2- 172.21.15.123:0/3512441279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940cdc60 0x7f12940b42c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1284009e50 tx=0x7f128402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2f643fba028ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:59.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.045+0000 7f1293400640 1 -- 172.21.15.123:0/3512441279 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284036070 con 0x7f12940cdc60 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.045+0000 7f1293400640 1 -- 172.21.15.123:0/3512441279 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1284031600 con 0x7f12940cdc60 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.045+0000 7f1293400640 1 -- 172.21.15.123:0/3512441279 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12840319b0 con 0x7f12940cdc60 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 -- 172.21.15.123:0/3512441279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940cdc60 msgr2=0x7f12940b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 --2- 172.21.15.123:0/3512441279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940cdc60 0x7f12940b42c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1284009e50 tx=0x7f128402a710 comp rx=0 tx=0).stop 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 -- 172.21.15.123:0/3512441279 shutdown_connections 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 --2- 172.21.15.123:0/3512441279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 --2- 172.21.15.123:0/3512441279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940cdc60 0x7f12940b42c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 -- 172.21.15.123:0/3512441279 >> 172.21.15.123:0/3512441279 conn(0x7f129408d840 msgr2=0x7f129408dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 -- 172.21.15.123:0/3512441279 shutdown_connections 2024-04-25T12:36:59.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.046+0000 7f129a000640 1 -- 172.21.15.123:0/3512441279 wait complete. 2024-04-25T12:36:59.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.048+0000 7f129a000640 1 Processor -- start 2024-04-25T12:36:59.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.048+0000 7f129a000640 1 -- start start 2024-04-25T12:36:59.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f129a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12940cdc60 0x7f12941e9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f129a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940d1ad0 0x7f12941545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f129a000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f129a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f129415bd30 con 0x7f12940d1ad0 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f129a000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f129415bbb0 con 0x7f12940cdc60 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f129a000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f129415beb0 con 0x7f129415cf60 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f1293e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940d1ad0 0x7f12941545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f1293e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940d1ad0 0x7f12941545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44482/0 (socket says 172.21.15.123:44482) 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f1293e00640 1 -- 172.21.15.123:0/2716753698 learned_addr learned my addr 172.21.15.123:0/2716753698 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f1298c00640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12940cdc60 0x7f12941e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.050+0000 7f1299600640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1293e00640 1 -- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 msgr2=0x7f12940ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1293e00640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1293e00640 1 -- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12940cdc60 msgr2=0x7f12941e9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1293e00640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12940cdc60 0x7f12941e9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1293e00640 1 -- 172.21.15.123:0/2716753698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12941e9740 con 0x7f12940d1ad0 2024-04-25T12:36:59.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1293e00640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940d1ad0 0x7f12941545f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f127c001eb0 tx=0x7f127c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:59.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1290c00640 1 -- 172.21.15.123:0/2716753698 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c017070 con 0x7f12940d1ad0 2024-04-25T12:36:59.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1290c00640 1 -- 172.21.15.123:0/2716753698 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c0052d0 con 0x7f12940d1ad0 2024-04-25T12:36:59.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f1290c00640 1 -- 172.21.15.123:0/2716753698 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c003810 con 0x7f12940d1ad0 2024-04-25T12:36:59.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12941e7640 con 0x7f12940d1ad0 2024-04-25T12:36:59.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.051+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12941e8a00 con 0x7f12940d1ad0 2024-04-25T12:36:59.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.053+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12940cf100 con 0x7f12940d1ad0 2024-04-25T12:36:59.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.054+0000 7f1290c00640 1 -- 172.21.15.123:0/2716753698 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f127c01a070 con 0x7f12940d1ad0 2024-04-25T12:36:59.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.055+0000 7f1290c00640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12700744b0 0x7f12700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.055+0000 7f1290c00640 1 -- 172.21.15.123:0/2716753698 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6685+0+0 (secure 0 0 0) 0x7f127c0948e0 con 0x7f12940d1ad0 2024-04-25T12:36:59.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.055+0000 7f1298c00640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12700744b0 0x7f12700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.056+0000 7f1298c00640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12700744b0 0x7f12700769c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1284033000 tx=0x7f1284009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:59.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.060+0000 7f1290c00640 1 -- 172.21.15.123:0/2716753698 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12940cf100 con 0x7f12940d1ad0 2024-04-25T12:36:59.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.262+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12940ce190 con 0x7f12700744b0 2024-04-25T12:36:59.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.272+0000 7f1290c00640 1 -- 172.21.15.123:0/2716753698 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333070 (secure 0 0 0) 0x7f12940ce190 con 0x7f12700744b0 2024-04-25T12:36:59.277 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:36:59.279 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12700744b0 msgr2=0x7f12700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12700744b0 0x7f12700769c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1284033000 tx=0x7f1284009000 comp rx=0 tx=0).stop 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940d1ad0 msgr2=0x7f12941545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940d1ad0 0x7f12941545f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f127c001eb0 tx=0x7f127c00c820 comp rx=0 tx=0).stop 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 shutdown_connections 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12700744b0 0x7f12700769c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f129415cf60 0x7f12940ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12940d1ad0 0x7f12941545f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 --2- 172.21.15.123:0/2716753698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12940cdc60 0x7f12941e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 >> 172.21.15.123:0/2716753698 conn(0x7f129408d840 msgr2=0x7f12940c96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 shutdown_connections 2024-04-25T12:36:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.280+0000 7f129a000640 1 -- 172.21.15.123:0/2716753698 wait complete. 2024-04-25T12:36:59.309 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-04-25T12:36:58.833817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":627931,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":67,"num_read_kb":56,"num_write":262,"num_write_kb":1718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":591572,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":200632,"kb_used_data":6084,"kb_used_omap":18,"kb_used_meta":194413,"kb_avail":660402248,"statfs":{"total":676457349120,"available":676251901952,"internally_reserved":0,"allocated":6230016,"data_stored":4278526,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":18931,"internal_metadata":199079437},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8374,"num_objects":0,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000724"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":125,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121732+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:36:55.121732+0000","last_peered":"2024-04-25T12:36:55.121732+0000","last_clean":"2024-04-25T12:36:55.121732+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:36:55.121732+0000","last_undegraded":"2024-04-25T12:36:55.121732+0000","last_fullsized":"2024-04-25T12:36:55.121732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175295+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:36:58.175295+0000","last_peered":"2024-04-25T12:36:58.175295+0000","last_clean":"2024-04-25T12:36:58.175295+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:36:58.175295+0000","last_undegraded":"2024-04-25T12:36:58.175295+0000","last_fullsized":"2024-04-25T12:36:58.175295+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":118,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.459883+0000","last_change":"2024-04-25T12:36:47.042028+0000","last_active":"2024-04-25T12:36:56.459883+0000","last_peered":"2024-04-25T12:36:56.459883+0000","last_clean":"2024-04-25T12:36:56.459883+0000","last_became_active":"2024-04-25T12:36:47.041474+0000","last_became_peered":"2024-04-25T12:36:47.041474+0000","last_unstale":"2024-04-25T12:36:56.459883+0000","last_undegraded":"2024-04-25T12:36:56.459883+0000","last_fullsized":"2024-04-25T12:36:56.459883+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:09.865004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,1],"acting":[7,5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175062+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:36:58.175062+0000","last_peered":"2024-04-25T12:36:58.175062+0000","last_clean":"2024-04-25T12:36:58.175062+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:36:58.175062+0000","last_undegraded":"2024-04-25T12:36:58.175062+0000","last_fullsized":"2024-04-25T12:36:58.175062+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175535+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:36:58.175535+0000","last_peered":"2024-04-25T12:36:58.175535+0000","last_clean":"2024-04-25T12:36:58.175535+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:36:58.175535+0000","last_undegraded":"2024-04-25T12:36:58.175535+0000","last_fullsized":"2024-04-25T12:36:58.175535+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146217+0000","last_change":"2024-04-25T12:36:47.048696+0000","last_active":"2024-04-25T12:36:56.146217+0000","last_peered":"2024-04-25T12:36:56.146217+0000","last_clean":"2024-04-25T12:36:56.146217+0000","last_became_active":"2024-04-25T12:36:47.048209+0000","last_became_peered":"2024-04-25T12:36:47.048209+0000","last_unstale":"2024-04-25T12:36:56.146217+0000","last_undegraded":"2024-04-25T12:36:56.146217+0000","last_fullsized":"2024-04-25T12:36:56.146217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:06.823061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,1],"acting":[2,7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":123,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164994+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:36:57.164994+0000","last_peered":"2024-04-25T12:36:57.164994+0000","last_clean":"2024-04-25T12:36:57.164994+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:36:57.164994+0000","last_undegraded":"2024-04-25T12:36:57.164994+0000","last_fullsized":"2024-04-25T12:36:57.164994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147838+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:36:56.147838+0000","last_peered":"2024-04-25T12:36:56.147838+0000","last_clean":"2024-04-25T12:36:56.147838+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:36:56.147838+0000","last_undegraded":"2024-04-25T12:36:56.147838+0000","last_fullsized":"2024-04-25T12:36:56.147838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":128,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146846+0000","last_change":"2024-04-25T12:36:47.060381+0000","last_active":"2024-04-25T12:36:56.146846+0000","last_peered":"2024-04-25T12:36:56.146846+0000","last_clean":"2024-04-25T12:36:56.146846+0000","last_became_active":"2024-04-25T12:36:47.059933+0000","last_became_peered":"2024-04-25T12:36:47.059933+0000","last_unstale":"2024-04-25T12:36:56.146846+0000","last_undegraded":"2024-04-25T12:36:56.146846+0000","last_fullsized":"2024-04-25T12:36:56.146846+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:42.613333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,0],"acting":[2,5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":120,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165373+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:36:57.165373+0000","last_peered":"2024-04-25T12:36:57.165373+0000","last_clean":"2024-04-25T12:36:57.165373+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:36:57.165373+0000","last_undegraded":"2024-04-25T12:36:57.165373+0000","last_fullsized":"2024-04-25T12:36:57.165373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350833+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:36:57.350833+0000","last_peered":"2024-04-25T12:36:57.350833+0000","last_clean":"2024-04-25T12:36:57.350833+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:36:57.350833+0000","last_undegraded":"2024-04-25T12:36:57.350833+0000","last_fullsized":"2024-04-25T12:36:57.350833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":128,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146424+0000","last_change":"2024-04-25T12:36:47.061434+0000","last_active":"2024-04-25T12:36:56.146424+0000","last_peered":"2024-04-25T12:36:56.146424+0000","last_clean":"2024-04-25T12:36:56.146424+0000","last_became_active":"2024-04-25T12:36:47.060986+0000","last_became_peered":"2024-04-25T12:36:47.060986+0000","last_unstale":"2024-04-25T12:36:56.146424+0000","last_undegraded":"2024-04-25T12:36:56.146424+0000","last_fullsized":"2024-04-25T12:36:56.146424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:54.714311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,7],"acting":[2,0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351025+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:36:57.351025+0000","last_peered":"2024-04-25T12:36:57.351025+0000","last_clean":"2024-04-25T12:36:57.351025+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:36:57.351025+0000","last_undegraded":"2024-04-25T12:36:57.351025+0000","last_fullsized":"2024-04-25T12:36:57.351025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350562+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:36:57.350562+0000","last_peered":"2024-04-25T12:36:57.350562+0000","last_clean":"2024-04-25T12:36:57.350562+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:36:57.350562+0000","last_undegraded":"2024-04-25T12:36:57.350562+0000","last_fullsized":"2024-04-25T12:36:57.350562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163514+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:36:57.163514+0000","last_peered":"2024-04-25T12:36:57.163514+0000","last_clean":"2024-04-25T12:36:57.163514+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:36:57.163514+0000","last_undegraded":"2024-04-25T12:36:57.163514+0000","last_fullsized":"2024-04-25T12:36:57.163514+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":121,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460155+0000","last_change":"2024-04-25T12:36:47.046901+0000","last_active":"2024-04-25T12:36:56.460155+0000","last_peered":"2024-04-25T12:36:56.460155+0000","last_clean":"2024-04-25T12:36:56.460155+0000","last_became_active":"2024-04-25T12:36:47.046621+0000","last_became_peered":"2024-04-25T12:36:47.046621+0000","last_unstale":"2024-04-25T12:36:56.460155+0000","last_undegraded":"2024-04-25T12:36:56.460155+0000","last_fullsized":"2024-04-25T12:36:56.460155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:30.345307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174563+0000","last_change":"2024-04-25T12:36:47.066385+0000","last_active":"2024-04-25T12:36:58.174563+0000","last_peered":"2024-04-25T12:36:58.174563+0000","last_clean":"2024-04-25T12:36:58.174563+0000","last_became_active":"2024-04-25T12:36:47.046898+0000","last_became_peered":"2024-04-25T12:36:47.046898+0000","last_unstale":"2024-04-25T12:36:58.174563+0000","last_undegraded":"2024-04-25T12:36:58.174563+0000","last_fullsized":"2024-04-25T12:36:58.174563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.673850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":118,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164308+0000","last_change":"2024-04-25T12:36:47.051974+0000","last_active":"2024-04-25T12:36:57.164308+0000","last_peered":"2024-04-25T12:36:57.164308+0000","last_clean":"2024-04-25T12:36:57.164308+0000","last_became_active":"2024-04-25T12:36:47.051537+0000","last_became_peered":"2024-04-25T12:36:47.051537+0000","last_unstale":"2024-04-25T12:36:57.164308+0000","last_undegraded":"2024-04-25T12:36:57.164308+0000","last_fullsized":"2024-04-25T12:36:57.164308+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.244233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.120969+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:36:55.120969+0000","last_peered":"2024-04-25T12:36:55.120969+0000","last_clean":"2024-04-25T12:36:55.120969+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:36:55.120969+0000","last_undegraded":"2024-04-25T12:36:55.120969+0000","last_fullsized":"2024-04-25T12:36:55.120969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":119,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121559+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:36:55.121559+0000","last_peered":"2024-04-25T12:36:55.121559+0000","last_clean":"2024-04-25T12:36:55.121559+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:36:55.121559+0000","last_undegraded":"2024-04-25T12:36:55.121559+0000","last_fullsized":"2024-04-25T12:36:55.121559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":111,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460524+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:36:56.460524+0000","last_peered":"2024-04-25T12:36:56.460524+0000","last_clean":"2024-04-25T12:36:56.460524+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:36:56.460524+0000","last_undegraded":"2024-04-25T12:36:56.460524+0000","last_fullsized":"2024-04-25T12:36:56.460524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349878+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:36:57.349878+0000","last_peered":"2024-04-25T12:36:57.349878+0000","last_clean":"2024-04-25T12:36:57.349878+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:36:57.349878+0000","last_undegraded":"2024-04-25T12:36:57.349878+0000","last_fullsized":"2024-04-25T12:36:57.349878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.348869+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:36:57.348869+0000","last_peered":"2024-04-25T12:36:57.348869+0000","last_clean":"2024-04-25T12:36:57.348869+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:36:57.348869+0000","last_undegraded":"2024-04-25T12:36:57.348869+0000","last_fullsized":"2024-04-25T12:36:57.348869+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":110,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148228+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:36:56.148228+0000","last_peered":"2024-04-25T12:36:56.148228+0000","last_clean":"2024-04-25T12:36:56.148228+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:36:56.148228+0000","last_undegraded":"2024-04-25T12:36:56.148228+0000","last_fullsized":"2024-04-25T12:36:56.148228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121355+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:36:55.121355+0000","last_peered":"2024-04-25T12:36:55.121355+0000","last_clean":"2024-04-25T12:36:55.121355+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:36:55.121355+0000","last_undegraded":"2024-04-25T12:36:55.121355+0000","last_fullsized":"2024-04-25T12:36:55.121355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352342+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:36:57.352342+0000","last_peered":"2024-04-25T12:36:57.352342+0000","last_clean":"2024-04-25T12:36:57.352342+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:36:57.352342+0000","last_undegraded":"2024-04-25T12:36:57.352342+0000","last_fullsized":"2024-04-25T12:36:57.352342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146333+0000","last_change":"2024-04-25T12:36:47.058724+0000","last_active":"2024-04-25T12:36:56.146333+0000","last_peered":"2024-04-25T12:36:56.146333+0000","last_clean":"2024-04-25T12:36:56.146333+0000","last_became_active":"2024-04-25T12:36:47.058280+0000","last_became_peered":"2024-04-25T12:36:47.058280+0000","last_unstale":"2024-04-25T12:36:56.146333+0000","last_undegraded":"2024-04-25T12:36:56.146333+0000","last_fullsized":"2024-04-25T12:36:56.146333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:02.910553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,1],"acting":[2,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":119,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121725+0000","last_change":"2024-04-25T12:36:47.041012+0000","last_active":"2024-04-25T12:36:55.121725+0000","last_peered":"2024-04-25T12:36:55.121725+0000","last_clean":"2024-04-25T12:36:55.121725+0000","last_became_active":"2024-04-25T12:36:47.040684+0000","last_became_peered":"2024-04-25T12:36:47.040684+0000","last_unstale":"2024-04-25T12:36:55.121725+0000","last_undegraded":"2024-04-25T12:36:55.121725+0000","last_fullsized":"2024-04-25T12:36:55.121725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"70'11","reported_seq":127,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121558+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:36:55.121558+0000","last_peered":"2024-04-25T12:36:55.121558+0000","last_clean":"2024-04-25T12:36:55.121558+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:36:55.121558+0000","last_undegraded":"2024-04-25T12:36:55.121558+0000","last_fullsized":"2024-04-25T12:36:55.121558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.120937+0000","last_change":"2024-04-25T12:36:47.044258+0000","last_active":"2024-04-25T12:36:55.120937+0000","last_peered":"2024-04-25T12:36:55.120937+0000","last_clean":"2024-04-25T12:36:55.120937+0000","last_became_active":"2024-04-25T12:36:47.043910+0000","last_became_peered":"2024-04-25T12:36:47.043910+0000","last_unstale":"2024-04-25T12:36:55.120937+0000","last_undegraded":"2024-04-25T12:36:55.120937+0000","last_fullsized":"2024-04-25T12:36:55.120937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,3],"acting":[4,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175167+0000","last_change":"2024-04-25T12:36:47.043858+0000","last_active":"2024-04-25T12:36:58.175167+0000","last_peered":"2024-04-25T12:36:58.175167+0000","last_clean":"2024-04-25T12:36:58.175167+0000","last_became_active":"2024-04-25T12:36:47.043265+0000","last_became_peered":"2024-04-25T12:36:47.043265+0000","last_unstale":"2024-04-25T12:36:58.175167+0000","last_undegraded":"2024-04-25T12:36:58.175167+0000","last_fullsized":"2024-04-25T12:36:58.175167+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:31.461355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":117,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352198+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:36:57.352198+0000","last_peered":"2024-04-25T12:36:57.352198+0000","last_clean":"2024-04-25T12:36:57.352198+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:36:57.352198+0000","last_undegraded":"2024-04-25T12:36:57.352198+0000","last_fullsized":"2024-04-25T12:36:57.352198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174633+0000","last_change":"2024-04-25T12:36:47.047427+0000","last_active":"2024-04-25T12:36:58.174633+0000","last_peered":"2024-04-25T12:36:58.174633+0000","last_clean":"2024-04-25T12:36:58.174633+0000","last_became_active":"2024-04-25T12:36:47.046961+0000","last_became_peered":"2024-04-25T12:36:47.046961+0000","last_unstale":"2024-04-25T12:36:58.174633+0000","last_undegraded":"2024-04-25T12:36:58.174633+0000","last_fullsized":"2024-04-25T12:36:58.174633+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.321075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,7],"acting":[5,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148302+0000","last_change":"2024-04-25T12:36:47.043679+0000","last_active":"2024-04-25T12:36:56.148302+0000","last_peered":"2024-04-25T12:36:56.148302+0000","last_clean":"2024-04-25T12:36:56.148302+0000","last_became_active":"2024-04-25T12:36:47.043290+0000","last_became_peered":"2024-04-25T12:36:47.043290+0000","last_unstale":"2024-04-25T12:36:56.148302+0000","last_undegraded":"2024-04-25T12:36:56.148302+0000","last_fullsized":"2024-04-25T12:36:56.148302+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:56.140737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":78,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174971+0000","last_change":"2024-04-25T12:36:47.043694+0000","last_active":"2024-04-25T12:36:58.174971+0000","last_peered":"2024-04-25T12:36:58.174971+0000","last_clean":"2024-04-25T12:36:58.174971+0000","last_became_active":"2024-04-25T12:36:47.043238+0000","last_became_peered":"2024-04-25T12:36:47.043238+0000","last_unstale":"2024-04-25T12:36:58.174971+0000","last_undegraded":"2024-04-25T12:36:58.174971+0000","last_fullsized":"2024-04-25T12:36:58.174971+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:58.619827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121112+0000","last_change":"2024-04-25T12:36:47.048814+0000","last_active":"2024-04-25T12:36:55.121112+0000","last_peered":"2024-04-25T12:36:55.121112+0000","last_clean":"2024-04-25T12:36:55.121112+0000","last_became_active":"2024-04-25T12:36:47.048345+0000","last_became_peered":"2024-04-25T12:36:47.048345+0000","last_unstale":"2024-04-25T12:36:55.121112+0000","last_undegraded":"2024-04-25T12:36:55.121112+0000","last_fullsized":"2024-04-25T12:36:55.121112+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,7],"acting":[4,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164628+0000","last_change":"2024-04-25T12:34:13.969240+0000","last_active":"2024-04-25T12:36:57.164628+0000","last_peered":"2024-04-25T12:36:57.164628+0000","last_clean":"2024-04-25T12:36:57.164628+0000","last_became_active":"2024-04-25T12:34:13.968220+0000","last_became_peered":"2024-04-25T12:34:13.968220+0000","last_unstale":"2024-04-25T12:36:57.164628+0000","last_undegraded":"2024-04-25T12:36:57.164628+0000","last_fullsized":"2024-04-25T12:36:57.164628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:36.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":122,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351311+0000","last_change":"2024-04-25T12:36:47.053780+0000","last_active":"2024-04-25T12:36:57.351311+0000","last_peered":"2024-04-25T12:36:57.351311+0000","last_clean":"2024-04-25T12:36:57.351311+0000","last_became_active":"2024-04-25T12:36:47.053245+0000","last_became_peered":"2024-04-25T12:36:47.053245+0000","last_unstale":"2024-04-25T12:36:57.351311+0000","last_undegraded":"2024-04-25T12:36:57.351311+0000","last_fullsized":"2024-04-25T12:36:57.351311+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:52.614962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147558+0000","last_change":"2024-04-25T12:36:47.055665+0000","last_active":"2024-04-25T12:36:56.147558+0000","last_peered":"2024-04-25T12:36:56.147558+0000","last_clean":"2024-04-25T12:36:56.147558+0000","last_became_active":"2024-04-25T12:36:47.055104+0000","last_became_peered":"2024-04-25T12:36:47.055104+0000","last_unstale":"2024-04-25T12:36:56.147558+0000","last_undegraded":"2024-04-25T12:36:56.147558+0000","last_fullsized":"2024-04-25T12:36:56.147558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0,3],"acting":[1,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174519+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:36:58.174519+0000","last_peered":"2024-04-25T12:36:58.174519+0000","last_clean":"2024-04-25T12:36:58.174519+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:36:58.174519+0000","last_undegraded":"2024-04-25T12:36:58.174519+0000","last_fullsized":"2024-04-25T12:36:58.174519+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"70'6","reported_seq":132,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460030+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:36:56.460030+0000","last_peered":"2024-04-25T12:36:56.460030+0000","last_clean":"2024-04-25T12:36:56.460030+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:36:56.460030+0000","last_undegraded":"2024-04-25T12:36:56.460030+0000","last_fullsized":"2024-04-25T12:36:56.460030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352147+0000","last_change":"2024-04-25T12:36:47.048616+0000","last_active":"2024-04-25T12:36:57.352147+0000","last_peered":"2024-04-25T12:36:57.352147+0000","last_clean":"2024-04-25T12:36:57.352147+0000","last_became_active":"2024-04-25T12:36:47.048179+0000","last_became_peered":"2024-04-25T12:36:47.048179+0000","last_unstale":"2024-04-25T12:36:57.352147+0000","last_undegraded":"2024-04-25T12:36:57.352147+0000","last_fullsized":"2024-04-25T12:36:57.352147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:00.979187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,7],"acting":[3,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146342+0000","last_change":"2024-04-25T12:36:47.036913+0000","last_active":"2024-04-25T12:36:56.146342+0000","last_peered":"2024-04-25T12:36:56.146342+0000","last_clean":"2024-04-25T12:36:56.146342+0000","last_became_active":"2024-04-25T12:36:47.032924+0000","last_became_peered":"2024-04-25T12:36:47.032924+0000","last_unstale":"2024-04-25T12:36:56.146342+0000","last_undegraded":"2024-04-25T12:36:56.146342+0000","last_fullsized":"2024-04-25T12:36:56.146342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:05.790594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":126,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349785+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:36:57.349785+0000","last_peered":"2024-04-25T12:36:57.349785+0000","last_clean":"2024-04-25T12:36:57.349785+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:36:57.349785+0000","last_undegraded":"2024-04-25T12:36:57.349785+0000","last_fullsized":"2024-04-25T12:36:57.349785+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175485+0000","last_change":"2024-04-25T12:36:47.048722+0000","last_active":"2024-04-25T12:36:58.175485+0000","last_peered":"2024-04-25T12:36:58.175485+0000","last_clean":"2024-04-25T12:36:58.175485+0000","last_became_active":"2024-04-25T12:36:47.047551+0000","last_became_peered":"2024-04-25T12:36:47.047551+0000","last_unstale":"2024-04-25T12:36:58.175485+0000","last_undegraded":"2024-04-25T12:36:58.175485+0000","last_fullsized":"2024-04-25T12:36:58.175485+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:27.524785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0,1],"acting":[5,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121559+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:36:55.121559+0000","last_peered":"2024-04-25T12:36:55.121559+0000","last_clean":"2024-04-25T12:36:55.121559+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:36:55.121559+0000","last_undegraded":"2024-04-25T12:36:55.121559+0000","last_fullsized":"2024-04-25T12:36:55.121559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":119,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350590+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:36:57.350590+0000","last_peered":"2024-04-25T12:36:57.350590+0000","last_clean":"2024-04-25T12:36:57.350590+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:36:57.350590+0000","last_undegraded":"2024-04-25T12:36:57.350590+0000","last_fullsized":"2024-04-25T12:36:57.350590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":224,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.145925+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:36:56.145925+0000","last_peered":"2024-04-25T12:36:56.145925+0000","last_clean":"2024-04-25T12:36:56.145925+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:36:56.145925+0000","last_undegraded":"2024-04-25T12:36:56.145925+0000","last_fullsized":"2024-04-25T12:36:56.145925+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":110,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351272+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:36:57.351272+0000","last_peered":"2024-04-25T12:36:57.351272+0000","last_clean":"2024-04-25T12:36:57.351272+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:36:57.351272+0000","last_undegraded":"2024-04-25T12:36:57.351272+0000","last_fullsized":"2024-04-25T12:36:57.351272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351077+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:36:57.351077+0000","last_peered":"2024-04-25T12:36:57.351077+0000","last_clean":"2024-04-25T12:36:57.351077+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:36:57.351077+0000","last_undegraded":"2024-04-25T12:36:57.351077+0000","last_fullsized":"2024-04-25T12:36:57.351077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165273+0000","last_change":"2024-04-25T12:36:47.073082+0000","last_active":"2024-04-25T12:36:57.165273+0000","last_peered":"2024-04-25T12:36:57.165273+0000","last_clean":"2024-04-25T12:36:57.165273+0000","last_became_active":"2024-04-25T12:36:47.047092+0000","last_became_peered":"2024-04-25T12:36:47.047092+0000","last_unstale":"2024-04-25T12:36:57.165273+0000","last_undegraded":"2024-04-25T12:36:57.165273+0000","last_fullsized":"2024-04-25T12:36:57.165273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:06.488375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":110,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146510+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:36:56.146510+0000","last_peered":"2024-04-25T12:36:56.146510+0000","last_clean":"2024-04-25T12:36:56.146510+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:36:56.146510+0000","last_undegraded":"2024-04-25T12:36:56.146510+0000","last_fullsized":"2024-04-25T12:36:56.146510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165094+0000","last_change":"2024-04-25T12:34:13.975405+0000","last_active":"2024-04-25T12:36:57.165094+0000","last_peered":"2024-04-25T12:36:57.165094+0000","last_clean":"2024-04-25T12:36:57.165094+0000","last_became_active":"2024-04-25T12:34:13.970758+0000","last_became_peered":"2024-04-25T12:34:13.970758+0000","last_unstale":"2024-04-25T12:36:57.165094+0000","last_undegraded":"2024-04-25T12:36:57.165094+0000","last_fullsized":"2024-04-25T12:36:57.165094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.099537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":86,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175180+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:36:58.175180+0000","last_peered":"2024-04-25T12:36:58.175180+0000","last_clean":"2024-04-25T12:36:58.175180+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:36:58.175180+0000","last_undegraded":"2024-04-25T12:36:58.175180+0000","last_fullsized":"2024-04-25T12:36:58.175180+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174797+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:36:58.174797+0000","last_peered":"2024-04-25T12:36:58.174797+0000","last_clean":"2024-04-25T12:36:58.174797+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:36:58.174797+0000","last_undegraded":"2024-04-25T12:36:58.174797+0000","last_fullsized":"2024-04-25T12:36:58.174797+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164976+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:36:57.164976+0000","last_peered":"2024-04-25T12:36:57.164976+0000","last_clean":"2024-04-25T12:36:57.164976+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:36:57.164976+0000","last_undegraded":"2024-04-25T12:36:57.164976+0000","last_fullsized":"2024-04-25T12:36:57.164976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":125,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121442+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:36:55.121442+0000","last_peered":"2024-04-25T12:36:55.121442+0000","last_clean":"2024-04-25T12:36:55.121442+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:36:55.121442+0000","last_undegraded":"2024-04-25T12:36:55.121442+0000","last_fullsized":"2024-04-25T12:36:55.121442+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":110,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349113+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:36:57.349113+0000","last_peered":"2024-04-25T12:36:57.349113+0000","last_clean":"2024-04-25T12:36:57.349113+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:36:57.349113+0000","last_undegraded":"2024-04-25T12:36:57.349113+0000","last_fullsized":"2024-04-25T12:36:57.349113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148438+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:36:56.148438+0000","last_peered":"2024-04-25T12:36:56.148438+0000","last_clean":"2024-04-25T12:36:56.148438+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:36:56.148438+0000","last_undegraded":"2024-04-25T12:36:56.148438+0000","last_fullsized":"2024-04-25T12:36:56.148438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":127,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165188+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:36:57.165188+0000","last_peered":"2024-04-25T12:36:57.165188+0000","last_clean":"2024-04-25T12:36:57.165188+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:36:57.165188+0000","last_undegraded":"2024-04-25T12:36:57.165188+0000","last_fullsized":"2024-04-25T12:36:57.165188+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146611+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:36:56.146611+0000","last_peered":"2024-04-25T12:36:56.146611+0000","last_clean":"2024-04-25T12:36:56.146611+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:36:56.146611+0000","last_undegraded":"2024-04-25T12:36:56.146611+0000","last_fullsized":"2024-04-25T12:36:56.146611+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349560+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:36:57.349560+0000","last_peered":"2024-04-25T12:36:57.349560+0000","last_clean":"2024-04-25T12:36:57.349560+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:36:57.349560+0000","last_undegraded":"2024-04-25T12:36:57.349560+0000","last_fullsized":"2024-04-25T12:36:57.349560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":131,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121444+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:36:55.121444+0000","last_peered":"2024-04-25T12:36:55.121444+0000","last_clean":"2024-04-25T12:36:55.121444+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:36:55.121444+0000","last_undegraded":"2024-04-25T12:36:55.121444+0000","last_fullsized":"2024-04-25T12:36:55.121444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175335+0000","last_change":"2024-04-25T12:36:47.048991+0000","last_active":"2024-04-25T12:36:58.175335+0000","last_peered":"2024-04-25T12:36:58.175335+0000","last_clean":"2024-04-25T12:36:58.175335+0000","last_became_active":"2024-04-25T12:36:47.047195+0000","last_became_peered":"2024-04-25T12:36:47.047195+0000","last_unstale":"2024-04-25T12:36:58.175335+0000","last_undegraded":"2024-04-25T12:36:58.175335+0000","last_fullsized":"2024-04-25T12:36:58.175335+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:15.211552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0,1],"acting":[5,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.145745+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:36:56.145745+0000","last_peered":"2024-04-25T12:36:56.145745+0000","last_clean":"2024-04-25T12:36:56.145745+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:36:56.145745+0000","last_undegraded":"2024-04-25T12:36:56.145745+0000","last_fullsized":"2024-04-25T12:36:56.145745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163852+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:36:57.163852+0000","last_peered":"2024-04-25T12:36:57.163852+0000","last_clean":"2024-04-25T12:36:57.163852+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:36:57.163852+0000","last_undegraded":"2024-04-25T12:36:57.163852+0000","last_fullsized":"2024-04-25T12:36:57.163852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175604+0000","last_change":"2024-04-25T12:36:47.048302+0000","last_active":"2024-04-25T12:36:58.175604+0000","last_peered":"2024-04-25T12:36:58.175604+0000","last_clean":"2024-04-25T12:36:58.175604+0000","last_became_active":"2024-04-25T12:36:47.047884+0000","last_became_peered":"2024-04-25T12:36:47.047884+0000","last_unstale":"2024-04-25T12:36:58.175604+0000","last_undegraded":"2024-04-25T12:36:58.175604+0000","last_fullsized":"2024-04-25T12:36:58.175604+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:02.311326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,4],"acting":[5,1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350004+0000","last_change":"2024-04-25T12:34:13.978003+0000","last_active":"2024-04-25T12:36:57.350004+0000","last_peered":"2024-04-25T12:36:57.350004+0000","last_clean":"2024-04-25T12:36:57.350004+0000","last_became_active":"2024-04-25T12:34:13.977466+0000","last_became_peered":"2024-04-25T12:34:13.977466+0000","last_unstale":"2024-04-25T12:36:57.350004+0000","last_undegraded":"2024-04-25T12:36:57.350004+0000","last_fullsized":"2024-04-25T12:36:57.350004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.975590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":120,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.351096+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:36:57.351096+0000","last_peered":"2024-04-25T12:36:57.351096+0000","last_clean":"2024-04-25T12:36:57.351096+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:36:57.351096+0000","last_undegraded":"2024-04-25T12:36:57.351096+0000","last_fullsized":"2024-04-25T12:36:57.351096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350113+0000","last_change":"2024-04-25T12:36:47.049569+0000","last_active":"2024-04-25T12:36:57.350113+0000","last_peered":"2024-04-25T12:36:57.350113+0000","last_clean":"2024-04-25T12:36:57.350113+0000","last_became_active":"2024-04-25T12:36:47.049150+0000","last_became_peered":"2024-04-25T12:36:47.049150+0000","last_unstale":"2024-04-25T12:36:57.350113+0000","last_undegraded":"2024-04-25T12:36:57.350113+0000","last_fullsized":"2024-04-25T12:36:57.350113+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349952+0000","last_change":"2024-04-25T12:36:47.055946+0000","last_active":"2024-04-25T12:36:57.349952+0000","last_peered":"2024-04-25T12:36:57.349952+0000","last_clean":"2024-04-25T12:36:57.349952+0000","last_became_active":"2024-04-25T12:36:47.055621+0000","last_became_peered":"2024-04-25T12:36:47.055621+0000","last_unstale":"2024-04-25T12:36:57.349952+0000","last_undegraded":"2024-04-25T12:36:57.349952+0000","last_fullsized":"2024-04-25T12:36:57.349952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:25.650869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":124,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460297+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:36:56.460297+0000","last_peered":"2024-04-25T12:36:56.460297+0000","last_clean":"2024-04-25T12:36:56.460297+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:36:56.460297+0000","last_undegraded":"2024-04-25T12:36:56.460297+0000","last_fullsized":"2024-04-25T12:36:56.460297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146693+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:36:56.146693+0000","last_peered":"2024-04-25T12:36:56.146693+0000","last_clean":"2024-04-25T12:36:56.146693+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:36:56.146693+0000","last_undegraded":"2024-04-25T12:36:56.146693+0000","last_fullsized":"2024-04-25T12:36:56.146693+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:49.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":107,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460310+0000","last_change":"2024-04-25T12:36:47.045792+0000","last_active":"2024-04-25T12:36:56.460310+0000","last_peered":"2024-04-25T12:36:56.460310+0000","last_clean":"2024-04-25T12:36:56.460310+0000","last_became_active":"2024-04-25T12:36:47.045430+0000","last_became_peered":"2024-04-25T12:36:47.045430+0000","last_unstale":"2024-04-25T12:36:56.460310+0000","last_undegraded":"2024-04-25T12:36:56.460310+0000","last_fullsized":"2024-04-25T12:36:56.460310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:56.140737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"70'23","reported_seq":140,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148164+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:36:56.148164+0000","last_peered":"2024-04-25T12:36:56.148164+0000","last_clean":"2024-04-25T12:36:56.148164+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:36:56.148164+0000","last_undegraded":"2024-04-25T12:36:56.148164+0000","last_fullsized":"2024-04-25T12:36:56.148164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":122,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121327+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:36:55.121327+0000","last_peered":"2024-04-25T12:36:55.121327+0000","last_clean":"2024-04-25T12:36:55.121327+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:36:55.121327+0000","last_undegraded":"2024-04-25T12:36:55.121327+0000","last_fullsized":"2024-04-25T12:36:55.121327+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350391+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:36:57.350391+0000","last_peered":"2024-04-25T12:36:57.350391+0000","last_clean":"2024-04-25T12:36:57.350391+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:36:57.350391+0000","last_undegraded":"2024-04-25T12:36:57.350391+0000","last_fullsized":"2024-04-25T12:36:57.350391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"70'6","reported_seq":124,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460598+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:36:56.460598+0000","last_peered":"2024-04-25T12:36:56.460598+0000","last_clean":"2024-04-25T12:36:56.460598+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:36:56.460598+0000","last_undegraded":"2024-04-25T12:36:56.460598+0000","last_fullsized":"2024-04-25T12:36:56.460598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":110,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146980+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:36:56.146980+0000","last_peered":"2024-04-25T12:36:56.146980+0000","last_clean":"2024-04-25T12:36:56.146980+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:36:56.146980+0000","last_undegraded":"2024-04-25T12:36:56.146980+0000","last_fullsized":"2024-04-25T12:36:56.146980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175363+0000","last_change":"2024-04-25T12:36:00.636634+0000","last_active":"2024-04-25T12:36:58.175363+0000","last_peered":"2024-04-25T12:36:58.175363+0000","last_clean":"2024-04-25T12:36:58.175363+0000","last_became_active":"2024-04-25T12:36:00.636371+0000","last_became_peered":"2024-04-25T12:36:00.636371+0000","last_unstale":"2024-04-25T12:36:58.175363+0000","last_undegraded":"2024-04-25T12:36:58.175363+0000","last_fullsized":"2024-04-25T12:36:58.175363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:36.185819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":120,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165416+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:36:57.165416+0000","last_peered":"2024-04-25T12:36:57.165416+0000","last_clean":"2024-04-25T12:36:57.165416+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:36:57.165416+0000","last_undegraded":"2024-04-25T12:36:57.165416+0000","last_fullsized":"2024-04-25T12:36:57.165416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":111,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.459748+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:36:56.459748+0000","last_peered":"2024-04-25T12:36:56.459748+0000","last_clean":"2024-04-25T12:36:56.459748+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:36:56.459748+0000","last_undegraded":"2024-04-25T12:36:56.459748+0000","last_fullsized":"2024-04-25T12:36:56.459748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148421+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:36:56.148421+0000","last_peered":"2024-04-25T12:36:56.148421+0000","last_clean":"2024-04-25T12:36:56.148421+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:36:56.148421+0000","last_undegraded":"2024-04-25T12:36:56.148421+0000","last_fullsized":"2024-04-25T12:36:56.148421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":121,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148133+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:36:56.148133+0000","last_peered":"2024-04-25T12:36:56.148133+0000","last_clean":"2024-04-25T12:36:56.148133+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:36:56.148133+0000","last_undegraded":"2024-04-25T12:36:56.148133+0000","last_fullsized":"2024-04-25T12:36:56.148133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":108,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121070+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:36:55.121070+0000","last_peered":"2024-04-25T12:36:55.121070+0000","last_clean":"2024-04-25T12:36:55.121070+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:36:55.121070+0000","last_undegraded":"2024-04-25T12:36:55.121070+0000","last_fullsized":"2024-04-25T12:36:55.121070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":119,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121239+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:36:55.121239+0000","last_peered":"2024-04-25T12:36:55.121239+0000","last_clean":"2024-04-25T12:36:55.121239+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:36:55.121239+0000","last_undegraded":"2024-04-25T12:36:55.121239+0000","last_fullsized":"2024-04-25T12:36:55.121239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121189+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:36:55.121189+0000","last_peered":"2024-04-25T12:36:55.121189+0000","last_clean":"2024-04-25T12:36:55.121189+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:36:55.121189+0000","last_undegraded":"2024-04-25T12:36:55.121189+0000","last_fullsized":"2024-04-25T12:36:55.121189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":117,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349591+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:36:57.349591+0000","last_peered":"2024-04-25T12:36:57.349591+0000","last_clean":"2024-04-25T12:36:57.349591+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:36:57.349591+0000","last_undegraded":"2024-04-25T12:36:57.349591+0000","last_fullsized":"2024-04-25T12:36:57.349591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.348823+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:36:57.348823+0000","last_peered":"2024-04-25T12:36:57.348823+0000","last_clean":"2024-04-25T12:36:57.348823+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:36:57.348823+0000","last_undegraded":"2024-04-25T12:36:57.348823+0000","last_fullsized":"2024-04-25T12:36:57.348823+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":78,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174603+0000","last_change":"2024-04-25T12:36:47.039070+0000","last_active":"2024-04-25T12:36:58.174603+0000","last_peered":"2024-04-25T12:36:58.174603+0000","last_clean":"2024-04-25T12:36:58.174603+0000","last_became_active":"2024-04-25T12:36:47.037815+0000","last_became_peered":"2024-04-25T12:36:47.037815+0000","last_unstale":"2024-04-25T12:36:58.174603+0000","last_undegraded":"2024-04-25T12:36:58.174603+0000","last_fullsized":"2024-04-25T12:36:58.174603+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:39.429293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163571+0000","last_change":"2024-04-25T12:36:47.057899+0000","last_active":"2024-04-25T12:36:57.163571+0000","last_peered":"2024-04-25T12:36:57.163571+0000","last_clean":"2024-04-25T12:36:57.163571+0000","last_became_active":"2024-04-25T12:36:47.047096+0000","last_became_peered":"2024-04-25T12:36:47.047096+0000","last_unstale":"2024-04-25T12:36:57.163571+0000","last_undegraded":"2024-04-25T12:36:57.163571+0000","last_fullsized":"2024-04-25T12:36:57.163571+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":119,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350790+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:36:57.350790+0000","last_peered":"2024-04-25T12:36:57.350790+0000","last_clean":"2024-04-25T12:36:57.350790+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:36:57.350790+0000","last_undegraded":"2024-04-25T12:36:57.350790+0000","last_fullsized":"2024-04-25T12:36:57.350790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121238+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:36:55.121238+0000","last_peered":"2024-04-25T12:36:55.121238+0000","last_clean":"2024-04-25T12:36:55.121238+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:36:55.121238+0000","last_undegraded":"2024-04-25T12:36:55.121238+0000","last_fullsized":"2024-04-25T12:36:55.121238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":126,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121269+0000","last_change":"2024-04-25T12:36:47.042017+0000","last_active":"2024-04-25T12:36:55.121269+0000","last_peered":"2024-04-25T12:36:55.121269+0000","last_clean":"2024-04-25T12:36:55.121269+0000","last_became_active":"2024-04-25T12:36:47.041560+0000","last_became_peered":"2024-04-25T12:36:47.041560+0000","last_unstale":"2024-04-25T12:36:55.121269+0000","last_undegraded":"2024-04-25T12:36:55.121269+0000","last_fullsized":"2024-04-25T12:36:55.121269+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:05.907270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,7],"acting":[4,1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":103,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174683+0000","last_change":"2024-04-25T12:36:47.074244+0000","last_active":"2024-04-25T12:36:58.174683+0000","last_peered":"2024-04-25T12:36:58.174683+0000","last_clean":"2024-04-25T12:36:58.174683+0000","last_became_active":"2024-04-25T12:36:47.050694+0000","last_became_peered":"2024-04-25T12:36:47.050694+0000","last_unstale":"2024-04-25T12:36:58.174683+0000","last_undegraded":"2024-04-25T12:36:58.174683+0000","last_fullsized":"2024-04-25T12:36:58.174683+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:17.710097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147139+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:36:56.147139+0000","last_peered":"2024-04-25T12:36:56.147139+0000","last_clean":"2024-04-25T12:36:56.147139+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:36:56.147139+0000","last_undegraded":"2024-04-25T12:36:56.147139+0000","last_fullsized":"2024-04-25T12:36:56.147139+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460342+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:36:56.460342+0000","last_peered":"2024-04-25T12:36:56.460342+0000","last_clean":"2024-04-25T12:36:56.460342+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:36:56.460342+0000","last_undegraded":"2024-04-25T12:36:56.460342+0000","last_fullsized":"2024-04-25T12:36:56.460342+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146501+0000","last_change":"2024-04-25T12:36:47.053095+0000","last_active":"2024-04-25T12:36:56.146501+0000","last_peered":"2024-04-25T12:36:56.146501+0000","last_clean":"2024-04-25T12:36:56.146501+0000","last_became_active":"2024-04-25T12:36:47.052678+0000","last_became_peered":"2024-04-25T12:36:47.052678+0000","last_unstale":"2024-04-25T12:36:56.146501+0000","last_undegraded":"2024-04-25T12:36:56.146501+0000","last_fullsized":"2024-04-25T12:36:56.146501+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:30.345307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.459978+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:36:56.459978+0000","last_peered":"2024-04-25T12:36:56.459978+0000","last_clean":"2024-04-25T12:36:56.459978+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:36:56.459978+0000","last_undegraded":"2024-04-25T12:36:56.459978+0000","last_fullsized":"2024-04-25T12:36:56.459978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.350411+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:36:57.350411+0000","last_peered":"2024-04-25T12:36:57.350411+0000","last_clean":"2024-04-25T12:36:57.350411+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:36:57.350411+0000","last_undegraded":"2024-04-25T12:36:57.350411+0000","last_fullsized":"2024-04-25T12:36:57.350411+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174850+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:36:58.174850+0000","last_peered":"2024-04-25T12:36:58.174850+0000","last_clean":"2024-04-25T12:36:58.174850+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:36:58.174850+0000","last_undegraded":"2024-04-25T12:36:58.174850+0000","last_fullsized":"2024-04-25T12:36:58.174850+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164355+0000","last_change":"2024-04-25T12:36:47.051425+0000","last_active":"2024-04-25T12:36:57.164355+0000","last_peered":"2024-04-25T12:36:57.164355+0000","last_clean":"2024-04-25T12:36:57.164355+0000","last_became_active":"2024-04-25T12:36:47.051047+0000","last_became_peered":"2024-04-25T12:36:47.051047+0000","last_unstale":"2024-04-25T12:36:57.164355+0000","last_undegraded":"2024-04-25T12:36:57.164355+0000","last_fullsized":"2024-04-25T12:36:57.164355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460459+0000","last_change":"2024-04-25T12:36:47.046741+0000","last_active":"2024-04-25T12:36:56.460459+0000","last_peered":"2024-04-25T12:36:56.460459+0000","last_clean":"2024-04-25T12:36:56.460459+0000","last_became_active":"2024-04-25T12:36:47.046344+0000","last_became_peered":"2024-04-25T12:36:47.046344+0000","last_unstale":"2024-04-25T12:36:56.460459+0000","last_undegraded":"2024-04-25T12:36:56.460459+0000","last_fullsized":"2024-04-25T12:36:56.460459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:02.748651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148276+0000","last_change":"2024-04-25T12:34:13.956737+0000","last_active":"2024-04-25T12:36:56.148276+0000","last_peered":"2024-04-25T12:36:56.148276+0000","last_clean":"2024-04-25T12:36:56.148276+0000","last_became_active":"2024-04-25T12:34:13.956367+0000","last_became_peered":"2024-04-25T12:34:13.956367+0000","last_unstale":"2024-04-25T12:36:56.148276+0000","last_undegraded":"2024-04-25T12:36:56.148276+0000","last_fullsized":"2024-04-25T12:36:56.148276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:12.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175485+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:36:58.175485+0000","last_peered":"2024-04-25T12:36:58.175485+0000","last_clean":"2024-04-25T12:36:58.175485+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:36:58.175485+0000","last_undegraded":"2024-04-25T12:36:58.175485+0000","last_fullsized":"2024-04-25T12:36:58.175485+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349358+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:36:57.349358+0000","last_peered":"2024-04-25T12:36:57.349358+0000","last_clean":"2024-04-25T12:36:57.349358+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:36:57.349358+0000","last_undegraded":"2024-04-25T12:36:57.349358+0000","last_fullsized":"2024-04-25T12:36:57.349358+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174698+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:36:58.174698+0000","last_peered":"2024-04-25T12:36:58.174698+0000","last_clean":"2024-04-25T12:36:58.174698+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:36:58.174698+0000","last_undegraded":"2024-04-25T12:36:58.174698+0000","last_fullsized":"2024-04-25T12:36:58.174698+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.147026+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:36:56.147026+0000","last_peered":"2024-04-25T12:36:56.147026+0000","last_clean":"2024-04-25T12:36:56.147026+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:36:56.147026+0000","last_undegraded":"2024-04-25T12:36:56.147026+0000","last_fullsized":"2024-04-25T12:36:56.147026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T12:36:55.121072+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:36:55.121072+0000","last_peered":"2024-04-25T12:36:55.121072+0000","last_clean":"2024-04-25T12:36:55.121072+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:36:55.121072+0000","last_undegraded":"2024-04-25T12:36:55.121072+0000","last_fullsized":"2024-04-25T12:36:55.121072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164898+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:36:57.164898+0000","last_peered":"2024-04-25T12:36:57.164898+0000","last_clean":"2024-04-25T12:36:57.164898+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:36:57.164898+0000","last_undegraded":"2024-04-25T12:36:57.164898+0000","last_fullsized":"2024-04-25T12:36:57.164898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352181+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:36:57.352181+0000","last_peered":"2024-04-25T12:36:57.352181+0000","last_clean":"2024-04-25T12:36:57.352181+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:36:57.352181+0000","last_undegraded":"2024-04-25T12:36:57.352181+0000","last_fullsized":"2024-04-25T12:36:57.352181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460544+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:36:56.460544+0000","last_peered":"2024-04-25T12:36:56.460544+0000","last_clean":"2024-04-25T12:36:56.460544+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:36:56.460544+0000","last_undegraded":"2024-04-25T12:36:56.460544+0000","last_fullsized":"2024-04-25T12:36:56.460544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.352144+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:36:57.352144+0000","last_peered":"2024-04-25T12:36:57.352144+0000","last_clean":"2024-04-25T12:36:57.352144+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:36:57.352144+0000","last_undegraded":"2024-04-25T12:36:57.352144+0000","last_fullsized":"2024-04-25T12:36:57.352144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.349338+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:36:57.349338+0000","last_peered":"2024-04-25T12:36:57.349338+0000","last_clean":"2024-04-25T12:36:57.349338+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:36:57.349338+0000","last_undegraded":"2024-04-25T12:36:57.349338+0000","last_fullsized":"2024-04-25T12:36:57.349338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174603+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:36:58.174603+0000","last_peered":"2024-04-25T12:36:58.174603+0000","last_clean":"2024-04-25T12:36:58.174603+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:36:58.174603+0000","last_undegraded":"2024-04-25T12:36:58.174603+0000","last_fullsized":"2024-04-25T12:36:58.174603+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"70'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.460030+0000","last_change":"2024-04-25T12:36:47.022124+0000","last_active":"2024-04-25T12:36:56.460030+0000","last_peered":"2024-04-25T12:36:56.460030+0000","last_clean":"2024-04-25T12:36:56.460030+0000","last_became_active":"2024-04-25T12:36:47.021756+0000","last_became_peered":"2024-04-25T12:36:47.021756+0000","last_unstale":"2024-04-25T12:36:56.460030+0000","last_undegraded":"2024-04-25T12:36:56.460030+0000","last_fullsized":"2024-04-25T12:36:56.460030+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:52.833074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164288+0000","last_change":"2024-04-25T12:36:47.045979+0000","last_active":"2024-04-25T12:36:57.164288+0000","last_peered":"2024-04-25T12:36:57.164288+0000","last_clean":"2024-04-25T12:36:57.164288+0000","last_became_active":"2024-04-25T12:36:47.045157+0000","last_became_peered":"2024-04-25T12:36:47.045157+0000","last_unstale":"2024-04-25T12:36:57.164288+0000","last_undegraded":"2024-04-25T12:36:57.164288+0000","last_fullsized":"2024-04-25T12:36:57.164288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:51.221823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164714+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:36:57.164714+0000","last_peered":"2024-04-25T12:36:57.164714+0000","last_clean":"2024-04-25T12:36:57.164714+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:36:57.164714+0000","last_undegraded":"2024-04-25T12:36:57.164714+0000","last_fullsized":"2024-04-25T12:36:57.164714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164801+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:36:57.164801+0000","last_peered":"2024-04-25T12:36:57.164801+0000","last_clean":"2024-04-25T12:36:57.164801+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:36:57.164801+0000","last_undegraded":"2024-04-25T12:36:57.164801+0000","last_fullsized":"2024-04-25T12:36:57.164801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.165285+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:36:57.165285+0000","last_peered":"2024-04-25T12:36:57.165285+0000","last_clean":"2024-04-25T12:36:57.165285+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:36:57.165285+0000","last_undegraded":"2024-04-25T12:36:57.165285+0000","last_fullsized":"2024-04-25T12:36:57.165285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148001+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:36:56.148001+0000","last_peered":"2024-04-25T12:36:56.148001+0000","last_clean":"2024-04-25T12:36:56.148001+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:36:56.148001+0000","last_undegraded":"2024-04-25T12:36:56.148001+0000","last_fullsized":"2024-04-25T12:36:56.148001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.174657+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:36:58.174657+0000","last_peered":"2024-04-25T12:36:58.174657+0000","last_clean":"2024-04-25T12:36:58.174657+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:36:58.174657+0000","last_undegraded":"2024-04-25T12:36:58.174657+0000","last_fullsized":"2024-04-25T12:36:58.174657+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.163777+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:36:57.163777+0000","last_peered":"2024-04-25T12:36:57.163777+0000","last_clean":"2024-04-25T12:36:57.163777+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:36:57.163777+0000","last_undegraded":"2024-04-25T12:36:57.163777+0000","last_fullsized":"2024-04-25T12:36:57.163777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.146669+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:36:56.146669+0000","last_peered":"2024-04-25T12:36:56.146669+0000","last_clean":"2024-04-25T12:36:56.146669+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:36:56.146669+0000","last_undegraded":"2024-04-25T12:36:56.146669+0000","last_fullsized":"2024-04-25T12:36:56.146669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":123,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164545+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:36:57.164545+0000","last_peered":"2024-04-25T12:36:57.164545+0000","last_clean":"2024-04-25T12:36:57.164545+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:36:57.164545+0000","last_undegraded":"2024-04-25T12:36:57.164545+0000","last_fullsized":"2024-04-25T12:36:57.164545+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":115,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T12:36:56.148034+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:36:56.148034+0000","last_peered":"2024-04-25T12:36:56.148034+0000","last_clean":"2024-04-25T12:36:56.148034+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:36:56.148034+0000","last_undegraded":"2024-04-25T12:36:56.148034+0000","last_fullsized":"2024-04-25T12:36:56.148034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175042+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:36:58.175042+0000","last_peered":"2024-04-25T12:36:58.175042+0000","last_clean":"2024-04-25T12:36:58.175042+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:36:58.175042+0000","last_undegraded":"2024-04-25T12:36:58.175042+0000","last_fullsized":"2024-04-25T12:36:58.175042+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":78,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T12:36:58.175223+0000","last_change":"2024-04-25T12:36:47.049286+0000","last_active":"2024-04-25T12:36:58.175223+0000","last_peered":"2024-04-25T12:36:58.175223+0000","last_clean":"2024-04-25T12:36:58.175223+0000","last_became_active":"2024-04-25T12:36:47.048252+0000","last_became_peered":"2024-04-25T12:36:47.048252+0000","last_unstale":"2024-04-25T12:36:58.175223+0000","last_undegraded":"2024-04-25T12:36:58.175223+0000","last_fullsized":"2024-04-25T12:36:58.175223+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:38.216266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T12:36:57.164734+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:36:57.164734+0000","last_peered":"2024-04-25T12:36:57.164734+0000","last_clean":"2024-04-25T12:36:57.164734+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:36:57.164734+0000","last_undegraded":"2024-04-25T12:36:57.164734+0000","last_fullsized":"2024-04-25T12:36:57.164734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":37563,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1204,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":76431,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":430081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":430793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":430074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":60129542181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462925,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29096,"kb_used_data":856,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342744,"statfs":{"total":96636764160,"available":96606969856,"internally_reserved":0,"allocated":876544,"data_stored":437445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":1054433,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":892928,"data_stored":469424,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":1026276,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":33676,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":35210,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:36:59.315 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-25T12:36:59.315 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T12:36:59.316 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-04-25T12:36:59.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-04-25T12:36:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.628+0000 7fd96c600640 1 Processor -- start 2024-04-25T12:36:59.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.628+0000 7fd96c600640 1 -- start start 2024-04-25T12:36:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd96c600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd9640c8fd0 0x7fd964153640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd96c600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9641426a0 0x7fd964159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd96c600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd964074400 con 0x7fd9640c8170 2024-04-25T12:36:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd96c600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd964153b80 con 0x7fd9640c8fd0 2024-04-25T12:36:59.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd96c600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd9641595e0 con 0x7fd9641426a0 2024-04-25T12:36:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd969200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9641426a0 0x7fd964159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd969c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd9640c8fd0 0x7fd964153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.630+0000 7fd96a600640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd9640c8170 0x7fd9640ca200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:54106/0 (socket says 172.21.15.123:54106) 2024-04-25T12:36:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.631+0000 7fd969200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9641426a0 0x7fd964159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:39920/0 (socket says 172.21.15.123:39920) 2024-04-25T12:36:59.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.631+0000 7fd96a600640 1 -- 172.21.15.123:0/176467474 learned_addr learned my addr 172.21.15.123:0/176467474 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:36:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.631+0000 7fd969200640 1 -- 172.21.15.123:0/176467474 >> v2:172.21.15.153:3300/0 conn(0x7fd9640c8fd0 msgr2=0x7fd964153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:36:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.631+0000 7fd969200640 1 --2- 172.21.15.123:0/176467474 >> v2:172.21.15.153:3300/0 conn(0x7fd9640c8fd0 0x7fd964153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.631+0000 7fd969200640 1 -- 172.21.15.123:0/176467474 >> v1:172.21.15.153:6789/0 conn(0x7fd9640c8170 legacy=0x7fd9640ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.631+0000 7fd969200640 1 -- 172.21.15.123:0/176467474 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd964074670 con 0x7fd9641426a0 2024-04-25T12:36:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.632+0000 7fd969200640 1 --2- 172.21.15.123:0/176467474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9641426a0 0x7fd964159dc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9500097a0 tx=0x7fd95002c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de93b3936b99d7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:59.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.632+0000 7fd963e00640 1 -- 172.21.15.123:0/176467474 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950039070 con 0x7fd9641426a0 2024-04-25T12:36:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.632+0000 7fd963e00640 1 -- 172.21.15.123:0/176467474 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd950034640 con 0x7fd9641426a0 2024-04-25T12:36:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.632+0000 7fd963e00640 1 -- 172.21.15.123:0/176467474 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950034a10 con 0x7fd9641426a0 2024-04-25T12:36:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 -- 172.21.15.123:0/176467474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9641426a0 msgr2=0x7fd964159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 --2- 172.21.15.123:0/176467474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9641426a0 0x7fd964159dc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9500097a0 tx=0x7fd95002c390 comp rx=0 tx=0).stop 2024-04-25T12:36:59.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 -- 172.21.15.123:0/176467474 shutdown_connections 2024-04-25T12:36:59.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 --2- 172.21.15.123:0/176467474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9641426a0 0x7fd964159dc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 --2- 172.21.15.123:0/176467474 >> v2:172.21.15.153:3300/0 conn(0x7fd9640c8fd0 0x7fd964153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 -- 172.21.15.123:0/176467474 >> 172.21.15.123:0/176467474 conn(0x7fd96406f440 msgr2=0x7fd96406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:36:59.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 -- 172.21.15.123:0/176467474 shutdown_connections 2024-04-25T12:36:59.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.633+0000 7fd96c600640 1 -- 172.21.15.123:0/176467474 wait complete. 2024-04-25T12:36:59.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.635+0000 7fd96c600640 1 Processor -- start 2024-04-25T12:36:59.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.636+0000 7fd96c600640 1 -- start start 2024-04-25T12:36:59.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96c600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9640c8170 0x7fd9640b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96c600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9640c8fd0 0x7fd9641e8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96c600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9641426a0 0x7fd9640cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96c600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd96415a300 con 0x7fd9641426a0 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96c600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641598a0 con 0x7fd9640c8fd0 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96c600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd9640c4880 con 0x7fd9640c8170 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9641426a0 0x7fd9640cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9641426a0 0x7fd9640cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44486/0 (socket says 172.21.15.123:44486) 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.637+0000 7fd96a600640 1 -- 172.21.15.123:0/3850670820 learned_addr learned my addr 172.21.15.123:0/3850670820 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd969200640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9640c8fd0 0x7fd9641e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd969c00640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9640c8170 0x7fd9640b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd96a600640 1 -- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9640c8170 msgr2=0x7fd9640b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd96a600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9640c8170 0x7fd9640b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd96a600640 1 -- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9640c8fd0 msgr2=0x7fd9641e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:36:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd96a600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9640c8fd0 0x7fd9641e8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd96a600640 1 -- 172.21.15.123:0/3850670820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9640af800 con 0x7fd9641426a0 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd969200640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9640c8fd0 0x7fd9641e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.638+0000 7fd96a600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9641426a0 0x7fd9640cd8b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c310 tx=0x7fd95c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.639+0000 7fd961600640 1 -- 172.21.15.123:0/3850670820 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c017070 con 0x7fd9641426a0 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.639+0000 7fd961600640 1 -- 172.21.15.123:0/3850670820 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c005ce0 con 0x7fd9641426a0 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.639+0000 7fd961600640 1 -- 172.21.15.123:0/3850670820 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0072d0 con 0x7fd9641426a0 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.639+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9641e7cc0 con 0x7fd9641426a0 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.639+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9640b0c40 con 0x7fd9641426a0 2024-04-25T12:36:59.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.640+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd964076700 con 0x7fd9641426a0 2024-04-25T12:36:59.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.641+0000 7fd961600640 1 -- 172.21.15.123:0/3850670820 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd95c01a070 con 0x7fd9641426a0 2024-04-25T12:36:59.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.642+0000 7fd961600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9440744e0 0x7fd9440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:36:59.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.642+0000 7fd961600640 1 -- 172.21.15.123:0/3850670820 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6685+0+0 (secure 0 0 0) 0x7fd95c094660 con 0x7fd9641426a0 2024-04-25T12:36:59.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.643+0000 7fd969c00640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9440744e0 0x7fd9440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:36:59.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.644+0000 7fd969c00640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9440744e0 0x7fd9440769f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd958011340 tx=0x7fd95800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:36:59.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.648+0000 7fd961600640 1 -- 172.21.15.123:0/3850670820 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd95c05e520 con 0x7fd9641426a0 2024-04-25T12:36:59.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:36:59.857+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd in", "ids": ["6"]} v 0) -- 0x7fd9640c95f0 con 0x7fd9641426a0 2024-04-25T12:36:59.858 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-04-25T12:36:59.858+0000 7fa57d600640 -1 mon.a@0(leader).osd e75 definitely_dead 0 2024-04-25T12:37:00.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.176+0000 7fd961600640 1 -- 172.21.15.123:0/3850670820 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["6"]}]=0 marked in osd.6. v76) ==== 85+0+0 (secure 0 0 0) 0x7fd95c062820 con 0x7fd9641426a0 2024-04-25T12:37:00.176 INFO:teuthology.orchestra.run.smithi123.stderr:marked in osd.6. 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.180+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9440744e0 msgr2=0x7fd9440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.180+0000 7fd96c600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9440744e0 0x7fd9440769f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd958011340 tx=0x7fd95800b040 comp rx=0 tx=0).stop 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9641426a0 msgr2=0x7fd9640cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9641426a0 0x7fd9640cd8b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c310 tx=0x7fd95c00c7d0 comp rx=0 tx=0).stop 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 shutdown_connections 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd9440744e0 0x7fd9440769f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9641426a0 0x7fd9640cd8b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9640c8fd0 0x7fd9641e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 --2- 172.21.15.123:0/3850670820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9640c8170 0x7fd9640b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 >> 172.21.15.123:0/3850670820 conn(0x7fd96406f440 msgr2=0x7fd964158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:00.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 shutdown_connections 2024-04-25T12:37:00.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:00.181+0000 7fd96c600640 1 -- 172.21.15.123:0/3850670820 wait complete. 2024-04-25T12:37:00.226 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-04-25T12:37:05.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-25T12:37:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.500+0000 7f95ca000640 1 Processor -- start 2024-04-25T12:37:05.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.500+0000 7f95ca000640 1 -- start start 2024-04-25T12:37:05.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.502+0000 7f95ca000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f95c4152c70 0x7f95c4074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.502+0000 7f95ca000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f95c40c8170 0x7f95c415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.502+0000 7f95ca000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c40c8580 con 0x7f95c40c2790 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.502+0000 7f95ca000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f95c4158bd0 con 0x7f95c40c8170 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.502+0000 7f95ca000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f95c4158eb0 con 0x7f95c4152c70 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f95c40c8170 0x7f95c415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f95c40c8170 0x7f95c415c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:32932/0 (socket says 172.21.15.123:32932) 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3e00640 1 -- 172.21.15.123:0/2300575499 learned_addr learned my addr 172.21.15.123:0/2300575499 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3400640 1 --2- 172.21.15.123:0/2300575499 >> v2:172.21.15.153:3300/0 conn(0x7f95c4152c70 0x7f95c4074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3e00640 1 -- 172.21.15.123:0/2300575499 >> v2:172.21.15.153:3300/0 conn(0x7f95c4152c70 msgr2=0x7f95c4074820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3e00640 1 --2- 172.21.15.123:0/2300575499 >> v2:172.21.15.153:3300/0 conn(0x7f95c4152c70 0x7f95c4074820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3e00640 1 -- 172.21.15.123:0/2300575499 >> v1:172.21.15.123:6789/0 conn(0x7f95c40c2790 legacy=0x7f95c40c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.503+0000 7f95c3e00640 1 -- 172.21.15.123:0/2300575499 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c4159190 con 0x7f95c40c8170 2024-04-25T12:37:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.504+0000 7f95c3e00640 1 --2- 172.21.15.123:0/2300575499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40c8170 0x7f95c415c3b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f95c4071ae0 tx=0x7f95bc02a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2412df9c161b0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.505+0000 7f95c2a00640 1 -- 172.21.15.123:0/2300575499 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc036070 con 0x7f95c40c8170 2024-04-25T12:37:05.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.505+0000 7f95c2a00640 1 -- 172.21.15.123:0/2300575499 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc031600 con 0x7f95c40c8170 2024-04-25T12:37:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.505+0000 7f95c2a00640 1 -- 172.21.15.123:0/2300575499 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0319d0 con 0x7f95c40c8170 2024-04-25T12:37:05.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.505+0000 7f95ca000640 1 -- 172.21.15.123:0/2300575499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40c8170 msgr2=0x7f95c415c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.505+0000 7f95ca000640 1 --2- 172.21.15.123:0/2300575499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40c8170 0x7f95c415c3b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f95c4071ae0 tx=0x7f95bc02a6b0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.506+0000 7f95ca000640 1 -- 172.21.15.123:0/2300575499 shutdown_connections 2024-04-25T12:37:05.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.506+0000 7f95ca000640 1 --2- 172.21.15.123:0/2300575499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40c8170 0x7f95c415c3b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.506+0000 7f95ca000640 1 --2- 172.21.15.123:0/2300575499 >> v2:172.21.15.153:3300/0 conn(0x7f95c4152c70 0x7f95c4074820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.506+0000 7f95ca000640 1 -- 172.21.15.123:0/2300575499 >> 172.21.15.123:0/2300575499 conn(0x7f95c406f440 msgr2=0x7f95c406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:05.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.506+0000 7f95ca000640 1 -- 172.21.15.123:0/2300575499 shutdown_connections 2024-04-25T12:37:05.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.506+0000 7f95ca000640 1 -- 172.21.15.123:0/2300575499 wait complete. 2024-04-25T12:37:05.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.508+0000 7f95ca000640 1 Processor -- start 2024-04-25T12:37:05.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.508+0000 7f95ca000640 1 -- start start 2024-04-25T12:37:05.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.509+0000 7f95ca000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c40c2790 0x7f95c4145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95ca000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c40c8170 0x7f95c41f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95ca000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4152c70 0x7f95c4158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95ca000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40c46f0 con 0x7f95c4152c70 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95ca000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4159750 con 0x7f95c40c2790 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95ca000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40c4870 con 0x7f95c40c8170 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4152c70 0x7f95c4158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4152c70 0x7f95c4158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:32942/0 (socket says 172.21.15.123:32942) 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 -- 172.21.15.123:0/4056764930 learned_addr learned my addr 172.21.15.123:0/4056764930 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c40c8170 0x7f95c41f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c40c2790 0x7f95c4145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c40c8170 msgr2=0x7f95c41f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c40c8170 0x7f95c41f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c40c2790 msgr2=0x7f95c4145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c40c2790 0x7f95c4145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.510+0000 7f95c3e00640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c41f44f0 con 0x7f95c4152c70 2024-04-25T12:37:05.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.511+0000 7f95c3e00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4152c70 0x7f95c4158090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009f40 tx=0x7f95bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.511+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc036050 con 0x7f95c4152c70 2024-04-25T12:37:05.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.511+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc042070 con 0x7f95c4152c70 2024-04-25T12:37:05.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.511+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc009510 con 0x7f95c4152c70 2024-04-25T12:37:05.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.511+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c414e4b0 con 0x7f95c4152c70 2024-04-25T12:37:05.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.511+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c414ea70 con 0x7f95c4152c70 2024-04-25T12:37:05.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.512+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f95c4076700 con 0x7f95c4152c70 2024-04-25T12:37:05.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.514+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f95bc007bf0 con 0x7f95c4152c70 2024-04-25T12:37:05.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.515+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f95a00744b0 0x7f95a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.515+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6859+0+0 (secure 0 0 0) 0x7f95bc0b63a0 con 0x7f95c4152c70 2024-04-25T12:37:05.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.515+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f95bc0b8a00 con 0x7f95c4152c70 2024-04-25T12:37:05.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.515+0000 7f95c3400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f95a00744b0 0x7f95a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.515+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41f46c0 0x7f95c40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.515+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40c8d40 con 0x7f95c41f46c0 2024-04-25T12:37:05.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.516+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41f46c0 0x7f95c40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.516+0000 7f95c3400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f95a00744b0 0x7f95a00769c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f95c40c87c0 tx=0x7f95b400a2a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.516+0000 7f95bbe00640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f95c40c7e60 con 0x7f95c41f46c0 2024-04-25T12:37:05.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.516+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41f46c0 0x7f95c40cdbd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f95c4147fb0 tx=0x7f95ac00f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.517+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac019070 con 0x7f95c41f46c0 2024-04-25T12:37:05.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.517+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f95ac005320 con 0x7f95c41f46c0 2024-04-25T12:37:05.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.517+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41f46c0 msgr2=0x7f95c40cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.517+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41f46c0 0x7f95c40cdbd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f95c4147fb0 tx=0x7f95ac00f900 comp rx=0 tx=0).stop 2024-04-25T12:37:05.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.517+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41f46c0 msgr2=0x7f95c40cdbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:37:05.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.517+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c41f46c0 0x7f95c40cdbd0 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f95c4147fb0 tx=0x7f95ac00f900 comp rx=0 tx=0).stop 2024-04-25T12:37:05.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.524+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40ce110 0x7f95c4144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.524+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95c414f570 con 0x7f95c40ce110 2024-04-25T12:37:05.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.524+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40ce110 0x7f95c4144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.525+0000 7f95bbe00640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f95c414f0c0 con 0x7f95c40ce110 2024-04-25T12:37:05.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.525+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40ce110 0x7f95c4144260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f95c4147fb0 tx=0x7f95ac000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.525+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac019070 con 0x7f95c40ce110 2024-04-25T12:37:05.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.527+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.0 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f95ac006aa0 con 0x7f95c40ce110 2024-04-25T12:37:05.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.527+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40ce110 msgr2=0x7f95c4144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.527+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40ce110 0x7f95c4144260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f95c4147fb0 tx=0x7f95ac000ed0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.527 INFO:teuthology.orchestra.run.smithi123.stdout:mon.a: {} 2024-04-25T12:37:05.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.527+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40ce110 msgr2=0x7f95c4144260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.527+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c40ce110 0x7f95c4144260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.527+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c41447a0 0x7f95c41463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.527+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40744f0 con 0x7f95c41447a0 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.528+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c41447a0 0x7f95c41463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.528+0000 7f95bbe00640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f95c41585d0 con 0x7f95c41447a0 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.528+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c41447a0 0x7f95c41463d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00f940 tx=0x7f95ac009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.529+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac01b480 con 0x7f95c41447a0 2024-04-25T12:37:05.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.529+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.1 v2:172.21.15.153:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f95ac003d40 con 0x7f95c41447a0 2024-04-25T12:37:05.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.529+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c41447a0 msgr2=0x7f95c41463d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.529+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c41447a0 0x7f95c41463d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00f940 tx=0x7f95ac009040 comp rx=0 tx=0).stop 2024-04-25T12:37:05.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.529+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c41447a0 msgr2=0x7f95c41463d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:37:05.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.529+0000 7f95c3e00640 1 -- 172.21.15.123:0/4056764930 reap_dead start 2024-04-25T12:37:05.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.529+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c41447a0 0x7f95c41463d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.536+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c4145cd0 0x7f95c41f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.536+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40746a0 con 0x7f95c4145cd0 2024-04-25T12:37:05.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.536+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c4145cd0 0x7f95c41f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.537+0000 7f95bbe00640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f95c414ecb0 con 0x7f95c4145cd0 2024-04-25T12:37:05.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.537+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c4145cd0 0x7f95c41f3fb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f95ac000f10 tx=0x7f95ac025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.537+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac039770 con 0x7f95c4145cd0 2024-04-25T12:37:05.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.539+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.1 v2:172.21.15.153:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f95ac039770 con 0x7f95c4145cd0 2024-04-25T12:37:05.539 INFO:teuthology.orchestra.run.smithi123.stdout:mon.b: {} 2024-04-25T12:37:05.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.539+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c4145cd0 msgr2=0x7f95c41f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.539+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c4145cd0 0x7f95c41f3fb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f95ac000f10 tx=0x7f95ac025000 comp rx=0 tx=0).stop 2024-04-25T12:37:05.540 INFO:teuthology.orchestra.run.smithi123.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T12:37:05.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.539+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c4145cd0 msgr2=0x7f95c41f3fb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:37:05.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.539+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95c4145cd0 0x7f95c41f3fb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.539+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c4146910 0x7f95c4146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.539+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f95c41587e0 con 0x7f95c4146910 2024-04-25T12:37:05.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.540+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c4146910 0x7f95c4146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.540+0000 7f95bbe00640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f95c4147260 con 0x7f95c4146910 2024-04-25T12:37:05.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.541+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c4146910 0x7f95c4146d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ff70 tx=0x7f95ac009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.541+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac027490 con 0x7f95c4146910 2024-04-25T12:37:05.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.542+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.2 v2:172.21.15.153:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f95ac027490 con 0x7f95c4146910 2024-04-25T12:37:05.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.542+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c4146910 msgr2=0x7f95c4146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.542+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c4146910 0x7f95c4146d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ff70 tx=0x7f95ac009040 comp rx=0 tx=0).stop 2024-04-25T12:37:05.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.542+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c4146910 msgr2=0x7f95c4146d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:37:05.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.542+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c4146910 0x7f95c4146d20 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ff70 tx=0x7f95ac009040 comp rx=0 tx=0).stop 2024-04-25T12:37:05.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.549+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c41510c0 0x7f95c41514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.549+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f95c4151a10 con 0x7f95c41510c0 2024-04-25T12:37:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.549+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c41510c0 0x7f95c41514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.549+0000 7f95bbe00640 1 -- 172.21.15.123:0/4056764930 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f95c4151c10 con 0x7f95c41510c0 2024-04-25T12:37:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.550+0000 7f95bbe00640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c41510c0 0x7f95c41514d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00f210 tx=0x7f95ac03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.550+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac020040 con 0x7f95c41510c0 2024-04-25T12:37:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.552+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 <== mon.2 v2:172.21.15.153:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f95ac039d30 con 0x7f95c41510c0 2024-04-25T12:37:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.552+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c41510c0 msgr2=0x7f95c41514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.552+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c41510c0 0x7f95c41514d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00f210 tx=0x7f95ac03e000 comp rx=0 tx=0).stop 2024-04-25T12:37:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.552+0000 7f95bb400640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c41510c0 msgr2=0x7f95c41514d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T12:37:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.552+0000 7f95bb400640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95c41510c0 0x7f95c41514d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.552 INFO:teuthology.orchestra.run.smithi123.stdout:mon.c: {} 2024-04-25T12:37:05.553 INFO:teuthology.orchestra.run.smithi123.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T12:37:05.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.552+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f95a00744b0 msgr2=0x7f95a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.552+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f95a00744b0 0x7f95a00769c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f95c40c87c0 tx=0x7f95b400a2a0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.553+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4152c70 msgr2=0x7f95c4158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.553+0000 7f95ca000640 1 --2- 172.21.15.123:0/4056764930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95c4152c70 0x7f95c4158090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009f40 tx=0x7f95bc004a40 comp rx=0 tx=0).stop 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.553+0000 7f95c3e00640 1 -- 172.21.15.123:0/4056764930 reap_dead start 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.553+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 shutdown_connections 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.553+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 >> 172.21.15.123:0/4056764930 conn(0x7f95c406f440 msgr2=0x7f95c40c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.553+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 shutdown_connections 2024-04-25T12:37:05.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.553+0000 7f95ca000640 1 -- 172.21.15.123:0/4056764930 wait complete. 2024-04-25T12:37:05.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-25T12:37:05.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.857+0000 7fefa7e00640 1 Processor -- start 2024-04-25T12:37:05.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.858+0000 7fefa7e00640 1 -- start start 2024-04-25T12:37:05.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa0154650 0x7fefa015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefa00c9f00 0x7fefa00c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa7e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fefa00730e0 0x7fefa00734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa7e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fefa015b380 con 0x7fefa00c9f00 2024-04-25T12:37:05.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa7e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fefa0074b20 con 0x7fefa00730e0 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fefa00ca780 con 0x7fefa0154650 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa5400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa0154650 0x7fefa015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa4a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefa00c9f00 0x7fefa00c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa5e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fefa00730e0 0x7fefa00734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa5400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa0154650 0x7fefa015ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:45270/0 (socket says 172.21.15.123:45270) 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa5e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fefa00730e0 0x7fefa00734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:57036/0 (socket says 172.21.15.123:57036) 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa4a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefa00c9f00 0x7fefa00c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:32972/0 (socket says 172.21.15.123:32972) 2024-04-25T12:37:05.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.860+0000 7fefa5400640 1 -- 172.21.15.123:0/17017730 learned_addr learned my addr 172.21.15.123:0/17017730 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:05.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.861+0000 7fefa5400640 1 -- 172.21.15.123:0/17017730 >> v2:172.21.15.153:3300/0 conn(0x7fefa00730e0 msgr2=0x7fefa00734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:05.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.861+0000 7fefa5400640 1 --2- 172.21.15.123:0/17017730 >> v2:172.21.15.153:3300/0 conn(0x7fefa00730e0 0x7fefa00734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.861+0000 7fefa5400640 1 -- 172.21.15.123:0/17017730 >> v2:172.21.15.123:3300/0 conn(0x7fefa00c9f00 msgr2=0x7fefa00c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:05.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.861+0000 7fefa5400640 1 --2- 172.21.15.123:0/17017730 >> v2:172.21.15.123:3300/0 conn(0x7fefa00c9f00 0x7fefa00c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.861+0000 7fefa5400640 1 -- 172.21.15.123:0/17017730 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa00caa60 con 0x7fefa0154650 2024-04-25T12:37:05.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.862+0000 7fefa5400640 1 --2- 172.21.15.123:0/17017730 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa0154650 0x7fefa015ae40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fefa00c5990 tx=0x7fef9002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57eb0e4ab26c67f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.862+0000 7fef9fe00640 1 -- 172.21.15.123:0/17017730 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90037070 con 0x7fefa0154650 2024-04-25T12:37:05.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.862+0000 7fef9fe00640 1 -- 172.21.15.123:0/17017730 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef90032440 con 0x7fefa0154650 2024-04-25T12:37:05.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.862+0000 7fef9fe00640 1 -- 172.21.15.123:0/17017730 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90032810 con 0x7fefa0154650 2024-04-25T12:37:05.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 -- 172.21.15.123:0/17017730 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa0154650 msgr2=0x7fefa015ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 --2- 172.21.15.123:0/17017730 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa0154650 0x7fefa015ae40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fefa00c5990 tx=0x7fef9002a270 comp rx=0 tx=0).stop 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 -- 172.21.15.123:0/17017730 shutdown_connections 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 --2- 172.21.15.123:0/17017730 >> v2:172.21.15.153:3300/0 conn(0x7fefa00730e0 0x7fefa00734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 --2- 172.21.15.123:0/17017730 >> v2:172.21.15.123:3300/0 conn(0x7fefa00c9f00 0x7fefa00c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 --2- 172.21.15.123:0/17017730 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa0154650 0x7fefa015ae40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 -- 172.21.15.123:0/17017730 >> 172.21.15.123:0/17017730 conn(0x7fefa006f440 msgr2=0x7fefa006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 -- 172.21.15.123:0/17017730 shutdown_connections 2024-04-25T12:37:05.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.863+0000 7fefa7e00640 1 -- 172.21.15.123:0/17017730 wait complete. 2024-04-25T12:37:05.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.866+0000 7fefa7e00640 1 Processor -- start 2024-04-25T12:37:05.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.866+0000 7fefa7e00640 1 -- start start 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.868+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa00730e0 0x7fefa00b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.868+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa00c9f00 0x7fefa0159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.868+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa0154650 0x7fefa0146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.868+0000 7fefa7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fefa015e010 con 0x7fefa0154650 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.868+0000 7fefa7e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fefa015dd30 con 0x7fefa00c9f00 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.868+0000 7fefa7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fefa015e2f0 con 0x7fefa00730e0 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.868+0000 7fefa5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa0154650 0x7fefa0146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa0154650 0x7fefa0146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:32974/0 (socket says 172.21.15.123:32974) 2024-04-25T12:37:05.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5e00640 1 -- 172.21.15.123:0/1297236385 learned_addr learned my addr 172.21.15.123:0/1297236385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5400640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa00730e0 0x7fefa00b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa4a00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa00c9f00 0x7fefa0159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5400640 1 -- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa00c9f00 msgr2=0x7fefa0159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5400640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa00c9f00 0x7fefa0159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5400640 1 -- 172.21.15.123:0/1297236385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa0154650 msgr2=0x7fefa0146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5400640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa0154650 0x7fefa0146eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5400640 1 -- 172.21.15.123:0/1297236385 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa00cc540 con 0x7fefa00730e0 2024-04-25T12:37:05.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5e00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa0154650 0x7fefa0146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:05.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.869+0000 7fefa5400640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa00730e0 0x7fefa00b17a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fef90031000 tx=0x7fef900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.870+0000 7fef9d600640 1 -- 172.21.15.123:0/1297236385 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90041020 con 0x7fefa00730e0 2024-04-25T12:37:05.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.870+0000 7fef9d600640 1 -- 172.21.15.123:0/1297236385 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9003d660 con 0x7fefa00730e0 2024-04-25T12:37:05.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.870+0000 7fef9d600640 1 -- 172.21.15.123:0/1297236385 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90037040 con 0x7fefa00730e0 2024-04-25T12:37:05.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.870+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa0152100 con 0x7fefa00730e0 2024-04-25T12:37:05.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.870+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa00cc020 con 0x7fefa00730e0 2024-04-25T12:37:05.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.871+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa0076700 con 0x7fefa00730e0 2024-04-25T12:37:05.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.873+0000 7fef9d600640 1 -- 172.21.15.123:0/1297236385 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fef90044020 con 0x7fefa00730e0 2024-04-25T12:37:05.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.874+0000 7fef9d600640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fef800744b0 0x7fef800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:05.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.874+0000 7fef9d600640 1 -- 172.21.15.123:0/1297236385 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(77..77 src has 1..77) ==== 6859+0+0 (secure 0 0 0) 0x7fef900b5fd0 con 0x7fefa00730e0 2024-04-25T12:37:05.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.874+0000 7fefa4a00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fef800744b0 0x7fef800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:05.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.875+0000 7fefa4a00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fef800744b0 0x7fef800769c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fefa00c7fd0 tx=0x7fef8c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:05.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:05.877+0000 7fef9d600640 1 -- 172.21.15.123:0/1297236385 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef9004b030 con 0x7fefa00730e0 2024-04-25T12:37:06.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.091+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7fefa006d320 con 0x7fefa00730e0 2024-04-25T12:37:06.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.213+0000 7fef9d600640 1 -- 172.21.15.123:0/1297236385 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v78) ==== 113+0+0 (secure 0 0 0) 0x7fef90088340 con 0x7fefa00730e0 2024-04-25T12:37:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.0 to 1 (10000) 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.217+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fef800744b0 msgr2=0x7fef800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.217+0000 7fefa7e00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fef800744b0 0x7fef800769c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fefa00c7fd0 tx=0x7fef8c00b040 comp rx=0 tx=0).stop 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.217+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa00730e0 msgr2=0x7fefa00b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.217+0000 7fefa7e00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa00730e0 0x7fefa00b17a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fef90031000 tx=0x7fef900046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 shutdown_connections 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fef800744b0 0x7fef800769c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa0154650 0x7fefa0146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefa00c9f00 0x7fefa0159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 --2- 172.21.15.123:0/1297236385 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fefa00730e0 0x7fefa00b17a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 >> 172.21.15.123:0/1297236385 conn(0x7fefa006f440 msgr2=0x7fefa00c85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:06.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 shutdown_connections 2024-04-25T12:37:06.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.218+0000 7fefa7e00640 1 -- 172.21.15.123:0/1297236385 wait complete. 2024-04-25T12:37:06.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-25T12:37:06.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.518+0000 7ff793000640 1 Processor -- start 2024-04-25T12:37:06.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.518+0000 7ff793000640 1 -- start start 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff793000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff793000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff78c072cc0 0x7ff78c06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff793000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff78c0730d0 0x7ff78c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff793000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c074290 con 0x7ff78c072cc0 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff793000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c074570 con 0x7ff78c0730d0 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff793000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff78c074850 con 0x7ff78c0c7e30 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff78c072cc0 0x7ff78c06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff78c072cc0 0x7ff78c06c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:32986/0 (socket says 172.21.15.123:32986) 2024-04-25T12:37:06.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 -- 172.21.15.123:0/4006987197 learned_addr learned my addr 172.21.15.123:0/4006987197 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791c00640 1 --2- 172.21.15.123:0/4006987197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff792600640 1 --2- 172.21.15.123:0/4006987197 >> v2:172.21.15.153:3300/0 conn(0x7ff78c0730d0 0x7ff78c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 -- 172.21.15.123:0/4006987197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 msgr2=0x7ff78c154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 --2- 172.21.15.123:0/4006987197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 -- 172.21.15.123:0/4006987197 >> v2:172.21.15.153:3300/0 conn(0x7ff78c0730d0 msgr2=0x7ff78c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 --2- 172.21.15.123:0/4006987197 >> v2:172.21.15.153:3300/0 conn(0x7ff78c0730d0 0x7ff78c06ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.520+0000 7ff791200640 1 -- 172.21.15.123:0/4006987197 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c074b30 con 0x7ff78c072cc0 2024-04-25T12:37:06.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.521+0000 7ff791200640 1 --2- 172.21.15.123:0/4006987197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c072cc0 0x7ff78c06c7b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0cb290 tx=0x7ff77802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9026fec5a29152 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.522+0000 7ff78be00640 1 -- 172.21.15.123:0/4006987197 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778037070 con 0x7ff78c072cc0 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.522+0000 7ff78be00640 1 -- 172.21.15.123:0/4006987197 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff778032440 con 0x7ff78c072cc0 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.522+0000 7ff78be00640 1 -- 172.21.15.123:0/4006987197 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778032810 con 0x7ff78c072cc0 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.522+0000 7ff793000640 1 -- 172.21.15.123:0/4006987197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c072cc0 msgr2=0x7ff78c06c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.522+0000 7ff793000640 1 --2- 172.21.15.123:0/4006987197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c072cc0 0x7ff78c06c7b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0cb290 tx=0x7ff77802a1e0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.523+0000 7ff793000640 1 -- 172.21.15.123:0/4006987197 shutdown_connections 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.523+0000 7ff793000640 1 --2- 172.21.15.123:0/4006987197 >> v2:172.21.15.153:3300/0 conn(0x7ff78c0730d0 0x7ff78c06ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.523+0000 7ff793000640 1 --2- 172.21.15.123:0/4006987197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c072cc0 0x7ff78c06c7b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.523+0000 7ff793000640 1 --2- 172.21.15.123:0/4006987197 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.523+0000 7ff793000640 1 -- 172.21.15.123:0/4006987197 >> 172.21.15.123:0/4006987197 conn(0x7ff78c070b70 msgr2=0x7ff78c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.523+0000 7ff793000640 1 -- 172.21.15.123:0/4006987197 shutdown_connections 2024-04-25T12:37:06.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.523+0000 7ff793000640 1 -- 172.21.15.123:0/4006987197 wait complete. 2024-04-25T12:37:06.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.525+0000 7ff793000640 1 Processor -- start 2024-04-25T12:37:06.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.525+0000 7ff793000640 1 -- start start 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.526+0000 7ff793000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff78c072cc0 0x7ff78c1e4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.526+0000 7ff793000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c0730d0 0x7ff78c1f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.526+0000 7ff793000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c14edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.526+0000 7ff793000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c0c8240 con 0x7ff78c0730d0 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.526+0000 7ff793000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1547c0 con 0x7ff78c072cc0 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.526+0000 7ff793000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff78c0c83c0 con 0x7ff78c0c7e30 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c0730d0 0x7ff78c1f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c0730d0 0x7ff78c1f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:32994/0 (socket says 172.21.15.123:32994) 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 -- 172.21.15.123:0/4290966096 learned_addr learned my addr 172.21.15.123:0/4290966096 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791c00640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff78c072cc0 0x7ff78c1e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:06.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff792600640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c14edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 -- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 msgr2=0x7ff78c14edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c14edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 -- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff78c072cc0 msgr2=0x7ff78c1e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff78c072cc0 0x7ff78c1e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 -- 172.21.15.123:0/4290966096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c1f24f0 con 0x7ff78c0730d0 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff792600640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c14edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.527+0000 7ff791200640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c0730d0 0x7ff78c1f29b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff778000c00 tx=0x7ff7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.528+0000 7ff789600640 1 -- 172.21.15.123:0/4290966096 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778041020 con 0x7ff78c0730d0 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.528+0000 7ff789600640 1 -- 172.21.15.123:0/4290966096 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77803d620 con 0x7ff78c0730d0 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.528+0000 7ff789600640 1 -- 172.21.15.123:0/4290966096 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778037040 con 0x7ff78c0730d0 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.528+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c1f2ef0 con 0x7ff78c0730d0 2024-04-25T12:37:06.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.528+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c156c60 con 0x7ff78c0730d0 2024-04-25T12:37:06.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.529+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c076da0 con 0x7ff78c0730d0 2024-04-25T12:37:06.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.530+0000 7ff789600640 1 -- 172.21.15.123:0/4290966096 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff778044020 con 0x7ff78c0730d0 2024-04-25T12:37:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.531+0000 7ff789600640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff76c0744e0 0x7ff76c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.531+0000 7ff789600640 1 -- 172.21.15.123:0/4290966096 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6859+0+0 (secure 0 0 0) 0x7ff7780b5ce0 con 0x7ff78c0730d0 2024-04-25T12:37:06.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.531+0000 7ff791c00640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff76c0744e0 0x7ff76c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:06.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.532+0000 7ff791c00640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff76c0744e0 0x7ff76c0769f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff780009a20 tx=0x7ff780009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:06.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.536+0000 7ff789600640 1 -- 172.21.15.123:0/4290966096 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff77804b030 con 0x7ff78c0730d0 2024-04-25T12:37:06.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:06.748+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) -- 0x7ff78c073690 con 0x7ff78c0730d0 2024-04-25T12:37:07.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.212+0000 7ff789600640 1 -- 172.21.15.123:0/4290966096 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v79) ==== 133+0+0 (secure 0 0 0) 0x7ff778084e20 con 0x7ff78c0730d0 2024-04-25T12:37:07.213 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-25T12:37:07.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff76c0744e0 msgr2=0x7ff76c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:07.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff76c0744e0 0x7ff76c0769f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff780009a20 tx=0x7ff780009190 comp rx=0 tx=0).stop 2024-04-25T12:37:07.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c0730d0 msgr2=0x7ff78c1f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c0730d0 0x7ff78c1f29b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff778000c00 tx=0x7ff7780046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 shutdown_connections 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff76c0744e0 0x7ff76c0769f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff78c0c7e30 0x7ff78c14edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c0730d0 0x7ff78c1f29b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 --2- 172.21.15.123:0/4290966096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff78c072cc0 0x7ff78c1e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 >> 172.21.15.123:0/4290966096 conn(0x7ff78c070b70 msgr2=0x7ff78c15bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 shutdown_connections 2024-04-25T12:37:07.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.216+0000 7ff793000640 1 -- 172.21.15.123:0/4290966096 wait complete. 2024-04-25T12:37:07.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-25T12:37:07.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.531+0000 7fee70600640 1 Processor -- start 2024-04-25T12:37:07.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.532+0000 7fee70600640 1 -- start start 2024-04-25T12:37:07.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee70600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6815ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:07.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee70600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee680c9f00 0x7fee680c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:07.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee70600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee6815b380 con 0x7fee680730e0 2024-04-25T12:37:07.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee70600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fee68074b20 con 0x7fee680c9f00 2024-04-25T12:37:07.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee70600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fee680ca780 con 0x7fee68154650 2024-04-25T12:37:07.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee6dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6815ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:07.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee6d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee680c9f00 0x7fee680c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:07.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee6dc00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6815ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:45306/0 (socket says 172.21.15.123:45306) 2024-04-25T12:37:07.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.534+0000 7fee6dc00640 1 -- 172.21.15.123:0/1075414264 learned_addr learned my addr 172.21.15.123:0/1075414264 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.535+0000 7fee6d200640 1 -- 172.21.15.123:0/1075414264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 msgr2=0x7fee6815ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.535+0000 7fee6d200640 1 --2- 172.21.15.123:0/1075414264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6815ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.535+0000 7fee6d200640 1 -- 172.21.15.123:0/1075414264 >> v1:172.21.15.123:6789/0 conn(0x7fee680730e0 legacy=0x7fee680708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.535+0000 7fee6d200640 1 -- 172.21.15.123:0/1075414264 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee680caa60 con 0x7fee680c9f00 2024-04-25T12:37:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.535+0000 7fee67e00640 1 -- 172.21.15.123:0/1075414264 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062678358 0 0) 0x7fee60001490 con 0x7fee680730e0 2024-04-25T12:37:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.535+0000 7fee6dc00640 1 --2- 172.21.15.123:0/1075414264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6815ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.536+0000 7fee6d200640 1 --2- 172.21.15.123:0/1075414264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680c9f00 0x7fee680c8170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee540097a0 tx=0x7fee5402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23306961f24c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.536+0000 7fee67e00640 1 -- 172.21.15.123:0/1075414264 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54037070 con 0x7fee680c9f00 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.536+0000 7fee67e00640 1 -- 172.21.15.123:0/1075414264 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee54032440 con 0x7fee680c9f00 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.536+0000 7fee67e00640 1 -- 172.21.15.123:0/1075414264 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54032810 con 0x7fee680c9f00 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.537+0000 7fee70600640 1 -- 172.21.15.123:0/1075414264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680c9f00 msgr2=0x7fee680c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.537+0000 7fee70600640 1 --2- 172.21.15.123:0/1075414264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680c9f00 0x7fee680c8170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee540097a0 tx=0x7fee5402a210 comp rx=0 tx=0).stop 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.537+0000 7fee70600640 1 -- 172.21.15.123:0/1075414264 shutdown_connections 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.537+0000 7fee70600640 1 --2- 172.21.15.123:0/1075414264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680c9f00 0x7fee680c8170 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.537+0000 7fee70600640 1 --2- 172.21.15.123:0/1075414264 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6815ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.537+0000 7fee70600640 1 -- 172.21.15.123:0/1075414264 >> 172.21.15.123:0/1075414264 conn(0x7fee6806f440 msgr2=0x7fee6806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.538+0000 7fee70600640 1 -- 172.21.15.123:0/1075414264 shutdown_connections 2024-04-25T12:37:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.538+0000 7fee70600640 1 -- 172.21.15.123:0/1075414264 wait complete. 2024-04-25T12:37:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.540+0000 7fee70600640 1 Processor -- start 2024-04-25T12:37:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.540+0000 7fee70600640 1 -- start start 2024-04-25T12:37:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee70600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680730e0 0x7fee680cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee70600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee680c9f00 0x7fee6814bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee70600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee70600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee6815e010 con 0x7fee680730e0 2024-04-25T12:37:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee70600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee6815dd30 con 0x7fee680c9f00 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee70600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fee6815e2f0 con 0x7fee68154650 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680730e0 0x7fee680cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680730e0 0x7fee680cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33014/0 (socket says 172.21.15.123:33014) 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6dc00640 1 -- 172.21.15.123:0/1427707071 learned_addr learned my addr 172.21.15.123:0/1427707071 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6e600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6d200640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee680c9f00 0x7fee6814bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6dc00640 1 -- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 msgr2=0x7fee6814ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6dc00640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6814ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6dc00640 1 -- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee680c9f00 msgr2=0x7fee6814bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.542+0000 7fee6dc00640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee680c9f00 0x7fee6814bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.543+0000 7fee6dc00640 1 -- 172.21.15.123:0/1427707071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee680cc2b0 con 0x7fee680730e0 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.543+0000 7fee6dc00640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680730e0 0x7fee680cbd70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00e9e0 tx=0x7fee5c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.543+0000 7fee65600640 1 -- 172.21.15.123:0/1427707071 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c017070 con 0x7fee680730e0 2024-04-25T12:37:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.543+0000 7fee65600640 1 -- 172.21.15.123:0/1427707071 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c00c9d0 con 0x7fee680730e0 2024-04-25T12:37:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.543+0000 7fee65600640 1 -- 172.21.15.123:0/1427707071 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00cd80 con 0x7fee680730e0 2024-04-25T12:37:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.543+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6814c290 con 0x7fee680730e0 2024-04-25T12:37:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.543+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6815bc70 con 0x7fee680730e0 2024-04-25T12:37:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.544+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee68076700 con 0x7fee680730e0 2024-04-25T12:37:07.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.546+0000 7fee65600640 1 -- 172.21.15.123:0/1427707071 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee5c01a070 con 0x7fee680730e0 2024-04-25T12:37:07.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.547+0000 7fee65600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee480744e0 0x7fee480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:07.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.547+0000 7fee65600640 1 -- 172.21.15.123:0/1427707071 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6891+0+0 (secure 0 0 0) 0x7fee5c0955b0 con 0x7fee680730e0 2024-04-25T12:37:07.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.547+0000 7fee6d200640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee480744e0 0x7fee480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:07.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.548+0000 7fee6d200640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee480744e0 0x7fee480769f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee54034000 tx=0x7fee5402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:07.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.553+0000 7fee65600640 1 -- 172.21.15.123:0/1427707071 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee5c05f1b0 con 0x7fee680730e0 2024-04-25T12:37:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:07.766+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7fee680c6770 con 0x7fee680730e0 2024-04-25T12:37:07.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:37:07.904 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:37:08.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.226+0000 7fee65600640 1 -- 172.21.15.123:0/1427707071 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v80) ==== 113+0+0 (secure 0 0 0) 0x7fee5c0634b0 con 0x7fee680730e0 2024-04-25T12:37:08.227 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.1 to 1 (10000) 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee480744e0 msgr2=0x7fee480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee480744e0 0x7fee480769f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee54034000 tx=0x7fee5402e000 comp rx=0 tx=0).stop 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680730e0 msgr2=0x7fee680cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680730e0 0x7fee680cbd70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00e9e0 tx=0x7fee5c00eea0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 shutdown_connections 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fee480744e0 0x7fee480769f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee68154650 0x7fee6814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee680c9f00 0x7fee6814bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 --2- 172.21.15.123:0/1427707071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee680730e0 0x7fee680cbd70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.230+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 >> 172.21.15.123:0/1427707071 conn(0x7fee6806f440 msgr2=0x7fee680c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:08.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.231+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 shutdown_connections 2024-04-25T12:37:08.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.231+0000 7fee70600640 1 -- 172.21.15.123:0/1427707071 wait complete. 2024-04-25T12:37:08.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-25T12:37:08.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.535+0000 7fa2dac00640 1 Processor -- start 2024-04-25T12:37:08.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.535+0000 7fa2dac00640 1 -- start start 2024-04-25T12:37:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2dac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c7dc0 0x7fa2d41565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2dac00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2d40c2720 0x7fa2d406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2dac00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d40c81d0 con 0x7fa2d406ccf0 2024-04-25T12:37:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2dac00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d40c8450 con 0x7fa2d40c2720 2024-04-25T12:37:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2dac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d4159310 con 0x7fa2d40c7dc0 2024-04-25T12:37:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2d9800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c7dc0 0x7fa2d41565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2da200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa2d406ccf0 0x7fa2d40c91c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54676/0 (socket says 172.21.15.123:54676) 2024-04-25T12:37:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2d8e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2d40c2720 0x7fa2d406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.537+0000 7fa2da200640 1 -- 172.21.15.123:0/1414234710 learned_addr learned my addr 172.21.15.123:0/1414234710 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.538+0000 7fa2d9800640 1 -- 172.21.15.123:0/1414234710 >> v2:172.21.15.153:3300/0 conn(0x7fa2d40c2720 msgr2=0x7fa2d406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.538+0000 7fa2d9800640 1 --2- 172.21.15.123:0/1414234710 >> v2:172.21.15.153:3300/0 conn(0x7fa2d40c2720 0x7fa2d406c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.538+0000 7fa2d9800640 1 -- 172.21.15.123:0/1414234710 >> v1:172.21.15.123:6789/0 conn(0x7fa2d406ccf0 legacy=0x7fa2d40c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.538+0000 7fa2d9800640 1 -- 172.21.15.123:0/1414234710 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41595f0 con 0x7fa2d40c7dc0 2024-04-25T12:37:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.538+0000 7fa2d3e00640 1 -- 172.21.15.123:0/1414234710 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2105354582 0 0) 0x7fa2cc001110 con 0x7fa2d406ccf0 2024-04-25T12:37:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.538+0000 7fa2d8e00640 1 --2- 172.21.15.123:0/1414234710 >> v2:172.21.15.153:3300/0 conn(0x7fa2d40c2720 0x7fa2d406c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.539+0000 7fa2d9800640 1 --2- 172.21.15.123:0/1414234710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c7dc0 0x7fa2d41565f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40097a0 tx=0x7fa2c402c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd03e4cba1385ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.539+0000 7fa2d3e00640 1 -- 172.21.15.123:0/1414234710 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4038070 con 0x7fa2d40c7dc0 2024-04-25T12:37:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.539+0000 7fa2d3e00640 1 -- 172.21.15.123:0/1414234710 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c4031710 con 0x7fa2d40c7dc0 2024-04-25T12:37:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.539+0000 7fa2d3e00640 1 -- 172.21.15.123:0/1414234710 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4031ae0 con 0x7fa2d40c7dc0 2024-04-25T12:37:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 -- 172.21.15.123:0/1414234710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c7dc0 msgr2=0x7fa2d41565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 --2- 172.21.15.123:0/1414234710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c7dc0 0x7fa2d41565f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40097a0 tx=0x7fa2c402c420 comp rx=0 tx=0).stop 2024-04-25T12:37:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 -- 172.21.15.123:0/1414234710 shutdown_connections 2024-04-25T12:37:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 --2- 172.21.15.123:0/1414234710 >> v2:172.21.15.153:3300/0 conn(0x7fa2d40c2720 0x7fa2d406c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 --2- 172.21.15.123:0/1414234710 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c7dc0 0x7fa2d41565f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 -- 172.21.15.123:0/1414234710 >> 172.21.15.123:0/1414234710 conn(0x7fa2d4070b70 msgr2=0x7fa2d4070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 -- 172.21.15.123:0/1414234710 shutdown_connections 2024-04-25T12:37:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.540+0000 7fa2dac00640 1 -- 172.21.15.123:0/1414234710 wait complete. 2024-04-25T12:37:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.542+0000 7fa2dac00640 1 Processor -- start 2024-04-25T12:37:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.543+0000 7fa2dac00640 1 -- start start 2024-04-25T12:37:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2dac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d406ccf0 0x7fa2d41f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2dac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c2720 0x7fa2d414df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2dac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40c7dc0 0x7fa2d414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2dac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d40c4cb0 con 0x7fa2d40c7dc0 2024-04-25T12:37:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2dac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4159bb0 con 0x7fa2d406ccf0 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2dac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d4158fc0 con 0x7fa2d40c2720 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2d9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d406ccf0 0x7fa2d41f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2d9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d406ccf0 0x7fa2d41f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:57104/0 (socket says 172.21.15.123:57104) 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2d9800640 1 -- 172.21.15.123:0/86670226 learned_addr learned my addr 172.21.15.123:0/86670226 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2da200640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40c7dc0 0x7fa2d414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.544+0000 7fa2d8e00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c2720 0x7fa2d414df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.545+0000 7fa2d9800640 1 -- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c2720 msgr2=0x7fa2d414df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.545+0000 7fa2d9800640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c2720 0x7fa2d414df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.545+0000 7fa2d9800640 1 -- 172.21.15.123:0/86670226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40c7dc0 msgr2=0x7fa2d414cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.545+0000 7fa2d9800640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40c7dc0 0x7fa2d414cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.545+0000 7fa2d9800640 1 -- 172.21.15.123:0/86670226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41f2fa0 con 0x7fa2d406ccf0 2024-04-25T12:37:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.545+0000 7fa2d9800640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d406ccf0 0x7fa2d41f2a60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4036040 tx=0x7fa2c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.546+0000 7fa2d1600640 1 -- 172.21.15.123:0/86670226 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4038040 con 0x7fa2d406ccf0 2024-04-25T12:37:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.546+0000 7fa2d1600640 1 -- 172.21.15.123:0/86670226 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c4004030 con 0x7fa2d406ccf0 2024-04-25T12:37:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.546+0000 7fa2d1600640 1 -- 172.21.15.123:0/86670226 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4006e10 con 0x7fa2d406ccf0 2024-04-25T12:37:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.546+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d41e5910 con 0x7fa2d406ccf0 2024-04-25T12:37:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.546+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41e61f0 con 0x7fa2d406ccf0 2024-04-25T12:37:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.547+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d4076da0 con 0x7fa2d406ccf0 2024-04-25T12:37:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.548+0000 7fa2d1600640 1 -- 172.21.15.123:0/86670226 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa2c4033830 con 0x7fa2d406ccf0 2024-04-25T12:37:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.549+0000 7fa2d1600640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa2b80744b0 0x7fa2b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.549+0000 7fa2d1600640 1 -- 172.21.15.123:0/86670226 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6891+0+0 (secure 0 0 0) 0x7fa2c40b6130 con 0x7fa2d406ccf0 2024-04-25T12:37:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.549+0000 7fa2d8e00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa2b80744b0 0x7fa2b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:08.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.550+0000 7fa2d8e00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa2b80744b0 0x7fa2b80769c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40c6650 tx=0x7fa2c800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:08.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.554+0000 7fa2d1600640 1 -- 172.21.15.123:0/86670226 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2c407fdc0 con 0x7fa2d406ccf0 2024-04-25T12:37:08.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:08.778+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) -- 0x7fa2d40ce310 con 0x7fa2d406ccf0 2024-04-25T12:37:09.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.244+0000 7fa2d1600640 1 -- 172.21.15.123:0/86670226 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v81) ==== 133+0+0 (secure 0 0 0) 0x7fa2c40840c0 con 0x7fa2d406ccf0 2024-04-25T12:37:09.245 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-25T12:37:09.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa2b80744b0 msgr2=0x7fa2b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa2b80744b0 0x7fa2b80769c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40c6650 tx=0x7fa2c800b040 comp rx=0 tx=0).stop 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d406ccf0 msgr2=0x7fa2d41f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d406ccf0 0x7fa2d41f2a60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4036040 tx=0x7fa2c40046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 shutdown_connections 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa2b80744b0 0x7fa2b80769c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40c7dc0 0x7fa2d414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2d40c2720 0x7fa2d414df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 --2- 172.21.15.123:0/86670226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d406ccf0 0x7fa2d41f2a60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 >> 172.21.15.123:0/86670226 conn(0x7fa2d4070b70 msgr2=0x7fa2d40c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.248+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 shutdown_connections 2024-04-25T12:37:09.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.249+0000 7fa2dac00640 1 -- 172.21.15.123:0/86670226 wait complete. 2024-04-25T12:37:09.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-25T12:37:09.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.553+0000 7fe24f600640 1 Processor -- start 2024-04-25T12:37:09.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.553+0000 7fe24f600640 1 -- start start 2024-04-25T12:37:09.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24f600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe2481426a0 0x7fe248159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:09.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24f600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2480c8170 0x7fe2480ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:09.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24f600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe248074400 con 0x7fe2480c8fd0 2024-04-25T12:37:09.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24f600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe2481595e0 con 0x7fe2480c8170 2024-04-25T12:37:09.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24f600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe248074670 con 0x7fe2481426a0 2024-04-25T12:37:09.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24cc00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe2480c8fd0 0x7fe248153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54690/0 (socket says 172.21.15.123:54690) 2024-04-25T12:37:09.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24cc00640 1 -- 172.21.15.123:0/3642482369 learned_addr learned my addr 172.21.15.123:0/3642482369 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:09.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe24d600640 1 --2- 172.21.15.123:0/3642482369 >> v2:172.21.15.123:3300/0 conn(0x7fe2480c8170 0x7fe2480ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:09.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.555+0000 7fe247e00640 1 --2- 172.21.15.123:0/3642482369 >> v2:172.21.15.153:3300/0 conn(0x7fe2481426a0 0x7fe248159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.556+0000 7fe247400640 1 -- 172.21.15.123:0/3642482369 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157015798 0 0) 0x7fe248074400 con 0x7fe2480c8fd0 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.556+0000 7fe247e00640 1 -- 172.21.15.123:0/3642482369 >> v2:172.21.15.123:3300/0 conn(0x7fe2480c8170 msgr2=0x7fe2480ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.556+0000 7fe247e00640 1 --2- 172.21.15.123:0/3642482369 >> v2:172.21.15.123:3300/0 conn(0x7fe2480c8170 0x7fe2480ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.556+0000 7fe247e00640 1 -- 172.21.15.123:0/3642482369 >> v1:172.21.15.123:6789/0 conn(0x7fe2480c8fd0 legacy=0x7fe248153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.556+0000 7fe247e00640 1 -- 172.21.15.123:0/3642482369 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe248074950 con 0x7fe2481426a0 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.556+0000 7fe24d600640 1 --2- 172.21.15.123:0/3642482369 >> v2:172.21.15.123:3300/0 conn(0x7fe2480c8170 0x7fe2480ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.557+0000 7fe247e00640 1 --2- 172.21.15.123:0/3642482369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2481426a0 0x7fe248159dc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe248068e80 tx=0x7fe23402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dead1418ef70c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.557+0000 7fe247400640 1 -- 172.21.15.123:0/3642482369 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234037070 con 0x7fe2481426a0 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.557+0000 7fe247400640 1 -- 172.21.15.123:0/3642482369 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe234032440 con 0x7fe2481426a0 2024-04-25T12:37:09.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.557+0000 7fe247400640 1 -- 172.21.15.123:0/3642482369 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234032810 con 0x7fe2481426a0 2024-04-25T12:37:09.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 -- 172.21.15.123:0/3642482369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2481426a0 msgr2=0x7fe248159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:09.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 --2- 172.21.15.123:0/3642482369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2481426a0 0x7fe248159dc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe248068e80 tx=0x7fe23402a1a0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 -- 172.21.15.123:0/3642482369 shutdown_connections 2024-04-25T12:37:09.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 --2- 172.21.15.123:0/3642482369 >> v2:172.21.15.123:3300/0 conn(0x7fe2480c8170 0x7fe2480ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 --2- 172.21.15.123:0/3642482369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2481426a0 0x7fe248159dc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 -- 172.21.15.123:0/3642482369 >> 172.21.15.123:0/3642482369 conn(0x7fe24806f440 msgr2=0x7fe24806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:09.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 -- 172.21.15.123:0/3642482369 shutdown_connections 2024-04-25T12:37:09.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.558+0000 7fe24f600640 1 -- 172.21.15.123:0/3642482369 wait complete. 2024-04-25T12:37:09.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.560+0000 7fe24f600640 1 Processor -- start 2024-04-25T12:37:09.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.561+0000 7fe24f600640 1 -- start start 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24f600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2480c8170 0x7fe2481f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2480c8fd0 0x7fe2481e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24f600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2481426a0 0x7fe2480c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2480c4880 con 0x7fe2480c8fd0 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24f600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe24815a300 con 0x7fe2480c8170 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24f600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe2480c8580 con 0x7fe2481426a0 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2480c8170 0x7fe2481f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2480c8170 0x7fe2481f3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:57128/0 (socket says 172.21.15.123:57128) 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24d600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2481426a0 0x7fe2480c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe24cc00640 1 -- 172.21.15.123:0/1613097257 learned_addr learned my addr 172.21.15.123:0/1613097257 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.562+0000 7fe247e00640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2480c8fd0 0x7fe2481e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24d600640 1 -- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2480c8170 msgr2=0x7fe2481f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24d600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2480c8170 0x7fe2481f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24d600640 1 -- 172.21.15.123:0/1613097257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2480c8fd0 msgr2=0x7fe2481e7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:09.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24d600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2480c8fd0 0x7fe2481e7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:09.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24d600640 1 -- 172.21.15.123:0/1613097257 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2481f4410 con 0x7fe2481426a0 2024-04-25T12:37:09.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24cc00640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2480c8170 0x7fe2481f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:09.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24d600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2481426a0 0x7fe2480c6b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe24000ef90 tx=0x7fe24000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:09.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe244c00640 1 -- 172.21.15.123:0/1613097257 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240018030 con 0x7fe2481426a0 2024-04-25T12:37:09.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe244c00640 1 -- 172.21.15.123:0/1613097257 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24000cd30 con 0x7fe2481426a0 2024-04-25T12:37:09.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2481f5fc0 con 0x7fe2481426a0 2024-04-25T12:37:09.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe244c00640 1 -- 172.21.15.123:0/1613097257 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240003bb0 con 0x7fe2481426a0 2024-04-25T12:37:09.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.563+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2481f6580 con 0x7fe2481426a0 2024-04-25T12:37:09.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.565+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe248076700 con 0x7fe2481426a0 2024-04-25T12:37:09.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.566+0000 7fe244c00640 1 -- 172.21.15.123:0/1613097257 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe24001b070 con 0x7fe2481426a0 2024-04-25T12:37:09.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.566+0000 7fe244c00640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2280744e0 0x7fe2280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:09.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.567+0000 7fe244c00640 1 -- 172.21.15.123:0/1613097257 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(81..81 src has 1..81) ==== 6891+0+0 (secure 0 0 0) 0x7fe240095f60 con 0x7fe2481426a0 2024-04-25T12:37:09.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.567+0000 7fe24cc00640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2280744e0 0x7fe2280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:09.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.567+0000 7fe24cc00640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2280744e0 0x7fe2280769f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe23c000f60 tx=0x7fe23c007ea0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:09.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.572+0000 7fe244c00640 1 -- 172.21.15.123:0/1613097257 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe240010040 con 0x7fe2481426a0 2024-04-25T12:37:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:09.786+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7fe2480cc8d0 con 0x7fe2481426a0 2024-04-25T12:37:10.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.258+0000 7fe244c00640 1 -- 172.21.15.123:0/1613097257 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v82) ==== 113+0+0 (secure 0 0 0) 0x7fe24005fb60 con 0x7fe2481426a0 2024-04-25T12:37:10.259 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.2 to 1 (10000) 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.263+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2280744e0 msgr2=0x7fe2280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.263+0000 7fe24f600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2280744e0 0x7fe2280769f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe23c000f60 tx=0x7fe23c007ea0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.263+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2481426a0 msgr2=0x7fe2480c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.263+0000 7fe24f600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2481426a0 0x7fe2480c6b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe24000ef90 tx=0x7fe24000c490 comp rx=0 tx=0).stop 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.264+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 shutdown_connections 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.264+0000 7fe24f600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2280744e0 0x7fe2280769f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.264+0000 7fe24f600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2481426a0 0x7fe2480c6b40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.264+0000 7fe24f600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2480c8fd0 0x7fe2481e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.264+0000 7fe24f600640 1 --2- 172.21.15.123:0/1613097257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2480c8170 0x7fe2481f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.264+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 >> 172.21.15.123:0/1613097257 conn(0x7fe24806f440 msgr2=0x7fe248073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:10.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.265+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 shutdown_connections 2024-04-25T12:37:10.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.265+0000 7fe24f600640 1 -- 172.21.15.123:0/1613097257 wait complete. 2024-04-25T12:37:10.293 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-25T12:37:10.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.558+0000 7f8e5c600640 1 Processor -- start 2024-04-25T12:37:10.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.558+0000 7f8e5c600640 1 -- start start 2024-04-25T12:37:10.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e5c600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e54154650 0x7f8e5415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:10.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e5c600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e540730e0 0x7f8e540734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:10.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e5c600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5415b380 con 0x7f8e540c9f00 2024-04-25T12:37:10.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e5c600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8e54074b20 con 0x7f8e54154650 2024-04-25T12:37:10.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e5c600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8e540c8880 con 0x7f8e540730e0 2024-04-25T12:37:10.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e59c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e54154650 0x7f8e5415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:10.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e59200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8e540c9f00 0x7f8e540c8170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54704/0 (socket says 172.21.15.123:54704) 2024-04-25T12:37:10.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e59200640 1 -- 172.21.15.123:0/4204407784 learned_addr learned my addr 172.21.15.123:0/4204407784 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:10.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.560+0000 7f8e5a600640 1 --2- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e540730e0 0x7f8e540734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:10.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.561+0000 7f8e59c00640 1 -- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e540730e0 msgr2=0x7f8e540734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:10.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.561+0000 7f8e59c00640 1 --2- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e540730e0 0x7f8e540734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.561+0000 7f8e59c00640 1 -- 172.21.15.123:0/4204407784 >> v1:172.21.15.123:6789/0 conn(0x7f8e540c9f00 legacy=0x7f8e540c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:10.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.561+0000 7f8e59c00640 1 -- 172.21.15.123:0/4204407784 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e540ca780 con 0x7f8e54154650 2024-04-25T12:37:10.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.562+0000 7f8e59c00640 1 --2- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e54154650 0x7f8e5415ae40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e480097a0 tx=0x7f8e4802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a668670086e5bf87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:10.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.562+0000 7f8e53e00640 1 -- 172.21.15.123:0/4204407784 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48037070 con 0x7f8e54154650 2024-04-25T12:37:10.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e53e00640 1 -- 172.21.15.123:0/4204407784 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e48032440 con 0x7f8e54154650 2024-04-25T12:37:10.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e53e00640 1 -- 172.21.15.123:0/4204407784 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48032810 con 0x7f8e54154650 2024-04-25T12:37:10.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 -- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e54154650 msgr2=0x7f8e5415ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:10.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 --2- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e54154650 0x7f8e5415ae40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e480097a0 tx=0x7f8e4802a210 comp rx=0 tx=0).stop 2024-04-25T12:37:10.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 -- 172.21.15.123:0/4204407784 shutdown_connections 2024-04-25T12:37:10.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 --2- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e540730e0 0x7f8e540734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 --2- 172.21.15.123:0/4204407784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e54154650 0x7f8e5415ae40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 -- 172.21.15.123:0/4204407784 >> 172.21.15.123:0/4204407784 conn(0x7f8e5406f440 msgr2=0x7f8e5406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:10.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 -- 172.21.15.123:0/4204407784 shutdown_connections 2024-04-25T12:37:10.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.563+0000 7f8e5c600640 1 -- 172.21.15.123:0/4204407784 wait complete. 2024-04-25T12:37:10.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.566+0000 7f8e5c600640 1 Processor -- start 2024-04-25T12:37:10.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.566+0000 7f8e5c600640 1 -- start start 2024-04-25T12:37:10.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.567+0000 7f8e5c600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e540730e0 0x7f8e540c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:10.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.567+0000 7f8e5c600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e540c9f00 0x7f8e5415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:10.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.567+0000 7f8e5c600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e54154650 0x7f8e541450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:10.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.567+0000 7f8e5c600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5415dd30 con 0x7f8e540730e0 2024-04-25T12:37:10.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.567+0000 7f8e5c600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5415da50 con 0x7f8e540c9f00 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.567+0000 7f8e5c600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5415e010 con 0x7f8e54154650 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e540730e0 0x7f8e540c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e540730e0 0x7f8e540c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33046/0 (socket says 172.21.15.123:33046) 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 -- 172.21.15.123:0/2103220599 learned_addr learned my addr 172.21.15.123:0/2103220599 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59200640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e540c9f00 0x7f8e5415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e5a600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e54154650 0x7f8e541450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 -- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e54154650 msgr2=0x7f8e541450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e54154650 0x7f8e541450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 -- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e540c9f00 msgr2=0x7f8e5415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e540c9f00 0x7f8e5415b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 -- 172.21.15.123:0/2103220599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e540c6700 con 0x7f8e540730e0 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e5a600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e54154650 0x7f8e541450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:10.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.568+0000 7f8e59c00640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e540730e0 0x7f8e540c4210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e48035040 tx=0x7f8e480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:10.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.569+0000 7f8e51600640 1 -- 172.21.15.123:0/2103220599 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48037040 con 0x7f8e540730e0 2024-04-25T12:37:10.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.569+0000 7f8e51600640 1 -- 172.21.15.123:0/2103220599 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e48030500 con 0x7f8e540730e0 2024-04-25T12:37:10.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.569+0000 7f8e51600640 1 -- 172.21.15.123:0/2103220599 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48030910 con 0x7f8e540730e0 2024-04-25T12:37:10.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.569+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e54151fc0 con 0x7f8e540730e0 2024-04-25T12:37:10.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.569+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e54153c40 con 0x7f8e540730e0 2024-04-25T12:37:10.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.570+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e54076700 con 0x7f8e540730e0 2024-04-25T12:37:10.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.572+0000 7f8e51600640 1 -- 172.21.15.123:0/2103220599 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e48044020 con 0x7f8e540730e0 2024-04-25T12:37:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.573+0000 7f8e51600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e380744b0 0x7f8e380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.573+0000 7f8e51600640 1 -- 172.21.15.123:0/2103220599 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6891+0+0 (secure 0 0 0) 0x7f8e480b5710 con 0x7f8e540730e0 2024-04-25T12:37:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.573+0000 7f8e59200640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e380744b0 0x7f8e380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.574+0000 7f8e59200640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e380744b0 0x7f8e380769c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8e54076260 tx=0x7f8e44007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:10.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.578+0000 7f8e51600640 1 -- 172.21.15.123:0/2103220599 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e4804b020 con 0x7f8e540730e0 2024-04-25T12:37:10.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:10.796+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) -- 0x7f8e54073c20 con 0x7f8e540730e0 2024-04-25T12:37:11.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.275+0000 7f8e51600640 1 -- 172.21.15.123:0/2103220599 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v83) ==== 133+0+0 (secure 0 0 0) 0x7f8e4807f3a0 con 0x7f8e540730e0 2024-04-25T12:37:11.276 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-25T12:37:11.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e380744b0 msgr2=0x7f8e380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:11.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e380744b0 0x7f8e380769c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8e54076260 tx=0x7f8e44007e80 comp rx=0 tx=0).stop 2024-04-25T12:37:11.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e540730e0 msgr2=0x7f8e540c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:11.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e540730e0 0x7f8e540c4210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e48035040 tx=0x7f8e480046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 shutdown_connections 2024-04-25T12:37:11.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8e380744b0 0x7f8e380769c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e54154650 0x7f8e541450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e540c9f00 0x7f8e5415b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 --2- 172.21.15.123:0/2103220599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e540730e0 0x7f8e540c4210 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.279+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 >> 172.21.15.123:0/2103220599 conn(0x7f8e5406f440 msgr2=0x7f8e541473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:11.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.280+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 shutdown_connections 2024-04-25T12:37:11.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.280+0000 7f8e5c600640 1 -- 172.21.15.123:0/2103220599 wait complete. 2024-04-25T12:37:11.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-25T12:37:11.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.591+0000 7f3495600640 1 Processor -- start 2024-04-25T12:37:11.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.591+0000 7f3495600640 1 -- start start 2024-04-25T12:37:11.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3495600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34900c7dc0 0x7f34901565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:11.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3495600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34900c2720 0x7f349006c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:11.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3495600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34900c81d0 con 0x7f349006ccf0 2024-04-25T12:37:11.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3495600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f34900c8450 con 0x7f34900c7dc0 2024-04-25T12:37:11.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3495600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f3490159310 con 0x7f34900c2720 2024-04-25T12:37:11.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f348fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34900c7dc0 0x7f34901565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:11.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3494c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f349006ccf0 0x7f34900c91c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54720/0 (socket says 172.21.15.123:54720) 2024-04-25T12:37:11.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3487e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34900c2720 0x7f349006c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:11.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f348fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34900c7dc0 0x7f34901565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33058/0 (socket says 172.21.15.123:33058) 2024-04-25T12:37:11.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.593+0000 7f3494c00640 1 -- 172.21.15.123:0/443690253 learned_addr learned my addr 172.21.15.123:0/443690253 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.594+0000 7f348fe00640 1 -- 172.21.15.123:0/443690253 >> v2:172.21.15.153:3300/0 conn(0x7f34900c2720 msgr2=0x7f349006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.594+0000 7f348fe00640 1 --2- 172.21.15.123:0/443690253 >> v2:172.21.15.153:3300/0 conn(0x7f34900c2720 0x7f349006c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.594+0000 7f348fe00640 1 -- 172.21.15.123:0/443690253 >> v1:172.21.15.123:6789/0 conn(0x7f349006ccf0 legacy=0x7f34900c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.594+0000 7f348fe00640 1 -- 172.21.15.123:0/443690253 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34901595f0 con 0x7f34900c7dc0 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.594+0000 7f348f400640 1 -- 172.21.15.123:0/443690253 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409528928 0 0) 0x7f3488001490 con 0x7f349006ccf0 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.595+0000 7f348fe00640 1 --2- 172.21.15.123:0/443690253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34900c7dc0 0x7f34901565f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3490074ba0 tx=0x7f348002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d70538d7d3bfc70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.595+0000 7f348f400640 1 -- 172.21.15.123:0/443690253 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480037070 con 0x7f34900c7dc0 2024-04-25T12:37:11.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.595+0000 7f348f400640 1 -- 172.21.15.123:0/443690253 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3480032440 con 0x7f34900c7dc0 2024-04-25T12:37:11.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.595+0000 7f348f400640 1 -- 172.21.15.123:0/443690253 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480032810 con 0x7f34900c7dc0 2024-04-25T12:37:11.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 -- 172.21.15.123:0/443690253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34900c7dc0 msgr2=0x7f34901565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:11.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 --2- 172.21.15.123:0/443690253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34900c7dc0 0x7f34901565f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3490074ba0 tx=0x7f348002a230 comp rx=0 tx=0).stop 2024-04-25T12:37:11.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 -- 172.21.15.123:0/443690253 shutdown_connections 2024-04-25T12:37:11.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 --2- 172.21.15.123:0/443690253 >> v2:172.21.15.153:3300/0 conn(0x7f34900c2720 0x7f349006c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 --2- 172.21.15.123:0/443690253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34900c7dc0 0x7f34901565f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 -- 172.21.15.123:0/443690253 >> 172.21.15.123:0/443690253 conn(0x7f3490070b70 msgr2=0x7f3490070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:11.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 -- 172.21.15.123:0/443690253 shutdown_connections 2024-04-25T12:37:11.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.596+0000 7f3495600640 1 -- 172.21.15.123:0/443690253 wait complete. 2024-04-25T12:37:11.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.598+0000 7f3495600640 1 Processor -- start 2024-04-25T12:37:11.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.598+0000 7f3495600640 1 -- start start 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3495600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f349006ccf0 0x7f34901f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3495600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900c2720 0x7f34900cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3495600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34900c7dc0 0x7f3490155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3495600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34900c4cb0 con 0x7f349006ccf0 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3495600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3490159bb0 con 0x7f34900c2720 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3495600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3490158fc0 con 0x7f34900c7dc0 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f348fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f349006ccf0 0x7f34901f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f348fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f349006ccf0 0x7f34901f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33062/0 (socket says 172.21.15.123:33062) 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f348fe00640 1 -- 172.21.15.123:0/4282911974 learned_addr learned my addr 172.21.15.123:0/4282911974 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3494c00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34900c7dc0 0x7f3490155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.600+0000 7f3487e00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900c2720 0x7f34900cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348fe00640 1 -- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34900c7dc0 msgr2=0x7f3490155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348fe00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34900c7dc0 0x7f3490155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348fe00640 1 -- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900c2720 msgr2=0x7f34900cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348fe00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900c2720 0x7f34900cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348fe00640 1 -- 172.21.15.123:0/4282911974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34901f3630 con 0x7f349006ccf0 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f3494c00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34900c7dc0 0x7f3490155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348fe00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f349006ccf0 0x7f34901f30f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3480009890 tx=0x7f34800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348cc00640 1 -- 172.21.15.123:0/4282911974 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480037050 con 0x7f349006ccf0 2024-04-25T12:37:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348cc00640 1 -- 172.21.15.123:0/4282911974 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3480038040 con 0x7f349006ccf0 2024-04-25T12:37:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34901f65f0 con 0x7f349006ccf0 2024-04-25T12:37:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.601+0000 7f348cc00640 1 -- 172.21.15.123:0/4282911974 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480004470 con 0x7f349006ccf0 2024-04-25T12:37:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.602+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34901f6bb0 con 0x7f349006ccf0 2024-04-25T12:37:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.603+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34900c9d40 con 0x7f349006ccf0 2024-04-25T12:37:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.604+0000 7f348cc00640 1 -- 172.21.15.123:0/4282911974 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3480003610 con 0x7f349006ccf0 2024-04-25T12:37:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.605+0000 7f348cc00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f346c0744e0 0x7f346c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.605+0000 7f348cc00640 1 -- 172.21.15.123:0/4282911974 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6891+0+0 (secure 0 0 0) 0x7f34800b5ee0 con 0x7f349006ccf0 2024-04-25T12:37:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.605+0000 7f3487e00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f346c0744e0 0x7f346c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.606+0000 7f3487e00640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f346c0744e0 0x7f346c0769f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f34901f09d0 tx=0x7f3478009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.609+0000 7f348cc00640 1 -- 172.21.15.123:0/4282911974 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34900c9d40 con 0x7f349006ccf0 2024-04-25T12:37:11.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:11.819+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7f3490157e70 con 0x7f349006ccf0 2024-04-25T12:37:12.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.288+0000 7f348cc00640 1 -- 172.21.15.123:0/4282911974 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v84) ==== 113+0+0 (secure 0 0 0) 0x7f3490157e70 con 0x7f349006ccf0 2024-04-25T12:37:12.289 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.3 to 1 (10000) 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f346c0744e0 msgr2=0x7f346c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f346c0744e0 0x7f346c0769f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f34901f09d0 tx=0x7f3478009190 comp rx=0 tx=0).stop 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f349006ccf0 msgr2=0x7f34901f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f349006ccf0 0x7f34901f30f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3480009890 tx=0x7f34800046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 shutdown_connections 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f346c0744e0 0x7f346c0769f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34900c7dc0 0x7f3490155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900c2720 0x7f34900cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 --2- 172.21.15.123:0/4282911974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f349006ccf0 0x7f34901f30f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 >> 172.21.15.123:0/4282911974 conn(0x7f3490070b70 msgr2=0x7f34900c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:12.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.292+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 shutdown_connections 2024-04-25T12:37:12.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.293+0000 7f3495600640 1 -- 172.21.15.123:0/4282911974 wait complete. 2024-04-25T12:37:12.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-25T12:37:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.591+0000 7fd221200640 1 Processor -- start 2024-04-25T12:37:12.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.591+0000 7fd221200640 1 -- start start 2024-04-25T12:37:12.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.593+0000 7fd221200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd21c152c70 0x7fd21c074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.593+0000 7fd221200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c8170 0x7fd21c15c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.593+0000 7fd221200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c0c8580 con 0x7fd21c0c2790 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd221200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd21c158bd0 con 0x7fd21c152c70 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd221200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c158eb0 con 0x7fd21c0c8170 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd219400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd21c0c2790 0x7fd21c0c3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54724/0 (socket says 172.21.15.123:54724) 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd219400640 1 -- 172.21.15.123:0/186223687 learned_addr learned my addr 172.21.15.123:0/186223687 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd219e00640 1 --2- 172.21.15.123:0/186223687 >> v2:172.21.15.153:3300/0 conn(0x7fd21c152c70 0x7fd21c074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd21a800640 1 --2- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c8170 0x7fd21c15c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd218a00640 1 -- 172.21.15.123:0/186223687 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045195699 0 0) 0x7fd21c0c8580 con 0x7fd21c0c2790 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd218a00640 1 -- 172.21.15.123:0/186223687 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1fc003520 con 0x7fd21c0c2790 2024-04-25T12:37:12.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd219e00640 1 -- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c8170 msgr2=0x7fd21c15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:12.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd219e00640 1 --2- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c8170 0x7fd21c15c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.594+0000 7fd219e00640 1 -- 172.21.15.123:0/186223687 >> v1:172.21.15.123:6789/0 conn(0x7fd21c0c2790 legacy=0x7fd21c0c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:12.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.595+0000 7fd219e00640 1 -- 172.21.15.123:0/186223687 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c159190 con 0x7fd21c152c70 2024-04-25T12:37:12.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.596+0000 7fd219e00640 1 --2- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c152c70 0x7fd21c074820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00a940 tx=0x7fd20c02b4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb1eb6d74e9e7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:12.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.596+0000 7fd218a00640 1 -- 172.21.15.123:0/186223687 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00b040 con 0x7fd21c152c70 2024-04-25T12:37:12.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.596+0000 7fd218a00640 1 -- 172.21.15.123:0/186223687 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c030710 con 0x7fd21c152c70 2024-04-25T12:37:12.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.596+0000 7fd218a00640 1 -- 172.21.15.123:0/186223687 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c030ae0 con 0x7fd21c152c70 2024-04-25T12:37:12.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 -- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c152c70 msgr2=0x7fd21c074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 --2- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c152c70 0x7fd21c074820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00a940 tx=0x7fd20c02b4c0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 -- 172.21.15.123:0/186223687 shutdown_connections 2024-04-25T12:37:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 --2- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c8170 0x7fd21c15c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 --2- 172.21.15.123:0/186223687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c152c70 0x7fd21c074820 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 -- 172.21.15.123:0/186223687 >> 172.21.15.123:0/186223687 conn(0x7fd21c06f440 msgr2=0x7fd21c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 -- 172.21.15.123:0/186223687 shutdown_connections 2024-04-25T12:37:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.597+0000 7fd221200640 1 -- 172.21.15.123:0/186223687 wait complete. 2024-04-25T12:37:12.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.599+0000 7fd221200640 1 Processor -- start 2024-04-25T12:37:12.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.599+0000 7fd221200640 1 -- start start 2024-04-25T12:37:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd221200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c2790 0x7fd21c1f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd221200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c0c8170 0x7fd21c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd221200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c152c70 0x7fd21c147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:12.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd221200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0c46f0 con 0x7fd21c152c70 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd221200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c159750 con 0x7fd21c0c8170 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd221200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c0c4870 con 0x7fd21c0c2790 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd21a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c152c70 0x7fd21c147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c2790 0x7fd21c1f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c0c8170 0x7fd21c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c2790 0x7fd21c1f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:45398/0 (socket says 172.21.15.123:45398) 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219e00640 1 -- 172.21.15.123:0/3183322587 learned_addr learned my addr 172.21.15.123:0/3183322587 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:12.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219400640 1 -- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c2790 msgr2=0x7fd21c1f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219400640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c2790 0x7fd21c1f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219400640 1 -- 172.21.15.123:0/3183322587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c152c70 msgr2=0x7fd21c147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.601+0000 7fd219400640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c152c70 0x7fd21c147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd219400640 1 -- 172.21.15.123:0/3183322587 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c1f3260 con 0x7fd21c0c8170 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd219e00640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c2790 0x7fd21c1f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd219400640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c0c8170 0x7fd21c158070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c9f0 tx=0x7fd20400ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd212000640 1 -- 172.21.15.123:0/3183322587 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204017070 con 0x7fd21c0c8170 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd212000640 1 -- 172.21.15.123:0/3183322587 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20400ab70 con 0x7fd21c0c8170 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd212000640 1 -- 172.21.15.123:0/3183322587 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204012400 con 0x7fd21c0c8170 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c14acc0 con 0x7fd21c0c8170 2024-04-25T12:37:12.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.602+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c15b540 con 0x7fd21c0c8170 2024-04-25T12:37:12.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.603+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c076700 con 0x7fd21c0c8170 2024-04-25T12:37:12.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.605+0000 7fd212000640 1 -- 172.21.15.123:0/3183322587 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd20401a070 con 0x7fd21c0c8170 2024-04-25T12:37:12.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.606+0000 7fd212000640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd1f80744b0 0x7fd1f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:12.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.606+0000 7fd212000640 1 -- 172.21.15.123:0/3183322587 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6891+0+0 (secure 0 0 0) 0x7fd204095400 con 0x7fd21c0c8170 2024-04-25T12:37:12.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.606+0000 7fd219e00640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd1f80744b0 0x7fd1f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:12.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.607+0000 7fd219e00640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd1f80744b0 0x7fd1f80769c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02b500 tx=0x7fd20c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:12.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.611+0000 7fd212000640 1 -- 172.21.15.123:0/3183322587 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd20405f000 con 0x7fd21c0c8170 2024-04-25T12:37:12.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:12.829+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) -- 0x7fd21c074600 con 0x7fd21c0c8170 2024-04-25T12:37:13.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.314+0000 7fd212000640 1 -- 172.21.15.123:0/3183322587 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v85) ==== 133+0+0 (secure 0 0 0) 0x7fd204063300 con 0x7fd21c0c8170 2024-04-25T12:37:13.315 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd1f80744b0 msgr2=0x7fd1f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd1f80744b0 0x7fd1f80769c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02b500 tx=0x7fd20c030000 comp rx=0 tx=0).stop 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c0c8170 msgr2=0x7fd21c158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c0c8170 0x7fd21c158070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c9f0 tx=0x7fd20400ceb0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 shutdown_connections 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd1f80744b0 0x7fd1f80769c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c152c70 0x7fd21c147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd21c0c8170 0x7fd21c158070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 --2- 172.21.15.123:0/3183322587 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd21c0c2790 0x7fd21c1f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.318+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 >> 172.21.15.123:0/3183322587 conn(0x7fd21c06f440 msgr2=0x7fd21c1e7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:13.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.319+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 shutdown_connections 2024-04-25T12:37:13.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.319+0000 7fd221200640 1 -- 172.21.15.123:0/3183322587 wait complete. 2024-04-25T12:37:13.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-25T12:37:13.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.629+0000 7f0982200640 1 Processor -- start 2024-04-25T12:37:13.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.630+0000 7f0982200640 1 -- start start 2024-04-25T12:37:13.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f0982200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c153640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:13.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f0982200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097c074400 con 0x7f097c0c8170 2024-04-25T12:37:13.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f0982200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097c153b80 con 0x7f097c1426a0 2024-04-25T12:37:13.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f0982200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f097c1595e0 con 0x7f097c0c8fd0 2024-04-25T12:37:13.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f097b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:13.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f097be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f097c0c8170 0x7f097c0ca200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42792/0 (socket says 172.21.15.123:42792) 2024-04-25T12:37:13.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f097be00640 1 -- 172.21.15.123:0/133487754 learned_addr learned my addr 172.21.15.123:0/133487754 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:13.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.632+0000 7f0972a00640 1 --1- 172.21.15.123:0/133487754 >> v1:172.21.15.153:6789/0 conn(0x7f097c1426a0 0x7f097c159dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:45888/0 (socket says 172.21.15.123:45888) 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.633+0000 7f097aa00640 1 -- 172.21.15.123:0/133487754 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3983730554 0 0) 0x7f0964001110 con 0x7f097c1426a0 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.633+0000 7f097b400640 1 -- 172.21.15.123:0/133487754 >> v1:172.21.15.153:6789/0 conn(0x7f097c1426a0 legacy=0x7f097c159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.633+0000 7f097b400640 1 -- 172.21.15.123:0/133487754 >> v1:172.21.15.123:6789/0 conn(0x7f097c0c8170 legacy=0x7f097c0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.633+0000 7f097b400640 1 -- 172.21.15.123:0/133487754 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c074670 con 0x7f097c0c8fd0 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.634+0000 7f097b400640 1 --2- 172.21.15.123:0/133487754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c153640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f096c009dc0 tx=0x7f096c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92aa1ac5f0e17138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.634+0000 7f097aa00640 1 -- 172.21.15.123:0/133487754 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c007740 con 0x7f097c0c8fd0 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.634+0000 7f097aa00640 1 -- 172.21.15.123:0/133487754 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c0078e0 con 0x7f097c0c8fd0 2024-04-25T12:37:13.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.634+0000 7f097aa00640 1 -- 172.21.15.123:0/133487754 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c007cb0 con 0x7f097c0c8fd0 2024-04-25T12:37:13.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.635+0000 7f0982200640 1 -- 172.21.15.123:0/133487754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 msgr2=0x7f097c153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.635+0000 7f0982200640 1 --2- 172.21.15.123:0/133487754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c153640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f096c009dc0 tx=0x7f096c02a5d0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.635+0000 7f0982200640 1 -- 172.21.15.123:0/133487754 shutdown_connections 2024-04-25T12:37:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.635+0000 7f0982200640 1 --2- 172.21.15.123:0/133487754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c153640 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.635+0000 7f0982200640 1 -- 172.21.15.123:0/133487754 >> 172.21.15.123:0/133487754 conn(0x7f097c06f440 msgr2=0x7f097c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.635+0000 7f0982200640 1 -- 172.21.15.123:0/133487754 shutdown_connections 2024-04-25T12:37:13.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.635+0000 7f0982200640 1 -- 172.21.15.123:0/133487754 wait complete. 2024-04-25T12:37:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.637+0000 7f0982200640 1 Processor -- start 2024-04-25T12:37:13.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.638+0000 7f0982200640 1 -- start start 2024-04-25T12:37:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f0982200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097c0c8170 0x7f097c0c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f0982200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c138a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:13.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f0982200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097c1426a0 0x7f097c1f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f0982200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f097c0c4880 con 0x7f097c0c8170 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f0982200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1598a0 con 0x7f097c1426a0 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f0982200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f097c0ca910 con 0x7f097c0c8fd0 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f097b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097c0c8170 0x7f097c0c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f097b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097c0c8170 0x7f097c0c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40128/0 (socket says 172.21.15.123:40128) 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f097b400640 1 -- 172.21.15.123:0/1748498620 learned_addr learned my addr 172.21.15.123:0/1748498620 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f0972a00640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c138a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:13.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.639+0000 7f097be00640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097c1426a0 0x7f097c1f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.640+0000 7f097b400640 1 -- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 msgr2=0x7f097c138a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.640+0000 7f097b400640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c138a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.640+0000 7f097b400640 1 -- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097c1426a0 msgr2=0x7f097c1f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.640+0000 7f097b400640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097c1426a0 0x7f097c1f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.640+0000 7f097b400640 1 -- 172.21.15.123:0/1748498620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c0c72d0 con 0x7f097c0c8170 2024-04-25T12:37:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.640+0000 7f097b400640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097c0c8170 0x7f097c0c6d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f096c02abc0 tx=0x7f096c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:13.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.641+0000 7f0973e00640 1 -- 172.21.15.123:0/1748498620 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c040070 con 0x7f097c0c8170 2024-04-25T12:37:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.641+0000 7f0973e00640 1 -- 172.21.15.123:0/1748498620 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c031c40 con 0x7f097c0c8170 2024-04-25T12:37:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.641+0000 7f0973e00640 1 -- 172.21.15.123:0/1748498620 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c03c420 con 0x7f097c0c8170 2024-04-25T12:37:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.641+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c1544d0 con 0x7f097c0c8170 2024-04-25T12:37:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.641+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c154eb0 con 0x7f097c0c8170 2024-04-25T12:37:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.642+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c076700 con 0x7f097c0c8170 2024-04-25T12:37:13.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.643+0000 7f0973e00640 1 -- 172.21.15.123:0/1748498620 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f096c039040 con 0x7f097c0c8170 2024-04-25T12:37:13.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.644+0000 7f0973e00640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09580744b0 0x7f09580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:13.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.644+0000 7f0973e00640 1 -- 172.21.15.123:0/1748498620 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6891+0+0 (secure 0 0 0) 0x7f096c0b6b60 con 0x7f097c0c8170 2024-04-25T12:37:13.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.644+0000 7f0972a00640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09580744b0 0x7f09580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:13.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.645+0000 7f0972a00640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09580744b0 0x7f09580769c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f097c0c75b0 tx=0x7f0964007e40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:13.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.649+0000 7f0973e00640 1 -- 172.21.15.123:0/1748498620 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f096c080880 con 0x7f097c0c8170 2024-04-25T12:37:13.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:13.863+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f097c157c60 con 0x7f097c0c8170 2024-04-25T12:37:14.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.311+0000 7f0973e00640 1 -- 172.21.15.123:0/1748498620 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v86) ==== 113+0+0 (secure 0 0 0) 0x7f096c084b80 con 0x7f097c0c8170 2024-04-25T12:37:14.311 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.4 to 1 (10000) 2024-04-25T12:37:14.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.316+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09580744b0 msgr2=0x7f09580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:14.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.316+0000 7f0982200640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09580744b0 0x7f09580769c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f097c0c75b0 tx=0x7f0964007e40 comp rx=0 tx=0).stop 2024-04-25T12:37:14.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.316+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097c0c8170 msgr2=0x7f097c0c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:14.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.316+0000 7f0982200640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097c0c8170 0x7f097c0c6d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f096c02abc0 tx=0x7f096c004a40 comp rx=0 tx=0).stop 2024-04-25T12:37:14.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.317+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 shutdown_connections 2024-04-25T12:37:14.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.317+0000 7f0982200640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f09580744b0 0x7f09580769c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.317+0000 7f0982200640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097c1426a0 0x7f097c1f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.317+0000 7f0982200640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f097c0c8fd0 0x7f097c138a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.317+0000 7f0982200640 1 --2- 172.21.15.123:0/1748498620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097c0c8170 0x7f097c0c6d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.317+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 >> 172.21.15.123:0/1748498620 conn(0x7f097c06f440 msgr2=0x7f097c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:14.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.317+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 shutdown_connections 2024-04-25T12:37:14.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.318+0000 7f0982200640 1 -- 172.21.15.123:0/1748498620 wait complete. 2024-04-25T12:37:14.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-25T12:37:14.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.619+0000 7f12db200640 1 Processor -- start 2024-04-25T12:37:14.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.619+0000 7f12db200640 1 -- start start 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.621+0000 7f12db200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c7dc0 0x7f12dc1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.621+0000 7f12db200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12dc0c2720 0x7f12dc06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.621+0000 7f12db200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f12dc06ccf0 0x7f12dc0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.621+0000 7f12db200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f12dc0c81d0 con 0x7f12dc0c2720 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.621+0000 7f12db200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f12dc0c8450 con 0x7f12dc06ccf0 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.621+0000 7f12db200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f12dc159310 con 0x7f12dc0c7dc0 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12da800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f12dc06ccf0 0x7f12dc0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c7dc0 0x7f12dc1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:14.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12da800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f12dc06ccf0 0x7f12dc0c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:45370/0 (socket says 172.21.15.123:45370) 2024-04-25T12:37:14.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c7dc0 0x7f12dc1565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:47260/0 (socket says 172.21.15.123:47260) 2024-04-25T12:37:14.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12da800640 1 -- 172.21.15.123:0/2052650156 learned_addr learned my addr 172.21.15.123:0/2052650156 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:14.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9400640 1 --2- 172.21.15.123:0/2052650156 >> v2:172.21.15.123:3300/0 conn(0x7f12dc0c2720 0x7f12dc06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9e00640 1 -- 172.21.15.123:0/2052650156 >> v2:172.21.15.153:3300/0 conn(0x7f12dc06ccf0 msgr2=0x7f12dc0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9e00640 1 --2- 172.21.15.123:0/2052650156 >> v2:172.21.15.153:3300/0 conn(0x7f12dc06ccf0 0x7f12dc0c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9e00640 1 -- 172.21.15.123:0/2052650156 >> v2:172.21.15.123:3300/0 conn(0x7f12dc0c2720 msgr2=0x7f12dc06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9e00640 1 --2- 172.21.15.123:0/2052650156 >> v2:172.21.15.123:3300/0 conn(0x7f12dc0c2720 0x7f12dc06c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12d9e00640 1 -- 172.21.15.123:0/2052650156 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc1595f0 con 0x7f12dc0c7dc0 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.622+0000 7f12da800640 1 --2- 172.21.15.123:0/2052650156 >> v2:172.21.15.153:3300/0 conn(0x7f12dc06ccf0 0x7f12dc0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.623+0000 7f12d9e00640 1 --2- 172.21.15.123:0/2052650156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c7dc0 0x7f12dc1565f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0c8760 tx=0x7f12cc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfd502e0b42c26c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.624+0000 7f12d8a00640 1 -- 172.21.15.123:0/2052650156 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007740 con 0x7f12dc0c7dc0 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.624+0000 7f12d8a00640 1 -- 172.21.15.123:0/2052650156 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12cc0078e0 con 0x7f12dc0c7dc0 2024-04-25T12:37:14.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.624+0000 7f12d8a00640 1 -- 172.21.15.123:0/2052650156 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007cb0 con 0x7f12dc0c7dc0 2024-04-25T12:37:14.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 -- 172.21.15.123:0/2052650156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c7dc0 msgr2=0x7f12dc1565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:14.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 --2- 172.21.15.123:0/2052650156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c7dc0 0x7f12dc1565f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0c8760 tx=0x7f12cc02a660 comp rx=0 tx=0).stop 2024-04-25T12:37:14.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 -- 172.21.15.123:0/2052650156 shutdown_connections 2024-04-25T12:37:14.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 --2- 172.21.15.123:0/2052650156 >> v2:172.21.15.153:3300/0 conn(0x7f12dc06ccf0 0x7f12dc0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 --2- 172.21.15.123:0/2052650156 >> v2:172.21.15.123:3300/0 conn(0x7f12dc0c2720 0x7f12dc06c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 --2- 172.21.15.123:0/2052650156 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c7dc0 0x7f12dc1565f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 -- 172.21.15.123:0/2052650156 >> 172.21.15.123:0/2052650156 conn(0x7f12dc070b70 msgr2=0x7f12dc070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:14.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 -- 172.21.15.123:0/2052650156 shutdown_connections 2024-04-25T12:37:14.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.625+0000 7f12db200640 1 -- 172.21.15.123:0/2052650156 wait complete. 2024-04-25T12:37:14.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.627+0000 7f12db200640 1 Processor -- start 2024-04-25T12:37:14.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.627+0000 7f12db200640 1 -- start start 2024-04-25T12:37:14.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12db200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12dc06ccf0 0x7f12dc1f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12db200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c2720 0x7f12dc147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12db200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12dc0c7dc0 0x7f12dc148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12db200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc0c4cb0 con 0x7f12dc0c7dc0 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12db200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc159bb0 con 0x7f12dc06ccf0 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12db200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f12dc158fc0 con 0x7f12dc0c2720 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12d9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12dc06ccf0 0x7f12dc1f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12d9400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c2720 0x7f12dc147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12d9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12dc06ccf0 0x7f12dc1f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:45372/0 (socket says 172.21.15.123:45372) 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12d9e00640 1 -- 172.21.15.123:0/3842223559 learned_addr learned my addr 172.21.15.123:0/3842223559 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12da800640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12dc0c7dc0 0x7f12dc148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.629+0000 7f12d9e00640 1 -- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c2720 msgr2=0x7f12dc147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:14.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d9e00640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c2720 0x7f12dc147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d9e00640 1 -- 172.21.15.123:0/3842223559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12dc0c7dc0 msgr2=0x7f12dc148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:14.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d9e00640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12dc0c7dc0 0x7f12dc148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:14.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d9e00640 1 -- 172.21.15.123:0/3842223559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc1579e0 con 0x7f12dc06ccf0 2024-04-25T12:37:14.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d9400640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c2720 0x7f12dc147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:14.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12da800640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12dc0c7dc0 0x7f12dc148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:14.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d9e00640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12dc06ccf0 0x7f12dc1f1d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12cc02ac50 tx=0x7f12cc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:14.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d2000640 1 -- 172.21.15.123:0/3842223559 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc03f070 con 0x7f12dc06ccf0 2024-04-25T12:37:14.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d2000640 1 -- 172.21.15.123:0/3842223559 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12cc031c40 con 0x7f12dc06ccf0 2024-04-25T12:37:14.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc153d30 con 0x7f12dc06ccf0 2024-04-25T12:37:14.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12d2000640 1 -- 172.21.15.123:0/3842223559 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc009450 con 0x7f12dc06ccf0 2024-04-25T12:37:14.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.630+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc1542f0 con 0x7f12dc06ccf0 2024-04-25T12:37:14.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.631+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc153a20 con 0x7f12dc06ccf0 2024-04-25T12:37:14.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.633+0000 7f12d2000640 1 -- 172.21.15.123:0/3842223559 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f12cc043410 con 0x7f12dc06ccf0 2024-04-25T12:37:14.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.634+0000 7f12d2000640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12b80744b0 0x7f12b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:14.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.634+0000 7f12d2000640 1 -- 172.21.15.123:0/3842223559 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6891+0+0 (secure 0 0 0) 0x7f12cc0b69c0 con 0x7f12dc06ccf0 2024-04-25T12:37:14.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.634+0000 7f12d9400640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12b80744b0 0x7f12b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:14.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.635+0000 7f12d9400640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12b80744b0 0x7f12b80769c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12dc1489d0 tx=0x7f12c4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:14.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.639+0000 7f12d2000640 1 -- 172.21.15.123:0/3842223559 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12cc080650 con 0x7f12dc06ccf0 2024-04-25T12:37:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:14.852+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) -- 0x7f12dc076da0 con 0x7f12dc06ccf0 2024-04-25T12:37:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.338+0000 7f12d2000640 1 -- 172.21.15.123:0/3842223559 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v87) ==== 133+0+0 (secure 0 0 0) 0x7f12cc084950 con 0x7f12dc06ccf0 2024-04-25T12:37:15.339 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-25T12:37:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12b80744b0 msgr2=0x7f12b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:15.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12b80744b0 0x7f12b80769c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12dc1489d0 tx=0x7f12c4009190 comp rx=0 tx=0).stop 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12dc06ccf0 msgr2=0x7f12dc1f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12dc06ccf0 0x7f12dc1f1d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12cc02ac50 tx=0x7f12cc004a40 comp rx=0 tx=0).stop 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 shutdown_connections 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f12b80744b0 0x7f12b80769c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12dc0c7dc0 0x7f12dc148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12dc0c2720 0x7f12dc147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 --2- 172.21.15.123:0/3842223559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12dc06ccf0 0x7f12dc1f1d90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 >> 172.21.15.123:0/3842223559 conn(0x7f12dc070b70 msgr2=0x7f12dc158700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 shutdown_connections 2024-04-25T12:37:15.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.342+0000 7f12db200640 1 -- 172.21.15.123:0/3842223559 wait complete. 2024-04-25T12:37:15.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-25T12:37:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.643+0000 7f4702400640 1 Processor -- start 2024-04-25T12:37:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.643+0000 7f4702400640 1 -- start start 2024-04-25T12:37:15.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.645+0000 7f4702400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f46fc072cc0 0x7f46fc06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:15.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.645+0000 7f4702400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46fc0730d0 0x7f46fc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.645+0000 7f4702400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46fc074290 con 0x7f46fc0c7e30 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.645+0000 7f4702400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f46fc074570 con 0x7f46fc072cc0 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.645+0000 7f4702400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f46fc074850 con 0x7f46fc0730d0 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f46fc072cc0 0x7f46fc06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f46fc072cc0 0x7f46fc06c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40144/0 (socket says 172.21.15.123:40144) 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fbe00640 1 -- 172.21.15.123:0/3047782909 learned_addr learned my addr 172.21.15.123:0/3047782909 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f4701a00640 1 --2- 172.21.15.123:0/3047782909 >> v2:172.21.15.153:3300/0 conn(0x7f46fc0730d0 0x7f46fc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:15.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fbe00640 1 -- 172.21.15.123:0/3047782909 >> v2:172.21.15.153:3300/0 conn(0x7f46fc0730d0 msgr2=0x7f46fc06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fbe00640 1 --2- 172.21.15.123:0/3047782909 >> v2:172.21.15.153:3300/0 conn(0x7f46fc0730d0 0x7f46fc06ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fb400640 1 -- 172.21.15.123:0/3047782909 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1569476980 0 0) 0x7f46fc074290 con 0x7f46fc0c7e30 2024-04-25T12:37:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fbe00640 1 -- 172.21.15.123:0/3047782909 >> v1:172.21.15.153:6789/0 conn(0x7f46fc0c7e30 legacy=0x7f46fc154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:15.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.646+0000 7f46fbe00640 1 -- 172.21.15.123:0/3047782909 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc074b30 con 0x7f46fc072cc0 2024-04-25T12:37:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.647+0000 7f46fbe00640 1 --2- 172.21.15.123:0/3047782909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc072cc0 0x7f46fc06c7b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46fc072850 tx=0x7f46e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c939bb22b7045f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.647+0000 7f46fb400640 1 -- 172.21.15.123:0/3047782909 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8037070 con 0x7f46fc072cc0 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.647+0000 7f46fb400640 1 -- 172.21.15.123:0/3047782909 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8032440 con 0x7f46fc072cc0 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f46fb400640 1 -- 172.21.15.123:0/3047782909 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8032810 con 0x7f46fc072cc0 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 -- 172.21.15.123:0/3047782909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc072cc0 msgr2=0x7f46fc06c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 --2- 172.21.15.123:0/3047782909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc072cc0 0x7f46fc06c7b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46fc072850 tx=0x7f46e802a270 comp rx=0 tx=0).stop 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 -- 172.21.15.123:0/3047782909 shutdown_connections 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 --2- 172.21.15.123:0/3047782909 >> v2:172.21.15.153:3300/0 conn(0x7f46fc0730d0 0x7f46fc06ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 --2- 172.21.15.123:0/3047782909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc072cc0 0x7f46fc06c7b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 -- 172.21.15.123:0/3047782909 >> 172.21.15.123:0/3047782909 conn(0x7f46fc070b70 msgr2=0x7f46fc070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 -- 172.21.15.123:0/3047782909 shutdown_connections 2024-04-25T12:37:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.648+0000 7f4702400640 1 -- 172.21.15.123:0/3047782909 wait complete. 2024-04-25T12:37:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.651+0000 7f4702400640 1 Processor -- start 2024-04-25T12:37:15.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.651+0000 7f4702400640 1 -- start start 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.652+0000 7f4702400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46fc072cc0 0x7f46fc1f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.652+0000 7f4702400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc0730d0 0x7f46fc1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.652+0000 7f4702400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc0c7e30 0x7f46fc14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.652+0000 7f4702400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0c83c0 con 0x7f46fc0c7e30 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.652+0000 7f4702400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0c8240 con 0x7f46fc0730d0 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.652+0000 7f4702400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc15dc70 con 0x7f46fc072cc0 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f4701a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc0c7e30 0x7f46fc14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc0730d0 0x7f46fc1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f4701a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc0c7e30 0x7f46fc14f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40148/0 (socket says 172.21.15.123:40148) 2024-04-25T12:37:15.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f4701a00640 1 -- 172.21.15.123:0/766621866 learned_addr learned my addr 172.21.15.123:0/766621866 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:15.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f4701000640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46fc072cc0 0x7f46fc1f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f46fbe00640 1 -- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46fc072cc0 msgr2=0x7f46fc1f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f46fbe00640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46fc072cc0 0x7f46fc1f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f46fbe00640 1 -- 172.21.15.123:0/766621866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc0c7e30 msgr2=0x7f46fc14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f46fbe00640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc0c7e30 0x7f46fc14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.653+0000 7f46fbe00640 1 -- 172.21.15.123:0/766621866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc151900 con 0x7f46fc0730d0 2024-04-25T12:37:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.654+0000 7f46fbe00640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc0730d0 0x7f46fc1f5d50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f46e8031000 tx=0x7f46e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:15.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.654+0000 7f46f8c00640 1 -- 172.21.15.123:0/766621866 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8037040 con 0x7f46fc0730d0 2024-04-25T12:37:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.654+0000 7f46f8c00640 1 -- 172.21.15.123:0/766621866 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8004030 con 0x7f46fc0730d0 2024-04-25T12:37:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.654+0000 7f46f8c00640 1 -- 172.21.15.123:0/766621866 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8004400 con 0x7f46fc0730d0 2024-04-25T12:37:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.654+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc1f6290 con 0x7f46fc0730d0 2024-04-25T12:37:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.654+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc1f27e0 con 0x7f46fc0730d0 2024-04-25T12:37:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.655+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc076da0 con 0x7f46fc0730d0 2024-04-25T12:37:15.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.657+0000 7f46f8c00640 1 -- 172.21.15.123:0/766621866 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46e803a070 con 0x7f46fc0730d0 2024-04-25T12:37:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.657+0000 7f46f8c00640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f46dc0744e0 0x7f46dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.658+0000 7f46f8c00640 1 -- 172.21.15.123:0/766621866 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6891+0+0 (secure 0 0 0) 0x7f46e80b58d0 con 0x7f46fc0730d0 2024-04-25T12:37:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.658+0000 7f4701000640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f46dc0744e0 0x7f46dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:15.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.658+0000 7f4701000640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f46dc0744e0 0x7f46dc0769f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f46f0000f60 tx=0x7f46f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:15.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.662+0000 7f46f8c00640 1 -- 172.21.15.123:0/766621866 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46e8041020 con 0x7f46fc0730d0 2024-04-25T12:37:15.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:15.874+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7f46fc073690 con 0x7f46fc0730d0 2024-04-25T12:37:16.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.339+0000 7f46f8c00640 1 -- 172.21.15.123:0/766621866 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v88) ==== 113+0+0 (secure 0 0 0) 0x7f46e807f560 con 0x7f46fc0730d0 2024-04-25T12:37:16.342 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.6 to 1 (10000) 2024-04-25T12:37:16.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f46dc0744e0 msgr2=0x7f46dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:16.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f46dc0744e0 0x7f46dc0769f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f46f0000f60 tx=0x7f46f0009040 comp rx=0 tx=0).stop 2024-04-25T12:37:16.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc0730d0 msgr2=0x7f46fc1f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc0730d0 0x7f46fc1f5d50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f46e8031000 tx=0x7f46e80046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 shutdown_connections 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f46dc0744e0 0x7f46dc0769f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46fc0c7e30 0x7f46fc14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc0730d0 0x7f46fc1f5d50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 --2- 172.21.15.123:0/766621866 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46fc072cc0 0x7f46fc1f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 >> 172.21.15.123:0/766621866 conn(0x7f46fc070b70 msgr2=0x7f46fc0cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 shutdown_connections 2024-04-25T12:37:16.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.344+0000 7f4702400640 1 -- 172.21.15.123:0/766621866 wait complete. 2024-04-25T12:37:16.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-25T12:37:16.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.648+0000 7f85ba600640 1 Processor -- start 2024-04-25T12:37:16.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.648+0000 7f85ba600640 1 -- start start 2024-04-25T12:37:16.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.649+0000 7f85ba600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b4161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:16.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.649+0000 7f85ba600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b415b0b0 con 0x7f85b40c3740 2024-04-25T12:37:16.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.649+0000 7f85ba600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b4073480 con 0x7f85b40c3b50 2024-04-25T12:37:16.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.649+0000 7f85ba600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f85b4074170 con 0x7f85b415aca0 2024-04-25T12:37:16.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.650+0000 7f85b2a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f85b40c3740 0x7f85b40ca060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42794/0 (socket says 172.21.15.123:42794) 2024-04-25T12:37:16.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.650+0000 7f85b2a00640 1 -- 172.21.15.123:0/3597515426 learned_addr learned my addr 172.21.15.123:0/3597515426 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:16.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.650+0000 7f85b3400640 1 --2- 172.21.15.123:0/3597515426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b4161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:16.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.650+0000 7f85b2000640 1 -- 172.21.15.123:0/3597515426 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321129409 0 0) 0x7f85b4073480 con 0x7f85b40c3b50 2024-04-25T12:37:16.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.650+0000 7f85b3400640 1 -- 172.21.15.123:0/3597515426 >> v1:172.21.15.153:6789/0 conn(0x7f85b40c3b50 legacy=0x7f85b416b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:16.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.650+0000 7f85b3400640 1 -- 172.21.15.123:0/3597515426 >> v1:172.21.15.123:6789/0 conn(0x7f85b40c3740 legacy=0x7f85b40ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:16.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.650+0000 7f85b3400640 1 -- 172.21.15.123:0/3597515426 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4073760 con 0x7f85b415aca0 2024-04-25T12:37:16.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.651+0000 7f85b3400640 1 --2- 172.21.15.123:0/3597515426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b4161470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009dc0 tx=0x7f85a402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f9e8a62e4a6645b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:16.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.652+0000 7f85b2000640 1 -- 172.21.15.123:0/3597515426 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4007740 con 0x7f85b415aca0 2024-04-25T12:37:16.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.652+0000 7f85b2000640 1 -- 172.21.15.123:0/3597515426 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a40078e0 con 0x7f85b415aca0 2024-04-25T12:37:16.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.652+0000 7f85b2000640 1 -- 172.21.15.123:0/3597515426 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4007cb0 con 0x7f85b415aca0 2024-04-25T12:37:16.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.652+0000 7f85ba600640 1 -- 172.21.15.123:0/3597515426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 msgr2=0x7f85b4161470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:16.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.652+0000 7f85ba600640 1 --2- 172.21.15.123:0/3597515426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b4161470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009dc0 tx=0x7f85a402a660 comp rx=0 tx=0).stop 2024-04-25T12:37:16.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.653+0000 7f85ba600640 1 -- 172.21.15.123:0/3597515426 shutdown_connections 2024-04-25T12:37:16.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.653+0000 7f85ba600640 1 --2- 172.21.15.123:0/3597515426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b4161470 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.653+0000 7f85ba600640 1 -- 172.21.15.123:0/3597515426 >> 172.21.15.123:0/3597515426 conn(0x7f85b406f440 msgr2=0x7f85b406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:16.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.653+0000 7f85ba600640 1 -- 172.21.15.123:0/3597515426 shutdown_connections 2024-04-25T12:37:16.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.653+0000 7f85ba600640 1 -- 172.21.15.123:0/3597515426 wait complete. 2024-04-25T12:37:16.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.655+0000 7f85ba600640 1 Processor -- start 2024-04-25T12:37:16.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.655+0000 7f85ba600640 1 -- start start 2024-04-25T12:37:16.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ba600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85b40c3740 0x7f85b41f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:16.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ba600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b40c3b50 0x7f85b4155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:16.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ba600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b414b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:16.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ba600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4071220 con 0x7f85b40c3b50 2024-04-25T12:37:16.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ba600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4073230 con 0x7f85b40c3740 2024-04-25T12:37:16.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ba600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f85b4070bd0 con 0x7f85b415aca0 2024-04-25T12:37:16.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b40c3b50 0x7f85b4155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:16.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b40c3b50 0x7f85b4155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40158/0 (socket says 172.21.15.123:40158) 2024-04-25T12:37:16.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 -- 172.21.15.123:0/638911294 learned_addr learned my addr 172.21.15.123:0/638911294 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:16.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b3400640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85b40c3740 0x7f85b41f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:16.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b3e00640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b414b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:16.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 -- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 msgr2=0x7f85b414b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:16.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b414b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 -- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85b40c3740 msgr2=0x7f85b41f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:16.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85b40c3740 0x7f85b41f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:16.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 -- 172.21.15.123:0/638911294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4156890 con 0x7f85b40c3b50 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b3400640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85b40c3740 0x7f85b41f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85b2a00640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b40c3b50 0x7f85b4155190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f85b40cb640 tx=0x7f859c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ab400640 1 -- 172.21.15.123:0/638911294 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c005ab0 con 0x7f85b40c3b50 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ab400640 1 -- 172.21.15.123:0/638911294 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c005c50 con 0x7f85b40c3b50 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b41546b0 con 0x7f85b40c3b50 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.658+0000 7f85ab400640 1 -- 172.21.15.123:0/638911294 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c007070 con 0x7f85b40c3b50 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.659+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b4151fe0 con 0x7f85b40c3b50 2024-04-25T12:37:16.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.660+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b4074c30 con 0x7f85b40c3b50 2024-04-25T12:37:16.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.661+0000 7f85ab400640 1 -- 172.21.15.123:0/638911294 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f859c007210 con 0x7f85b40c3b50 2024-04-25T12:37:16.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.662+0000 7f85ab400640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85900744e0 0x7f85900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:16.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.662+0000 7f85ab400640 1 -- 172.21.15.123:0/638911294 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6891+0+0 (secure 0 0 0) 0x7f859c095220 con 0x7f85b40c3b50 2024-04-25T12:37:16.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.662+0000 7f85b3400640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85900744e0 0x7f85900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:16.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.663+0000 7f85b3400640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85900744e0 0x7f85900769f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85a4034000 tx=0x7f85a402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:16.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.668+0000 7f85ab400640 1 -- 172.21.15.123:0/638911294 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f859c05ef70 con 0x7f85b40c3b50 2024-04-25T12:37:16.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:16.885+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) -- 0x7f85b4076700 con 0x7f85b40c3b50 2024-04-25T12:37:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.349+0000 7f85ab400640 1 -- 172.21.15.123:0/638911294 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v89) ==== 133+0+0 (secure 0 0 0) 0x7f859c063270 con 0x7f85b40c3b50 2024-04-25T12:37:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-25T12:37:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85900744e0 msgr2=0x7f85900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85900744e0 0x7f85900769f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85a4034000 tx=0x7f85a402f000 comp rx=0 tx=0).stop 2024-04-25T12:37:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b40c3b50 msgr2=0x7f85b4155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b40c3b50 0x7f85b4155190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f85b40cb640 tx=0x7f859c00cbe0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 shutdown_connections 2024-04-25T12:37:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85900744e0 0x7f85900769f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85b415aca0 0x7f85b414b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b40c3b50 0x7f85b4155190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 --2- 172.21.15.123:0/638911294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85b40c3740 0x7f85b41f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.354+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 >> 172.21.15.123:0/638911294 conn(0x7f85b406f440 msgr2=0x7f85b4153c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:17.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.355+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 shutdown_connections 2024-04-25T12:37:17.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.356+0000 7f85ba600640 1 -- 172.21.15.123:0/638911294 wait complete. 2024-04-25T12:37:17.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-25T12:37:17.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.660+0000 7fae23000640 1 Processor -- start 2024-04-25T12:37:17.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.660+0000 7fae23000640 1 -- start start 2024-04-25T12:37:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.662+0000 7fae23000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fae1c0c9f00 0x7fae1c0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.662+0000 7fae23000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae1c074b20 con 0x7fae1c0730e0 2024-04-25T12:37:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.662+0000 7fae23000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae1c0ca780 con 0x7fae1c154650 2024-04-25T12:37:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.662+0000 7fae23000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fae1c0caa60 con 0x7fae1c0c9f00 2024-04-25T12:37:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae21000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fae1c0730e0 0x7fae1c164d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42796/0 (socket says 172.21.15.123:42796) 2024-04-25T12:37:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae21000640 1 -- 172.21.15.123:0/1527573484 learned_addr learned my addr 172.21.15.123:0/1527573484 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1b400640 1 --2- 172.21.15.123:0/1527573484 >> v2:172.21.15.153:3300/0 conn(0x7fae1c0c9f00 0x7fae1c0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1aa00640 1 -- 172.21.15.123:0/1527573484 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2269075362 0 0) 0x7fae1c0ca780 con 0x7fae1c154650 2024-04-25T12:37:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1aa00640 1 -- 172.21.15.123:0/1527573484 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae00003520 con 0x7fae1c154650 2024-04-25T12:37:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1aa00640 1 -- 172.21.15.123:0/1527573484 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155836443 0 0) 0x7fae1c074b20 con 0x7fae1c0730e0 2024-04-25T12:37:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1aa00640 1 -- 172.21.15.123:0/1527573484 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae1c0ca780 con 0x7fae1c0730e0 2024-04-25T12:37:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1b400640 1 -- 172.21.15.123:0/1527573484 >> v1:172.21.15.153:6789/0 conn(0x7fae1c154650 legacy=0x7fae1c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1b400640 1 -- 172.21.15.123:0/1527573484 >> v1:172.21.15.123:6789/0 conn(0x7fae1c0730e0 legacy=0x7fae1c164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.663+0000 7fae1b400640 1 -- 172.21.15.123:0/1527573484 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c15da50 con 0x7fae1c0c9f00 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.664+0000 7fae1b400640 1 --2- 172.21.15.123:0/1527573484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c0c9f00 0x7fae1c0c8170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae080097a0 tx=0x7fae0802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4685aaffc9257b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.665+0000 7fae1aa00640 1 -- 172.21.15.123:0/1527573484 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08037070 con 0x7fae1c0c9f00 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.665+0000 7fae1aa00640 1 -- 172.21.15.123:0/1527573484 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae08032440 con 0x7fae1c0c9f00 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.665+0000 7fae1aa00640 1 -- 172.21.15.123:0/1527573484 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08032810 con 0x7fae1c0c9f00 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.665+0000 7fae23000640 1 -- 172.21.15.123:0/1527573484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c0c9f00 msgr2=0x7fae1c0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.665+0000 7fae23000640 1 --2- 172.21.15.123:0/1527573484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c0c9f00 0x7fae1c0c8170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae080097a0 tx=0x7fae0802a230 comp rx=0 tx=0).stop 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.666+0000 7fae23000640 1 -- 172.21.15.123:0/1527573484 shutdown_connections 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.666+0000 7fae23000640 1 --2- 172.21.15.123:0/1527573484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c0c9f00 0x7fae1c0c8170 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.666+0000 7fae23000640 1 -- 172.21.15.123:0/1527573484 >> 172.21.15.123:0/1527573484 conn(0x7fae1c06f440 msgr2=0x7fae1c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.666+0000 7fae23000640 1 -- 172.21.15.123:0/1527573484 shutdown_connections 2024-04-25T12:37:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.666+0000 7fae23000640 1 -- 172.21.15.123:0/1527573484 wait complete. 2024-04-25T12:37:17.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.668+0000 7fae23000640 1 Processor -- start 2024-04-25T12:37:17.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.668+0000 7fae23000640 1 -- start start 2024-04-25T12:37:17.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae23000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae1c0730e0 0x7fae1c14cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:17.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae23000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae1c0c9f00 0x7fae1c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:17.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae23000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c154650 0x7fae1c15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae23000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c15e2f0 con 0x7fae1c0730e0 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae23000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c15e010 con 0x7fae1c154650 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae23000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c154a60 con 0x7fae1c0c9f00 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae1c0730e0 0x7fae1c14cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae1c0c9f00 0x7fae1c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae21000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c154650 0x7fae1c15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae1c0730e0 0x7fae1c14cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40172/0 (socket says 172.21.15.123:40172) 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1be00640 1 -- 172.21.15.123:0/4291437298 learned_addr learned my addr 172.21.15.123:0/4291437298 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1b400640 1 -- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c154650 msgr2=0x7fae1c15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1b400640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c154650 0x7fae1c15a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1b400640 1 -- 172.21.15.123:0/4291437298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae1c0730e0 msgr2=0x7fae1c14cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1b400640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae1c0730e0 0x7fae1c14cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.670+0000 7fae1b400640 1 -- 172.21.15.123:0/4291437298 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c14d320 con 0x7fae1c0c9f00 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.671+0000 7fae1b400640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae1c0c9f00 0x7fae1c14f910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fae08031000 tx=0x7fae080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.671+0000 7fae13e00640 1 -- 172.21.15.123:0/4291437298 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08041030 con 0x7fae1c0c9f00 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.671+0000 7fae13e00640 1 -- 172.21.15.123:0/4291437298 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0803d420 con 0x7fae1c0c9f00 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.671+0000 7fae13e00640 1 -- 172.21.15.123:0/4291437298 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08037040 con 0x7fae1c0c9f00 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.671+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c15b510 con 0x7fae1c0c9f00 2024-04-25T12:37:17.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.671+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c1f6b60 con 0x7fae1c0c9f00 2024-04-25T12:37:17.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.672+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1c0c66f0 con 0x7fae1c0c9f00 2024-04-25T12:37:17.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.674+0000 7fae13e00640 1 -- 172.21.15.123:0/4291437298 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fae08044070 con 0x7fae1c0c9f00 2024-04-25T12:37:17.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.675+0000 7fae13e00640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fadfc0744b0 0x7fadfc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:17.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.675+0000 7fae13e00640 1 -- 172.21.15.123:0/4291437298 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 6891+0+0 (secure 0 0 0) 0x7fae080b5cf0 con 0x7fae1c0c9f00 2024-04-25T12:37:17.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.675+0000 7fae1be00640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fadfc0744b0 0x7fadfc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:17.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.675+0000 7fae1be00640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fadfc0744b0 0x7fadfc0769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007fd0 tx=0x7fae0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:17.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.680+0000 7fae13e00640 1 -- 172.21.15.123:0/4291437298 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae1c0c66f0 con 0x7fae1c0c9f00 2024-04-25T12:37:17.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:17.892+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7fae1c076700 con 0x7fae1c0c9f00 2024-04-25T12:37:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.365+0000 7fae13e00640 1 -- 172.21.15.123:0/4291437298 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v90) ==== 113+0+0 (secure 0 0 0) 0x7fae0807f8f0 con 0x7fae1c0c9f00 2024-04-25T12:37:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.7 to 1 (10000) 2024-04-25T12:37:18.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fadfc0744b0 msgr2=0x7fadfc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:18.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fadfc0744b0 0x7fadfc0769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007fd0 tx=0x7fae0c009040 comp rx=0 tx=0).stop 2024-04-25T12:37:18.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae1c0c9f00 msgr2=0x7fae1c14f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:18.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae1c0c9f00 0x7fae1c14f910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fae08031000 tx=0x7fae080046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 shutdown_connections 2024-04-25T12:37:18.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fadfc0744b0 0x7fadfc0769c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae1c154650 0x7fae1c15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fae1c0c9f00 0x7fae1c14f910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 --2- 172.21.15.123:0/4291437298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae1c0730e0 0x7fae1c14cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.370+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 >> 172.21.15.123:0/4291437298 conn(0x7fae1c06f440 msgr2=0x7fae1c152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:18.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.371+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 shutdown_connections 2024-04-25T12:37:18.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.371+0000 7fae23000640 1 -- 172.21.15.123:0/4291437298 wait complete. 2024-04-25T12:37:18.399 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-25T12:37:18.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.679+0000 7febe3200640 1 Processor -- start 2024-04-25T12:37:18.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.679+0000 7febe3200640 1 -- start start 2024-04-25T12:37:18.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe3200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe4072cc0 0x7febe406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:18.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe4074290 con 0x7febe40730d0 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe3200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe4074570 con 0x7febe40c7e30 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe3200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7febe4074850 con 0x7febe4072cc0 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe2800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7febe40730d0 0x7febe415dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42804/0 (socket says 172.21.15.123:42804) 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe2800640 1 -- 172.21.15.123:0/30403603 learned_addr learned my addr 172.21.15.123:0/30403603 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe1400640 1 --2- 172.21.15.123:0/30403603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe4072cc0 0x7febe406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.682+0000 7febe0a00640 1 -- 172.21.15.123:0/30403603 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726502360 0 0) 0x7febe4074290 con 0x7febe40730d0 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.683+0000 7febe0a00640 1 -- 172.21.15.123:0/30403603 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febc8003520 con 0x7febe40730d0 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.683+0000 7febe0a00640 1 -- 172.21.15.123:0/30403603 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44934935 0 0) 0x7febe4074570 con 0x7febe40c7e30 2024-04-25T12:37:18.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.683+0000 7febe1400640 1 -- 172.21.15.123:0/30403603 >> v1:172.21.15.153:6789/0 conn(0x7febe40c7e30 legacy=0x7febe4154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:18.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.683+0000 7febe1400640 1 -- 172.21.15.123:0/30403603 >> v1:172.21.15.123:6789/0 conn(0x7febe40730d0 legacy=0x7febe415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:18.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.683+0000 7febe1400640 1 -- 172.21.15.123:0/30403603 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe4074b30 con 0x7febe4072cc0 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.684+0000 7febe1400640 1 --2- 172.21.15.123:0/30403603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe4072cc0 0x7febe406c7b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7febd4009dc0 tx=0x7febd402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b1053f78ddcd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.684+0000 7febe0a00640 1 -- 172.21.15.123:0/30403603 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4007740 con 0x7febe4072cc0 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.684+0000 7febe0a00640 1 -- 172.21.15.123:0/30403603 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd40078e0 con 0x7febe4072cc0 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.684+0000 7febe0a00640 1 -- 172.21.15.123:0/30403603 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4007cb0 con 0x7febe4072cc0 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.685+0000 7febe3200640 1 -- 172.21.15.123:0/30403603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe4072cc0 msgr2=0x7febe406c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.685+0000 7febe3200640 1 --2- 172.21.15.123:0/30403603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe4072cc0 0x7febe406c7b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7febd4009dc0 tx=0x7febd402a6d0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.685+0000 7febe3200640 1 -- 172.21.15.123:0/30403603 shutdown_connections 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.685+0000 7febe3200640 1 --2- 172.21.15.123:0/30403603 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe4072cc0 0x7febe406c7b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.685+0000 7febe3200640 1 -- 172.21.15.123:0/30403603 >> 172.21.15.123:0/30403603 conn(0x7febe4070b70 msgr2=0x7febe4070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.685+0000 7febe3200640 1 -- 172.21.15.123:0/30403603 shutdown_connections 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.685+0000 7febe3200640 1 -- 172.21.15.123:0/30403603 wait complete. 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.687+0000 7febe3200640 1 Processor -- start 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.688+0000 7febe3200640 1 -- start start 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febe4072cc0 0x7febe41f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe3200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe40730d0 0x7febe415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe3200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe40c7e30 0x7febe4075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febe40c83c0 con 0x7febe4072cc0 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe3200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febe40c8240 con 0x7febe40730d0 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe3200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7febe415e380 con 0x7febe40c7e30 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febe4072cc0 0x7febe41f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe2800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe40c7e30 0x7febe4075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe40730d0 0x7febe415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febe4072cc0 0x7febe41f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40184/0 (socket says 172.21.15.123:40184) 2024-04-25T12:37:18.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.689+0000 7febe1e00640 1 -- 172.21.15.123:0/3569612545 learned_addr learned my addr 172.21.15.123:0/3569612545 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:18.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe1e00640 1 -- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe40c7e30 msgr2=0x7febe4075eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:18.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe1e00640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe40c7e30 0x7febe4075eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe1e00640 1 -- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe40730d0 msgr2=0x7febe415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:18.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe1e00640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe40730d0 0x7febe415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:18.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe1e00640 1 -- 172.21.15.123:0/3569612545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe415c2e0 con 0x7febe4072cc0 2024-04-25T12:37:18.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe1400640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe40730d0 0x7febe415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:18.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe1e00640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febe4072cc0 0x7febe41f5620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7febcc00c470 tx=0x7febcc00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:18.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febda000640 1 -- 172.21.15.123:0/3569612545 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc017070 con 0x7febe4072cc0 2024-04-25T12:37:18.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febda000640 1 -- 172.21.15.123:0/3569612545 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc0035f0 con 0x7febe4072cc0 2024-04-25T12:37:18.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febda000640 1 -- 172.21.15.123:0/3569612545 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc0039e0 con 0x7febe4072cc0 2024-04-25T12:37:18.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.690+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe41f26f0 con 0x7febe4072cc0 2024-04-25T12:37:18.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.691+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe41f2cb0 con 0x7febe4072cc0 2024-04-25T12:37:18.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.692+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe4076da0 con 0x7febe4072cc0 2024-04-25T12:37:18.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.693+0000 7febda000640 1 -- 172.21.15.123:0/3569612545 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febcc01a070 con 0x7febe4072cc0 2024-04-25T12:37:18.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.694+0000 7febda000640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febc00744e0 0x7febc00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:18.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.694+0000 7febda000640 1 -- 172.21.15.123:0/3569612545 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6891+0+0 (secure 0 0 0) 0x7febcc0949e0 con 0x7febe4072cc0 2024-04-25T12:37:18.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.694+0000 7febe1400640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febc00744e0 0x7febc00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:18.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.695+0000 7febe1400640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febc00744e0 0x7febc00769f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febd4034000 tx=0x7febd402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:18.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.699+0000 7febda000640 1 -- 172.21.15.123:0/3569612545 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febcc05e5e0 con 0x7febe4072cc0 2024-04-25T12:37:18.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:18.918+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) -- 0x7febe4073690 con 0x7febe4072cc0 2024-04-25T12:37:19.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.372+0000 7febda000640 1 -- 172.21.15.123:0/3569612545 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v91) ==== 133+0+0 (secure 0 0 0) 0x7febcc0628e0 con 0x7febe4072cc0 2024-04-25T12:37:19.372 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-25T12:37:19.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.376+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febc00744e0 msgr2=0x7febc00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:19.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.376+0000 7febe3200640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febc00744e0 0x7febc00769f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febd4034000 tx=0x7febd402f000 comp rx=0 tx=0).stop 2024-04-25T12:37:19.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.376+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febe4072cc0 msgr2=0x7febe41f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:19.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.376+0000 7febe3200640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febe4072cc0 0x7febe41f5620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7febcc00c470 tx=0x7febcc00c930 comp rx=0 tx=0).stop 2024-04-25T12:37:19.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.377+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 shutdown_connections 2024-04-25T12:37:19.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.377+0000 7febe3200640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7febc00744e0 0x7febc00769f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.377+0000 7febe3200640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7febe40c7e30 0x7febe4075eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.377+0000 7febe3200640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe40730d0 0x7febe415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.377+0000 7febe3200640 1 --2- 172.21.15.123:0/3569612545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febe4072cc0 0x7febe41f5620 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.377+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 >> 172.21.15.123:0/3569612545 conn(0x7febe4070b70 msgr2=0x7febe41e3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:19.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.378+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 shutdown_connections 2024-04-25T12:37:19.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.378+0000 7febe3200640 1 -- 172.21.15.123:0/3569612545 wait complete. 2024-04-25T12:37:19.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-25T12:37:19.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.688+0000 7fe1a1600640 1 Processor -- start 2024-04-25T12:37:19.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.688+0000 7fe1a1600640 1 -- start start 2024-04-25T12:37:19.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.690+0000 7fe1a1600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe19c0730e0 0x7fe19c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.690+0000 7fe1a1600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19c074b20 con 0x7fe19c154650 2024-04-25T12:37:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.690+0000 7fe1a1600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19c0c8880 con 0x7fe19c0c9f00 2024-04-25T12:37:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.690+0000 7fe1a1600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe19c0ca780 con 0x7fe19c0730e0 2024-04-25T12:37:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe19b400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe19c0730e0 0x7fe19c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe19aa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe19c154650 0x7fe19c15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42806/0 (socket says 172.21.15.123:42806) 2024-04-25T12:37:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe19aa00640 1 -- 172.21.15.123:0/1281405365 learned_addr learned my addr 172.21.15.123:0/1281405365 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe19a000640 1 --1- 172.21.15.123:0/1281405365 >> v1:172.21.15.153:6789/0 conn(0x7fe19c0c9f00 0x7fe19c0c8170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:45930/0 (socket says 172.21.15.123:45930) 2024-04-25T12:37:19.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe199600640 1 -- 172.21.15.123:0/1281405365 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1373705226 0 0) 0x7fe19c0c8880 con 0x7fe19c0c9f00 2024-04-25T12:37:19.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe199600640 1 -- 172.21.15.123:0/1281405365 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe180003520 con 0x7fe19c0c9f00 2024-04-25T12:37:19.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe199600640 1 -- 172.21.15.123:0/1281405365 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3126138941 0 0) 0x7fe19c074b20 con 0x7fe19c154650 2024-04-25T12:37:19.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe19b400640 1 -- 172.21.15.123:0/1281405365 >> v1:172.21.15.153:6789/0 conn(0x7fe19c0c9f00 legacy=0x7fe19c0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:19.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe19b400640 1 -- 172.21.15.123:0/1281405365 >> v1:172.21.15.123:6789/0 conn(0x7fe19c154650 legacy=0x7fe19c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:19.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.691+0000 7fe19b400640 1 -- 172.21.15.123:0/1281405365 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c0caa60 con 0x7fe19c0730e0 2024-04-25T12:37:19.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.692+0000 7fe19b400640 1 --2- 172.21.15.123:0/1281405365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c0734f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe194009d90 tx=0x7fe19402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=164951b97603e4a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:19.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.693+0000 7fe199600640 1 -- 172.21.15.123:0/1281405365 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194007740 con 0x7fe19c0730e0 2024-04-25T12:37:19.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.693+0000 7fe199600640 1 -- 172.21.15.123:0/1281405365 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1940078e0 con 0x7fe19c0730e0 2024-04-25T12:37:19.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.693+0000 7fe199600640 1 -- 172.21.15.123:0/1281405365 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194007cb0 con 0x7fe19c0730e0 2024-04-25T12:37:19.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.694+0000 7fe1a1600640 1 -- 172.21.15.123:0/1281405365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 msgr2=0x7fe19c0734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:19.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.694+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1281405365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c0734f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe194009d90 tx=0x7fe19402a6f0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.694+0000 7fe1a1600640 1 -- 172.21.15.123:0/1281405365 shutdown_connections 2024-04-25T12:37:19.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.694+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1281405365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c0734f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.694+0000 7fe1a1600640 1 -- 172.21.15.123:0/1281405365 >> 172.21.15.123:0/1281405365 conn(0x7fe19c06f440 msgr2=0x7fe19c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:19.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.694+0000 7fe1a1600640 1 -- 172.21.15.123:0/1281405365 shutdown_connections 2024-04-25T12:37:19.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.694+0000 7fe1a1600640 1 -- 172.21.15.123:0/1281405365 wait complete. 2024-04-25T12:37:19.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.696+0000 7fe1a1600640 1 Processor -- start 2024-04-25T12:37:19.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.696+0000 7fe1a1600640 1 -- start start 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe1a1600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c1f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe1a1600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe19c0c9f00 0x7fe19c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe1a1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe19c154650 0x7fe19c1e8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe1a1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c15e010 con 0x7fe19c154650 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe1a1600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c15dd30 con 0x7fe19c0730e0 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe1a1600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c15e2f0 con 0x7fe19c0c9f00 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe19b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe19c154650 0x7fe19c1e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:19.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe19aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c1f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe19b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe19c154650 0x7fe19c1e8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40200/0 (socket says 172.21.15.123:40200) 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe19b400640 1 -- 172.21.15.123:0/1992142915 learned_addr learned my addr 172.21.15.123:0/1992142915 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.698+0000 7fe19a000640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe19c0c9f00 0x7fe19c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe19aa00640 1 -- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe19c0c9f00 msgr2=0x7fe19c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe19aa00640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe19c0c9f00 0x7fe19c15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe19aa00640 1 -- 172.21.15.123:0/1992142915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe19c154650 msgr2=0x7fe19c1e8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe19aa00640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe19c154650 0x7fe19c1e8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe19aa00640 1 -- 172.21.15.123:0/1992142915 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c15c440 con 0x7fe19c0730e0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe19aa00640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c1f2e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c440 tx=0x7fe18c00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe192a00640 1 -- 172.21.15.123:0/1992142915 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0074e0 con 0x7fe19c0730e0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe192a00640 1 -- 172.21.15.123:0/1992142915 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18c0047e0 con 0x7fe19c0730e0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe192a00640 1 -- 172.21.15.123:0/1992142915 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c004bd0 con 0x7fe19c0730e0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1f2400 con 0x7fe19c0730e0 2024-04-25T12:37:19.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.699+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1e8850 con 0x7fe19c0730e0 2024-04-25T12:37:19.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.700+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c076700 con 0x7fe19c0730e0 2024-04-25T12:37:19.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.702+0000 7fe192a00640 1 -- 172.21.15.123:0/1992142915 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe18c011070 con 0x7fe19c0730e0 2024-04-25T12:37:19.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.703+0000 7fe192a00640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe17c0744b0 0x7fe17c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:19.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.703+0000 7fe192a00640 1 -- 172.21.15.123:0/1992142915 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6891+0+0 (secure 0 0 0) 0x7fe18c095120 con 0x7fe19c0730e0 2024-04-25T12:37:19.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.703+0000 7fe19a000640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe17c0744b0 0x7fe17c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:19.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.704+0000 7fe19a000640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe17c0744b0 0x7fe17c0769c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe19c14af50 tx=0x7fe188007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:19.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.708+0000 7fe192a00640 1 -- 172.21.15.123:0/1992142915 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe18c05ee70 con 0x7fe19c0730e0 2024-04-25T12:37:19.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:19.919+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7fe19c0744e0 con 0x7fe19c0730e0 2024-04-25T12:37:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.388+0000 7fe192a00640 1 -- 172.21.15.123:0/1992142915 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v92) ==== 113+0+0 (secure 0 0 0) 0x7fe18c063170 con 0x7fe19c0730e0 2024-04-25T12:37:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:reweighted osd.5 to 1 (10000) 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.392+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe17c0744b0 msgr2=0x7fe17c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.392+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe17c0744b0 0x7fe17c0769c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe19c14af50 tx=0x7fe188007d30 comp rx=0 tx=0).stop 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.392+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 msgr2=0x7fe19c1f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.392+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c1f2e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c440 tx=0x7fe18c00c900 comp rx=0 tx=0).stop 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.392+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 shutdown_connections 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.393+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe17c0744b0 0x7fe17c0769c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.393+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe19c154650 0x7fe19c1e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.393+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe19c0c9f00 0x7fe19c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.393+0000 7fe1a1600640 1 --2- 172.21.15.123:0/1992142915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe19c0730e0 0x7fe19c1f2e70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.393+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 >> 172.21.15.123:0/1992142915 conn(0x7fe19c06f440 msgr2=0x7fe19c075d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.393+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 shutdown_connections 2024-04-25T12:37:20.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.393+0000 7fe1a1600640 1 -- 172.21.15.123:0/1992142915 wait complete. 2024-04-25T12:37:20.421 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-25T12:37:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.697+0000 7f8129000640 1 Processor -- start 2024-04-25T12:37:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.697+0000 7f8129000640 1 -- start start 2024-04-25T12:37:20.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.699+0000 7f8129000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8124072cc0 0x7f812406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:20.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.699+0000 7f8129000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8124074290 con 0x7f81240730d0 2024-04-25T12:37:20.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.699+0000 7f8129000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8124074570 con 0x7f81240c7e30 2024-04-25T12:37:20.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.699+0000 7f8129000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8124074850 con 0x7f8124072cc0 2024-04-25T12:37:20.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.699+0000 7f8123e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f81240730d0 0x7f812415dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42818/0 (socket says 172.21.15.123:42818) 2024-04-25T12:37:20.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.699+0000 7f8123e00640 1 -- 172.21.15.123:0/4180712582 learned_addr learned my addr 172.21.15.123:0/4180712582 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:20.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.699+0000 7f8122a00640 1 --2- 172.21.15.123:0/4180712582 >> v2:172.21.15.153:3300/0 conn(0x7f8124072cc0 0x7f812406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:20.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.700+0000 7f8122000640 1 -- 172.21.15.123:0/4180712582 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565380747 0 0) 0x7f8124074290 con 0x7f81240730d0 2024-04-25T12:37:20.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.700+0000 7f8122a00640 1 -- 172.21.15.123:0/4180712582 >> v1:172.21.15.153:6789/0 conn(0x7f81240c7e30 legacy=0x7f8124154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:20.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.700+0000 7f8122a00640 1 -- 172.21.15.123:0/4180712582 >> v1:172.21.15.123:6789/0 conn(0x7f81240730d0 legacy=0x7f812415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:20.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.700+0000 7f8122a00640 1 -- 172.21.15.123:0/4180712582 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8124074b30 con 0x7f8124072cc0 2024-04-25T12:37:20.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.701+0000 7f8122a00640 1 --2- 172.21.15.123:0/4180712582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 0x7f812406c7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f81100097a0 tx=0x7f811002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31db96cfc7e2c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:20.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.701+0000 7f8122000640 1 -- 172.21.15.123:0/4180712582 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110037070 con 0x7f8124072cc0 2024-04-25T12:37:20.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.701+0000 7f8122000640 1 -- 172.21.15.123:0/4180712582 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8110032440 con 0x7f8124072cc0 2024-04-25T12:37:20.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.701+0000 7f8122000640 1 -- 172.21.15.123:0/4180712582 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110032810 con 0x7f8124072cc0 2024-04-25T12:37:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.702+0000 7f8129000640 1 -- 172.21.15.123:0/4180712582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 msgr2=0x7f812406c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.702+0000 7f8129000640 1 --2- 172.21.15.123:0/4180712582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 0x7f812406c7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f81100097a0 tx=0x7f811002a230 comp rx=0 tx=0).stop 2024-04-25T12:37:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.702+0000 7f8129000640 1 -- 172.21.15.123:0/4180712582 shutdown_connections 2024-04-25T12:37:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.702+0000 7f8129000640 1 --2- 172.21.15.123:0/4180712582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 0x7f812406c7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.702+0000 7f8129000640 1 -- 172.21.15.123:0/4180712582 >> 172.21.15.123:0/4180712582 conn(0x7f8124070b70 msgr2=0x7f8124070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.702+0000 7f8129000640 1 -- 172.21.15.123:0/4180712582 shutdown_connections 2024-04-25T12:37:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.702+0000 7f8129000640 1 -- 172.21.15.123:0/4180712582 wait complete. 2024-04-25T12:37:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.705+0000 7f8129000640 1 Processor -- start 2024-04-25T12:37:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.705+0000 7f8129000640 1 -- start start 2024-04-25T12:37:20.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.706+0000 7f8129000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 0x7f81241f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:20.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.706+0000 7f8129000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81240730d0 0x7f81240c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:20.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.706+0000 7f8129000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81240c7e30 0x7f812415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.706+0000 7f8129000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81240c83c0 con 0x7f81240730d0 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.706+0000 7f8129000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f81240c8240 con 0x7f8124072cc0 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.706+0000 7f8129000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f812415e380 con 0x7f81240c7e30 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.706+0000 7f8122a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81240730d0 0x7f81240c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81240730d0 0x7f81240c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40216/0 (socket says 172.21.15.123:40216) 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 -- 172.21.15.123:0/938996069 learned_addr learned my addr 172.21.15.123:0/938996069 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8123400640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 0x7f81241f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:20.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8123e00640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81240c7e30 0x7f812415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:20.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 -- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81240c7e30 msgr2=0x7f812415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:20.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81240c7e30 0x7f812415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:20.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 -- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 msgr2=0x7f81241f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:20.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 0x7f81241f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:20.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 -- 172.21.15.123:0/938996069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812415bb40 con 0x7f81240730d0 2024-04-25T12:37:20.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.707+0000 7f8122a00640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81240730d0 0x7f81240c46e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f81100097a0 tx=0x7f81100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:20.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.708+0000 7f811b400640 1 -- 172.21.15.123:0/938996069 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110037050 con 0x7f81240730d0 2024-04-25T12:37:20.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.708+0000 7f811b400640 1 -- 172.21.15.123:0/938996069 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8110038040 con 0x7f81240730d0 2024-04-25T12:37:20.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.708+0000 7f811b400640 1 -- 172.21.15.123:0/938996069 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110004490 con 0x7f81240730d0 2024-04-25T12:37:20.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.708+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812415c530 con 0x7f81240730d0 2024-04-25T12:37:20.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.708+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81240c5790 con 0x7f81240730d0 2024-04-25T12:37:20.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.709+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8124076da0 con 0x7f81240730d0 2024-04-25T12:37:20.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.710+0000 7f811b400640 1 -- 172.21.15.123:0/938996069 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8110030900 con 0x7f81240730d0 2024-04-25T12:37:20.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.712+0000 7f811b400640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f81040744e0 0x7f81040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:20.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.712+0000 7f811b400640 1 -- 172.21.15.123:0/938996069 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6891+0+0 (secure 0 0 0) 0x7f81100b5fd0 con 0x7f81240730d0 2024-04-25T12:37:20.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.712+0000 7f8123400640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f81040744e0 0x7f81040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:20.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.713+0000 7f8123400640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f81040744e0 0x7f81040769f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8114000f60 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.720+0000 7f811b400640 1 -- 172.21.15.123:0/938996069 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f811004c020 con 0x7f81240730d0 2024-04-25T12:37:20.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:20.937+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7f81240caa80 con 0x7f81240730d0 2024-04-25T12:37:21.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.399+0000 7f811b400640 1 -- 172.21.15.123:0/938996069 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v93) ==== 133+0+0 (secure 0 0 0) 0x7f8110088330 con 0x7f81240730d0 2024-04-25T12:37:21.399 INFO:teuthology.orchestra.run.smithi123.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-25T12:37:21.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.404+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f81040744e0 msgr2=0x7f81040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.404+0000 7f8129000640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f81040744e0 0x7f81040769f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8114000f60 tx=0x7f8114009040 comp rx=0 tx=0).stop 2024-04-25T12:37:21.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.404+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81240730d0 msgr2=0x7f81240c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.404+0000 7f8129000640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81240730d0 0x7f81240c46e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f81100097a0 tx=0x7f81100046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.405+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 shutdown_connections 2024-04-25T12:37:21.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.405+0000 7f8129000640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f81040744e0 0x7f81040769f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.405+0000 7f8129000640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f81240c7e30 0x7f812415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.405+0000 7f8129000640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81240730d0 0x7f81240c46e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.405+0000 7f8129000640 1 --2- 172.21.15.123:0/938996069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8124072cc0 0x7f81241f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.405+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 >> 172.21.15.123:0/938996069 conn(0x7f8124070b70 msgr2=0x7f81240c2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:21.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.405+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 shutdown_connections 2024-04-25T12:37:21.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.406+0000 7f8129000640 1 -- 172.21.15.123:0/938996069 wait complete. 2024-04-25T12:37:21.438 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-25T12:37:21.438 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-25T12:37:21.438 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-25T12:37:21.438 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-25T12:37:21.439 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T12:37:21.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:37:21.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.718+0000 7fd065800640 1 Processor -- start 2024-04-25T12:37:21.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.718+0000 7fd065800640 1 -- start start 2024-04-25T12:37:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.720+0000 7fd065800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd060074870 0x7fd060154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.720+0000 7fd065800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd06015b3b0 con 0x7fd06015afa0 2024-04-25T12:37:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd065800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd060073ce0 con 0x7fd0600c4d50 2024-04-25T12:37:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd065800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd0600c61f0 con 0x7fd060074870 2024-04-25T12:37:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd057e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd060074870 0x7fd060154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd057e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd060074870 0x7fd060154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40224/0 (socket says 172.21.15.123:40224) 2024-04-25T12:37:21.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd057e00640 1 -- 172.21.15.123:0/1043059397 learned_addr learned my addr 172.21.15.123:0/1043059397 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:21.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd057e00640 1 -- 172.21.15.123:0/1043059397 >> v1:172.21.15.153:6789/0 conn(0x7fd0600c4d50 legacy=0x7fd0600c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd057e00640 1 -- 172.21.15.123:0/1043059397 >> v1:172.21.15.123:6789/0 conn(0x7fd06015afa0 legacy=0x7fd0600cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:21.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.721+0000 7fd057e00640 1 -- 172.21.15.123:0/1043059397 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd060073fc0 con 0x7fd060074870 2024-04-25T12:37:21.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.722+0000 7fd057e00640 1 --2- 172.21.15.123:0/1043059397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd060074870 0x7fd060154890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd048009870 tx=0x7fd04802a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336b35a7b973329b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:21.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.723+0000 7fd05e000640 1 -- 172.21.15.123:0/1043059397 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048037070 con 0x7fd060074870 2024-04-25T12:37:21.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.723+0000 7fd05e000640 1 -- 172.21.15.123:0/1043059397 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd048032640 con 0x7fd060074870 2024-04-25T12:37:21.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.723+0000 7fd05e000640 1 -- 172.21.15.123:0/1043059397 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048032a10 con 0x7fd060074870 2024-04-25T12:37:21.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.723+0000 7fd065800640 1 -- 172.21.15.123:0/1043059397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd060074870 msgr2=0x7fd060154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.723+0000 7fd065800640 1 --2- 172.21.15.123:0/1043059397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd060074870 0x7fd060154890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd048009870 tx=0x7fd04802a3c0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.724+0000 7fd065800640 1 -- 172.21.15.123:0/1043059397 shutdown_connections 2024-04-25T12:37:21.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.724+0000 7fd065800640 1 --2- 172.21.15.123:0/1043059397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd060074870 0x7fd060154890 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.724+0000 7fd065800640 1 -- 172.21.15.123:0/1043059397 >> 172.21.15.123:0/1043059397 conn(0x7fd06006c750 msgr2=0x7fd06006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:21.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.724+0000 7fd065800640 1 -- 172.21.15.123:0/1043059397 shutdown_connections 2024-04-25T12:37:21.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.724+0000 7fd065800640 1 -- 172.21.15.123:0/1043059397 wait complete. 2024-04-25T12:37:21.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.726+0000 7fd065800640 1 Processor -- start 2024-04-25T12:37:21.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.726+0000 7fd065800640 1 -- start start 2024-04-25T12:37:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd065800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd060074870 0x7fd0601f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd065800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0600c4d50 0x7fd06015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd065800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd06015afa0 0x7fd0600cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd065800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0600c8460 con 0x7fd0600c4d50 2024-04-25T12:37:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd065800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0600c87d0 con 0x7fd06015afa0 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd065800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd0600c9370 con 0x7fd060074870 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd057e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0600c4d50 0x7fd06015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd05ea00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd060074870 0x7fd0601f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd057e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0600c4d50 0x7fd06015b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40234/0 (socket says 172.21.15.123:40234) 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd057e00640 1 -- 172.21.15.123:0/2539804976 learned_addr learned my addr 172.21.15.123:0/2539804976 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd05f400640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd06015afa0 0x7fd0600cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd057e00640 1 -- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd060074870 msgr2=0x7fd0601f0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd057e00640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd060074870 0x7fd0601f0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd057e00640 1 -- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd06015afa0 msgr2=0x7fd0600cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.728+0000 7fd057e00640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd06015afa0 0x7fd0600cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.729+0000 7fd057e00640 1 -- 172.21.15.123:0/2539804976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06015e070 con 0x7fd0600c4d50 2024-04-25T12:37:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.729+0000 7fd057e00640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0600c4d50 0x7fd06015b890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd048000c00 tx=0x7fd0480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.729+0000 7fd056a00640 1 -- 172.21.15.123:0/2539804976 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048037040 con 0x7fd0600c4d50 2024-04-25T12:37:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.729+0000 7fd056a00640 1 -- 172.21.15.123:0/2539804976 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd048042070 con 0x7fd0600c4d50 2024-04-25T12:37:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.729+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0601f44e0 con 0x7fd0600c4d50 2024-04-25T12:37:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.729+0000 7fd056a00640 1 -- 172.21.15.123:0/2539804976 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048030bd0 con 0x7fd0600c4d50 2024-04-25T12:37:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.730+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0601f4aa0 con 0x7fd0600c4d50 2024-04-25T12:37:21.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.731+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0600cd220 con 0x7fd0600c4d50 2024-04-25T12:37:21.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.732+0000 7fd056a00640 1 -- 172.21.15.123:0/2539804976 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd048032500 con 0x7fd0600c4d50 2024-04-25T12:37:21.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.733+0000 7fd056a00640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd03c0744b0 0x7fd03c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:21.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.733+0000 7fd056a00640 1 -- 172.21.15.123:0/2539804976 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fd0480b6120 con 0x7fd0600c4d50 2024-04-25T12:37:21.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.733+0000 7fd05ea00640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd03c0744b0 0x7fd03c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:21.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.734+0000 7fd05ea00640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd03c0744b0 0x7fd03c0769c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd050000f60 tx=0x7fd050009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:21.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.737+0000 7fd056a00640 1 -- 172.21.15.123:0/2539804976 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd04807fd20 con 0x7fd0600c4d50 2024-04-25T12:37:21.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.941+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd060076700 con 0x7fd0600c4d50 2024-04-25T12:37:21.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.945+0000 7fd056a00640 1 -- 172.21.15.123:0/2539804976 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+19058 (secure 0 0 0) 0x7fd048084020 con 0x7fd0600c4d50 2024-04-25T12:37:21.945 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:21.945 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":93,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:37:21.396436+0000","last_up_change":"2024-04-25T12:35:59.575545+0000","last_in_change":"2024-04-25T12:37:00.163243+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":39,"up_from":47,"up_thru":76,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6827","nonce":4270711740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6829","nonce":4270711740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6833","nonce":4270711740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6831","nonce":4270711740}]},"public_addr":"172.21.15.153:6827/4270711740","cluster_addr":"172.21.15.153:6829/4270711740","heartbeat_back_addr":"172.21.15.153:6833/4270711740","heartbeat_front_addr":"172.21.15.153:6831/4270711740","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":40},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[3,1]},{"pgid":"4.15","osds":[1,0,7,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":5,"to":4}]},{"pgid":"2.c","mappings":[{"from":0,"to":2}]},{"pgid":"3.3","mappings":[{"from":0,"to":7}]},{"pgid":"3.c","mappings":[{"from":0,"to":1}]},{"pgid":"3.13","mappings":[{"from":0,"to":1}]},{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"3.17","mappings":[{"from":3,"to":7}]},{"pgid":"3.26","mappings":[{"from":3,"to":1}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.947+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd03c0744b0 msgr2=0x7fd03c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.947+0000 7fd065800640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd03c0744b0 0x7fd03c0769c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd050000f60 tx=0x7fd050009040 comp rx=0 tx=0).stop 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.947+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0600c4d50 msgr2=0x7fd06015b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.947+0000 7fd065800640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0600c4d50 0x7fd06015b890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd048000c00 tx=0x7fd0480046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 shutdown_connections 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd03c0744b0 0x7fd03c0769c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd06015afa0 0x7fd0600cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0600c4d50 0x7fd06015b890 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 --2- 172.21.15.123:0/2539804976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd060074870 0x7fd0601f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 >> 172.21.15.123:0/2539804976 conn(0x7fd06006c750 msgr2=0x7fd06014dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 shutdown_connections 2024-04-25T12:37:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:21.948+0000 7fd065800640 1 -- 172.21.15.123:0/2539804976 wait complete. 2024-04-25T12:37:21.975 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T12:37:21.975 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:37:22.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.249+0000 7fe7c1600640 1 Processor -- start 2024-04-25T12:37:22.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.249+0000 7fe7c1600640 1 -- start start 2024-04-25T12:37:22.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c1600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe7bc0c94f0 0x7fe7bc0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7bc0c2720 0x7fe7bc0c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c1600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7bc1593c0 con 0x7fe7bc072d70 2024-04-25T12:37:22.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c1600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe7bc159fd0 con 0x7fe7bc0c2720 2024-04-25T12:37:22.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c1600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe7bc1596a0 con 0x7fe7bc0c94f0 2024-04-25T12:37:22.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c0c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7bc0c2720 0x7fe7bc0c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c0c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7bc0c2720 0x7fe7bc0c2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40238/0 (socket says 172.21.15.123:40238) 2024-04-25T12:37:22.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7c0c00640 1 -- 172.21.15.123:0/609180887 learned_addr learned my addr 172.21.15.123:0/609180887 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.251+0000 7fe7bb400640 1 --2- 172.21.15.123:0/609180887 >> v2:172.21.15.153:3300/0 conn(0x7fe7bc0c94f0 0x7fe7bc0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.252+0000 7fe7c0c00640 1 -- 172.21.15.123:0/609180887 >> v2:172.21.15.153:3300/0 conn(0x7fe7bc0c94f0 msgr2=0x7fe7bc0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.252+0000 7fe7c0c00640 1 --2- 172.21.15.123:0/609180887 >> v2:172.21.15.153:3300/0 conn(0x7fe7bc0c94f0 0x7fe7bc0c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.252+0000 7fe7c0c00640 1 -- 172.21.15.123:0/609180887 >> v1:172.21.15.153:6789/0 conn(0x7fe7bc072d70 legacy=0x7fe7bc1420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.252+0000 7fe7c0c00640 1 -- 172.21.15.123:0/609180887 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc159980 con 0x7fe7bc0c2720 2024-04-25T12:37:22.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.252+0000 7fe7bb400640 1 --2- 172.21.15.123:0/609180887 >> v2:172.21.15.153:3300/0 conn(0x7fe7bc0c94f0 0x7fe7bc0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.253+0000 7fe7c0c00640 1 --2- 172.21.15.123:0/609180887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc0c2720 0x7fe7bc0c2b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc072260 tx=0x7fe7b402c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379dfdbdec844f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.253+0000 7fe7baa00640 1 -- 172.21.15.123:0/609180887 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4035020 con 0x7fe7bc0c2720 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.253+0000 7fe7baa00640 1 -- 172.21.15.123:0/609180887 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b4009600 con 0x7fe7bc0c2720 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.253+0000 7fe7baa00640 1 -- 172.21.15.123:0/609180887 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40099d0 con 0x7fe7bc0c2720 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 -- 172.21.15.123:0/609180887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc0c2720 msgr2=0x7fe7bc0c2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 --2- 172.21.15.123:0/609180887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc0c2720 0x7fe7bc0c2b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc072260 tx=0x7fe7b402c9e0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 -- 172.21.15.123:0/609180887 shutdown_connections 2024-04-25T12:37:22.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 --2- 172.21.15.123:0/609180887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc0c2720 0x7fe7bc0c2b30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 --2- 172.21.15.123:0/609180887 >> v2:172.21.15.153:3300/0 conn(0x7fe7bc0c94f0 0x7fe7bc0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 -- 172.21.15.123:0/609180887 >> 172.21.15.123:0/609180887 conn(0x7fe7bc0cd600 msgr2=0x7fe7bc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 -- 172.21.15.123:0/609180887 shutdown_connections 2024-04-25T12:37:22.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.254+0000 7fe7c1600640 1 -- 172.21.15.123:0/609180887 wait complete. 2024-04-25T12:37:22.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.257+0000 7fe7c1600640 1 Processor -- start 2024-04-25T12:37:22.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.257+0000 7fe7c1600640 1 -- start start 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.258+0000 7fe7c1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc072d70 0x7fe7bc0c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.258+0000 7fe7c1600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7bc0c2720 0x7fe7bc13d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.258+0000 7fe7c1600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc0c94f0 0x7fe7bc15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.258+0000 7fe7c1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc0c8280 con 0x7fe7bc072d70 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.258+0000 7fe7c1600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc0c8100 con 0x7fe7bc0c94f0 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.258+0000 7fe7c1600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe7bc0c8400 con 0x7fe7bc0c2720 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.258+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc072d70 0x7fe7bc0c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc072d70 0x7fe7bc0c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40250/0 (socket says 172.21.15.123:40250) 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 -- 172.21.15.123:0/3952739812 learned_addr learned my addr 172.21.15.123:0/3952739812 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7c0c00640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc0c94f0 0x7fe7bc15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bb400640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7bc0c2720 0x7fe7bc13d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 -- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7bc0c2720 msgr2=0x7fe7bc13d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7bc0c2720 0x7fe7bc13d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 -- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc0c94f0 msgr2=0x7fe7bc15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc0c94f0 0x7fe7bc15c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 -- 172.21.15.123:0/3952739812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc0c5390 con 0x7fe7bc072d70 2024-04-25T12:37:22.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.259+0000 7fe7bbe00640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc072d70 0x7fe7bc0c4e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c3f0 tx=0x7fe7ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.260+0000 7fe7b3e00640 1 -- 172.21.15.123:0/3952739812 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0052d0 con 0x7fe7bc072d70 2024-04-25T12:37:22.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.260+0000 7fe7b3e00640 1 -- 172.21.15.123:0/3952739812 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac005490 con 0x7fe7bc072d70 2024-04-25T12:37:22.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.260+0000 7fe7b3e00640 1 -- 172.21.15.123:0/3952739812 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac007260 con 0x7fe7bc072d70 2024-04-25T12:37:22.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.260+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc1e25e0 con 0x7fe7bc072d70 2024-04-25T12:37:22.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.260+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc1e2880 con 0x7fe7bc072d70 2024-04-25T12:37:22.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.261+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7bc0c5560 con 0x7fe7bc072d70 2024-04-25T12:37:22.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.262+0000 7fe7b3e00640 1 -- 172.21.15.123:0/3952739812 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe7ac007400 con 0x7fe7bc072d70 2024-04-25T12:37:22.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.263+0000 7fe7b3e00640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe79c0744e0 0x7fe79c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.263+0000 7fe7b3e00640 1 -- 172.21.15.123:0/3952739812 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fe7ac094b50 con 0x7fe7bc072d70 2024-04-25T12:37:22.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.264+0000 7fe7bb400640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe79c0744e0 0x7fe79c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.264+0000 7fe7bb400640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe79c0744e0 0x7fe79c0769f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0cd380 tx=0x7fe7a8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.268+0000 7fe7b3e00640 1 -- 172.21.15.123:0/3952739812 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7ac05e750 con 0x7fe7bc072d70 2024-04-25T12:37:22.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.468+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7bc076da0 con 0x7fe7bc072d70 2024-04-25T12:37:22.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.470+0000 7fe7b3e00640 1 -- 172.21.15.123:0/3952739812 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+19058 (secure 0 0 0) 0x7fe7ac062a50 con 0x7fe7bc072d70 2024-04-25T12:37:22.470 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:22.470 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":93,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:37:21.396436+0000","last_up_change":"2024-04-25T12:35:59.575545+0000","last_in_change":"2024-04-25T12:37:00.163243+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":39,"up_from":47,"up_thru":76,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6827","nonce":4270711740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6829","nonce":4270711740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6833","nonce":4270711740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6831","nonce":4270711740}]},"public_addr":"172.21.15.153:6827/4270711740","cluster_addr":"172.21.15.153:6829/4270711740","heartbeat_back_addr":"172.21.15.153:6833/4270711740","heartbeat_front_addr":"172.21.15.153:6831/4270711740","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":40},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[3,1]},{"pgid":"4.15","osds":[1,0,7,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":5,"to":4}]},{"pgid":"2.c","mappings":[{"from":0,"to":2}]},{"pgid":"3.3","mappings":[{"from":0,"to":7}]},{"pgid":"3.c","mappings":[{"from":0,"to":1}]},{"pgid":"3.13","mappings":[{"from":0,"to":1}]},{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"3.17","mappings":[{"from":3,"to":7}]},{"pgid":"3.26","mappings":[{"from":3,"to":1}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:37:22.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe79c0744e0 msgr2=0x7fe79c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe79c0744e0 0x7fe79c0769f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0cd380 tx=0x7fe7a8009190 comp rx=0 tx=0).stop 2024-04-25T12:37:22.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc072d70 msgr2=0x7fe7bc0c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc072d70 0x7fe7bc0c4e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c3f0 tx=0x7fe7ac00c8b0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 shutdown_connections 2024-04-25T12:37:22.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe79c0744e0 0x7fe79c0769f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc0c94f0 0x7fe7bc15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7bc0c2720 0x7fe7bc13d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 --2- 172.21.15.123:0/3952739812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7bc072d70 0x7fe7bc0c4e50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 >> 172.21.15.123:0/3952739812 conn(0x7fe7bc0cd600 msgr2=0x7fe7bc0c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 shutdown_connections 2024-04-25T12:37:22.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.473+0000 7fe7c1600640 1 -- 172.21.15.123:0/3952739812 wait complete. 2024-04-25T12:37:22.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T12:37:22.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T12:37:22.499 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T12:37:22.499 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T12:37:22.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T12:37:22.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T12:37:22.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T12:37:22.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T12:37:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.914+0000 7ff1e3c00640 1 Processor -- start 2024-04-25T12:37:22.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.915+0000 7ff1e3c00640 1 -- start start 2024-04-25T12:37:22.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.915+0000 7f3124a00640 1 Processor -- start 2024-04-25T12:37:22.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.915+0000 7f3124a00640 1 -- start start 2024-04-25T12:37:22.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.916+0000 7fdf40e00640 1 Processor -- start 2024-04-25T12:37:22.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.916+0000 7fdf40e00640 1 -- start start 2024-04-25T12:37:22.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7f258d600640 1 Processor -- start 2024-04-25T12:37:22.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7f3124a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3120165e30 0x7f3120168250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7ff1e3c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff1dc0743b0 0x7ff1dc0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7ff1e3c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0c9710 0x7ff1dc0c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7ff1e3c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1dc071300 con 0x7ff1dc0ca130 2024-04-25T12:37:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7ff1e3c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff1dc071880 con 0x7ff1dc0743b0 2024-04-25T12:37:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7ff1e3c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc0c8e40 con 0x7ff1dc0c9710 2024-04-25T12:37:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7ff1dbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0c9710 0x7ff1dc0c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7ff1dbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0c9710 0x7ff1dc0c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56758/0 (socket says 172.21.15.123:56758) 2024-04-25T12:37:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7ff1dbe00640 1 -- 172.21.15.123:0/3158263577 learned_addr learned my addr 172.21.15.123:0/3158263577 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.918+0000 7f258d600640 1 -- start start 2024-04-25T12:37:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf40e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf40e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf3c0c3b50 0x7fdf3c16b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf39e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf39e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56742/0 (socket says 172.21.15.123:56742) 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf39e00640 1 -- 172.21.15.123:0/4058579570 learned_addr learned my addr 172.21.15.123:0/4058579570 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf3c15b0b0 con 0x7fdf3c0c3740 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdf3c074170 con 0x7fdf3c0c3b50 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.919+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3c073230 con 0x7fdf3c15aca0 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7fdf3a800640 1 --2- 172.21.15.123:0/4058579570 >> v2:172.21.15.123:3300/0 conn(0x7fdf3c0c3b50 0x7fdf3c16b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7ff1e1200640 1 --2- 172.21.15.123:0/3158263577 >> v2:172.21.15.153:3300/0 conn(0x7ff1dc0743b0 0x7ff1dc0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7fdf39e00640 1 -- 172.21.15.123:0/4058579570 >> v2:172.21.15.123:3300/0 conn(0x7fdf3c0c3b50 msgr2=0x7fdf3c16b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7fdf39e00640 1 --2- 172.21.15.123:0/4058579570 >> v2:172.21.15.123:3300/0 conn(0x7fdf3c0c3b50 0x7fdf3c16b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7fdf39e00640 1 -- 172.21.15.123:0/4058579570 >> v1:172.21.15.123:6789/0 conn(0x7fdf3c0c3740 legacy=0x7fdf3c0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7fdf39e00640 1 -- 172.21.15.123:0/4058579570 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c073480 con 0x7fdf3c15aca0 2024-04-25T12:37:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7f3124a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312015c4b0 con 0x7f31200cb420 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7f3124a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31200cb140 con 0x7f31201568d0 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7f3124a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f312006d810 con 0x7f3120165e30 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7f311de00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f31201568d0 0x7f312015fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:55904/0 (socket says 172.21.15.123:55904) 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7f311de00640 1 -- 172.21.15.123:0/2874118498 learned_addr learned my addr 172.21.15.123:0/2874118498 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.921+0000 7f258d600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f258004df90 0x7f25800d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.921+0000 7f258d600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2580036780 0x7f2580036b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.921+0000 7f258d600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258004dcd0 con 0x7f25800dde50 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.921+0000 7f258d600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f25800e09b0 con 0x7f2580036780 2024-04-25T12:37:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.921+0000 7f258d600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f25800e1530 con 0x7f258004df90 2024-04-25T12:37:22.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7fa931200640 1 Processor -- start 2024-04-25T12:37:22.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7fdf39e00640 1 --2- 172.21.15.123:0/4058579570 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c161470 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c06e6e0 tx=0x7fdf3402a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33140728039fd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7ff1dbe00640 1 -- 172.21.15.123:0/3158263577 >> v2:172.21.15.153:3300/0 conn(0x7ff1dc0743b0 msgr2=0x7ff1dc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f2587e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f258004df90 0x7f25800d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f2587e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f258004df90 0x7f25800d7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48212/0 (socket says 172.21.15.123:48212) 2024-04-25T12:37:22.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7ff1dbe00640 1 --2- 172.21.15.123:0/3158263577 >> v2:172.21.15.153:3300/0 conn(0x7ff1dc0743b0 0x7ff1dc0c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7ff1dbe00640 1 -- 172.21.15.123:0/3158263577 >> v1:172.21.15.123:6789/0 conn(0x7ff1dc0ca130 legacy=0x7ff1dc0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.920+0000 7ff1dbe00640 1 -- 172.21.15.123:0/3158263577 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc0737d0 con 0x7ff1dc0c9710 2024-04-25T12:37:22.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7ff1e1200640 1 --2- 172.21.15.123:0/3158263577 >> v2:172.21.15.153:3300/0 conn(0x7ff1dc0743b0 0x7ff1dc0c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:22.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f2587e00640 1 -- 172.21.15.123:0/1583498948 learned_addr learned my addr 172.21.15.123:0/1583498948 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.921+0000 7f311f200640 1 --2- 172.21.15.123:0/2874118498 >> v2:172.21.15.123:3300/0 conn(0x7f3120165e30 0x7f3120168250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7ff1dbe00640 1 --2- 172.21.15.123:0/3158263577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0c9710 0x7ff1dc0c9b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00083f0 tx=0x7ff1d000abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7907f9be115351c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7fdf38a00640 1 -- 172.21.15.123:0/4058579570 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34036070 con 0x7fdf3c15aca0 2024-04-25T12:37:22.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976355382 0 0) 0x7f31200cb140 con 0x7f31201568d0 2024-04-25T12:37:22.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3104003520 con 0x7f31201568d0 2024-04-25T12:37:22.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493159758 0 0) 0x7f312015c4b0 con 0x7f31200cb420 2024-04-25T12:37:22.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31200cb140 con 0x7f31200cb420 2024-04-25T12:37:22.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7fdf38a00640 1 -- 172.21.15.123:0/4058579570 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf34031600 con 0x7fdf3c15aca0 2024-04-25T12:37:22.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7fdf38a00640 1 -- 172.21.15.123:0/4058579570 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf340319d0 con 0x7fdf3c15aca0 2024-04-25T12:37:22.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7f2587e00640 1 -- 172.21.15.123:0/1583498948 >> v2:172.21.15.123:3300/0 conn(0x7f2580036780 msgr2=0x7f2580036b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f258cc00640 1 --2- 172.21.15.123:0/1583498948 >> v2:172.21.15.123:3300/0 conn(0x7f2580036780 0x7f2580036b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f2587e00640 1 --2- 172.21.15.123:0/1583498948 >> v2:172.21.15.123:3300/0 conn(0x7f2580036780 0x7f2580036b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f2587e00640 1 -- 172.21.15.123:0/1583498948 >> v1:172.21.15.123:6789/0 conn(0x7f25800dde50 legacy=0x7f2580052570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f2587e00640 1 -- 172.21.15.123:0/1583498948 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25800e0c00 con 0x7f258004df90 2024-04-25T12:37:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2623685249 0 0) 0x7f3104003520 con 0x7f31201568d0 2024-04-25T12:37:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 >> v2:172.21.15.123:3300/0 conn(0x7f3120165e30 msgr2=0x7f3120168250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f311d400640 1 --2- 172.21.15.123:0/2874118498 >> v2:172.21.15.123:3300/0 conn(0x7f3120165e30 0x7f3120168250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 >> v1:172.21.15.123:6789/0 conn(0x7f31200cb420 legacy=0x7f31200cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312015c770 con 0x7f31201568d0 2024-04-25T12:37:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 msgr2=0x7fdf3c161470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7fdf40e00640 1 --2- 172.21.15.123:0/4058579570 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c161470 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c06e6e0 tx=0x7fdf3402a740 comp rx=0 tx=0).stop 2024-04-25T12:37:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f258cc00640 1 --2- 172.21.15.123:0/1583498948 >> v2:172.21.15.123:3300/0 conn(0x7f2580036780 0x7f2580036b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.923+0000 7ff1db400640 1 -- 172.21.15.123:0/3158263577 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00089e0 con 0x7ff1dc0c9710 2024-04-25T12:37:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7ff1db400640 1 -- 172.21.15.123:0/3158263577 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d0008b80 con 0x7ff1dc0c9710 2024-04-25T12:37:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7ff1db400640 1 -- 172.21.15.123:0/3158263577 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000a0d0 con 0x7ff1dc0c9710 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f310c001670 con 0x7f31201568d0 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 >> v1:172.21.15.153:6789/0 conn(0x7f31201568d0 legacy=0x7f312015fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3104004ba0 0x7f3104004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31040037a0 0x7f3104003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f310400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f312006d810 con 0x7f31040037a0 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f310400c110 con 0x7f3104004ba0 2024-04-25T12:37:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f310400c310 con 0x7f31040041c0 2024-04-25T12:37:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 shutdown_connections 2024-04-25T12:37:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7fdf40e00640 1 --2- 172.21.15.123:0/4058579570 >> v2:172.21.15.123:3300/0 conn(0x7fdf3c0c3b50 0x7fdf3c16b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7fdf40e00640 1 --2- 172.21.15.123:0/4058579570 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c161470 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 >> 172.21.15.123:0/4058579570 conn(0x7fdf3c06f440 msgr2=0x7fdf3c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.924+0000 7f311f200640 1 --2- 172.21.15.123:0/2874118498 >> v2:172.21.15.123:3300/0 conn(0x7f3120165e30 0x7f3120168250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.922+0000 7fa931200640 1 -- start start 2024-04-25T12:37:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7fa931200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 shutdown_connections 2024-04-25T12:37:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7fdf40e00640 1 -- 172.21.15.123:0/4058579570 wait complete. 2024-04-25T12:37:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7fa929e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7fa929e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c15a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56776/0 (socket says 172.21.15.123:56776) 2024-04-25T12:37:22.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7f311e800640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3104004ba0 0x7f3104004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7f311f200640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f310400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.925+0000 7f311de00640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31040037a0 0x7f3104003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7fa929400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa92c0c7e60 0x7fa92c073df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58538/0 (socket says 172.21.15.123:58538) 2024-04-25T12:37:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7f311f200640 1 -- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3104004ba0 msgr2=0x7f3104004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7f311f200640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3104004ba0 0x7f3104004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7f311f200640 1 -- 172.21.15.123:0/2874118498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31040037a0 msgr2=0x7f3104003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7fa929e00640 1 -- 172.21.15.123:0/2838873213 learned_addr learned my addr 172.21.15.123:0/2838873213 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7ff1e3c00640 1 -- 172.21.15.123:0/3158263577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0c9710 msgr2=0x7ff1dc0c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7ff1e3c00640 1 --2- 172.21.15.123:0/3158263577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0c9710 0x7ff1dc0c9b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00083f0 tx=0x7ff1d000abe0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7f311f200640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31040037a0 0x7f3104003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7f2587e00640 1 --2- 172.21.15.123:0/1583498948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258004df90 0x7f25800d7790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2588065b40 tx=0x7f25880a78c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483198703f58b106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7fa931200640 1 --2- 172.21.15.123:0/2838873213 >> v2:172.21.15.153:3300/0 conn(0x7fa92c160b30 0x7fa92c160f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7f2586a00640 1 -- 172.21.15.123:0/1583498948 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25880ad430 con 0x7f258004df90 2024-04-25T12:37:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7f311f200640 1 -- 172.21.15.123:0/2874118498 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3104003520 con 0x7f31040041c0 2024-04-25T12:37:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.926+0000 7f311e800640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3104004ba0 0x7f3104004fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7f2586a00640 1 -- 172.21.15.123:0/1583498948 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25880ad5f0 con 0x7f258004df90 2024-04-25T12:37:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7f2586a00640 1 -- 172.21.15.123:0/1583498948 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25880ada20 con 0x7f258004df90 2024-04-25T12:37:22.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa92c0c85a0 con 0x7fa92c0c7e60 2024-04-25T12:37:22.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7f311de00640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31040037a0 0x7f3104003bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:22.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7fa928a00640 1 -- 172.21.15.123:0/2838873213 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035941728 0 0) 0x7fa92c0c85a0 con 0x7fa92c0c7e60 2024-04-25T12:37:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa92c0c70a0 con 0x7fa92c160b30 2024-04-25T12:37:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c0c6560 con 0x7fa92c0c4960 2024-04-25T12:37:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7ff1e3c00640 1 -- 172.21.15.123:0/3158263577 shutdown_connections 2024-04-25T12:37:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7ff1e3c00640 1 --2- 172.21.15.123:0/3158263577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0c9710 0x7ff1dc0c9b20 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00083f0 tx=0x7ff1d000abe0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7ff1e3c00640 1 --2- 172.21.15.123:0/3158263577 >> v2:172.21.15.153:3300/0 conn(0x7ff1dc0743b0 0x7ff1dc0c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.927+0000 7ff1e3c00640 1 -- 172.21.15.123:0/3158263577 >> 172.21.15.123:0/3158263577 conn(0x7ff1dc06f440 msgr2=0x7ff1dc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7fa928a00640 1 -- 172.21.15.123:0/2838873213 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa910003520 con 0x7fa92c0c7e60 2024-04-25T12:37:22.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7f311f200640 1 -- 172.21.15.123:0/2874118498 reap_dead start 2024-04-25T12:37:22.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7fa929e00640 1 -- 172.21.15.123:0/2838873213 >> v2:172.21.15.153:3300/0 conn(0x7fa92c160b30 msgr2=0x7fa92c160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7fa929e00640 1 --2- 172.21.15.123:0/2838873213 >> v2:172.21.15.153:3300/0 conn(0x7fa92c160b30 0x7fa92c160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7fa929e00640 1 -- 172.21.15.123:0/2838873213 >> v1:172.21.15.123:6789/0 conn(0x7fa92c0c7e60 legacy=0x7fa92c073df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7f311f200640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f310400b9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f31200d0640 tx=0x7f3104005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c32f0ab581863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7fa929e00640 1 -- 172.21.15.123:0/2838873213 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c0c6800 con 0x7fa92c0c4960 2024-04-25T12:37:22.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7ff1e3c00640 1 -- 172.21.15.123:0/3158263577 shutdown_connections 2024-04-25T12:37:22.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.928+0000 7fdf40e00640 1 Processor -- start 2024-04-25T12:37:22.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7ff1e3c00640 1 -- 172.21.15.123:0/3158263577 wait complete. 2024-04-25T12:37:22.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f258d600640 1 -- 172.21.15.123:0/1583498948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258004df90 msgr2=0x7f25800d7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f258d600640 1 --2- 172.21.15.123:0/1583498948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258004df90 0x7f25800d7790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2588065b40 tx=0x7f25880a78c0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f258d600640 1 -- 172.21.15.123:0/1583498948 shutdown_connections 2024-04-25T12:37:22.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f258d600640 1 --2- 172.21.15.123:0/1583498948 >> v2:172.21.15.123:3300/0 conn(0x7f2580036780 0x7f2580036b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f258d600640 1 --2- 172.21.15.123:0/1583498948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258004df90 0x7f25800d7790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f258d600640 1 -- 172.21.15.123:0/1583498948 >> 172.21.15.123:0/1583498948 conn(0x7f258000e2e0 msgr2=0x7f258000e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110007a50 con 0x7f31040041c0 2024-04-25T12:37:22.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311802c630 con 0x7f31040041c0 2024-04-25T12:37:22.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7f311d400640 1 -- 172.21.15.123:0/2874118498 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118009c80 con 0x7f31040041c0 2024-04-25T12:37:22.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7fdf40e00640 1 -- start start 2024-04-25T12:37:22.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7fdcf6c00640 1 Processor -- start 2024-04-25T12:37:22.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.929+0000 7fdcf6c00640 1 -- start start 2024-04-25T12:37:22.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7f258d600640 1 -- 172.21.15.123:0/1583498948 shutdown_connections 2024-04-25T12:37:22.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7fa929e00640 1 --2- 172.21.15.123:0/2838873213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c15a8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009870 tx=0x7fa91c02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe57b3e5e66c8983 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7f3124a00640 1 -- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 msgr2=0x7f310400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7f3124a00640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f310400b9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f31200d0640 tx=0x7f3104005830 comp rx=0 tx=0).stop 2024-04-25T12:37:22.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7f258d600640 1 -- 172.21.15.123:0/1583498948 wait complete. 2024-04-25T12:37:22.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7f3124a00640 1 -- 172.21.15.123:0/2874118498 shutdown_connections 2024-04-25T12:37:22.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7f3124a00640 1 --2- 172.21.15.123:0/2874118498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f310400b9c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.930+0000 7f3124a00640 1 -- 172.21.15.123:0/2874118498 >> 172.21.15.123:0/2874118498 conn(0x7f3120068810 msgr2=0x7f3120068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.931+0000 7fa928a00640 1 -- 172.21.15.123:0/2838873213 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c037070 con 0x7fa92c0c4960 2024-04-25T12:37:22.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.931+0000 7fa928a00640 1 -- 172.21.15.123:0/2838873213 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c02ae50 con 0x7fa92c0c4960 2024-04-25T12:37:22.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.931+0000 7fdf40e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0c3740 0x7fdf3c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.931+0000 7fdf40e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf3c0c3b50 0x7fdf3c14ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.931+0000 7fa928a00640 1 -- 172.21.15.123:0/2838873213 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c037070 con 0x7fa92c0c4960 2024-04-25T12:37:22.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.931+0000 7fdcf6c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcf006cc80 0x7fdcf00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdcf6c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00743b0 0x7fdcf00ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdcf6c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf0071300 con 0x7fdcf00ca670 2024-04-25T12:37:22.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdcf6c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf0071880 con 0x7fdcf006cc80 2024-04-25T12:37:22.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdcf6c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00c8e40 con 0x7fdcf00743b0 2024-04-25T12:37:22.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7f3124a00640 1 -- 172.21.15.123:0/2874118498 shutdown_connections 2024-04-25T12:37:22.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7f3124a00640 1 -- 172.21.15.123:0/2874118498 wait complete. 2024-04-25T12:37:22.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 msgr2=0x7fa92c15a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdcefe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcf006cc80 0x7fdcf00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdcefe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcf006cc80 0x7fdcf00c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41014/0 (socket says 172.21.15.123:41014) 2024-04-25T12:37:22.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdcefe00640 1 -- 172.21.15.123:0/3634463852 learned_addr learned my addr 172.21.15.123:0/3634463852 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fa931200640 1 --2- 172.21.15.123:0/2838873213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c15a8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009870 tx=0x7fa91c02a300 comp rx=0 tx=0).stop 2024-04-25T12:37:22.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7ff1e3c00640 1 Processor -- start 2024-04-25T12:37:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.932+0000 7fdf40e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdf39e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0c3740 0x7fdf3c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdce7400640 1 --2- 172.21.15.123:0/3634463852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00743b0 0x7fdcf00ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7ff1e3c00640 1 -- start start 2024-04-25T12:37:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 shutdown_connections 2024-04-25T12:37:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fa931200640 1 --2- 172.21.15.123:0/2838873213 >> v2:172.21.15.153:3300/0 conn(0x7fa92c160b30 0x7fa92c160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fa931200640 1 --2- 172.21.15.123:0/2838873213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c15a8c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 >> 172.21.15.123:0/2838873213 conn(0x7fa92c0710e0 msgr2=0x7fa92c0cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdf40e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c0708f0 con 0x7fdf3c0c3740 2024-04-25T12:37:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdf40e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c071220 con 0x7fdf3c0c3b50 2024-04-25T12:37:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdf40e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3c070bd0 con 0x7fdf3c15aca0 2024-04-25T12:37:22.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdcef400640 1 -- 172.21.15.123:0/3634463852 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2953053141 0 0) 0x7fdcf0071300 con 0x7fdcf00ca670 2024-04-25T12:37:22.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdf39e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0c3740 0x7fdf3c14ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41004/0 (socket says 172.21.15.123:41004) 2024-04-25T12:37:22.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdf39e00640 1 -- 172.21.15.123:0/3434227578 learned_addr learned my addr 172.21.15.123:0/3434227578 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fcef2200640 1 Processor -- start 2024-04-25T12:37:22.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdf39400640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf3c0c3b50 0x7fdf3c14ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdf3a800640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.933+0000 7fdcefe00640 1 -- 172.21.15.123:0/3634463852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00743b0 msgr2=0x7fdcf00ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 shutdown_connections 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdf39e00640 1 -- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 msgr2=0x7fdf3c14b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdf39e00640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf3c15aca0 0x7fdf3c14b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdf39e00640 1 -- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf3c0c3b50 msgr2=0x7fdf3c14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdf39e00640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf3c0c3b50 0x7fdf3c14ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdf39e00640 1 -- 172.21.15.123:0/3434227578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c14d200 con 0x7fdf3c0c3740 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdcefe00640 1 --2- 172.21.15.123:0/3634463852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00743b0 0x7fdcf00ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdcefe00640 1 -- 172.21.15.123:0/3634463852 >> v1:172.21.15.153:6789/0 conn(0x7fdcf00ca670 legacy=0x7fdcf00c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdcefe00640 1 -- 172.21.15.123:0/3634463852 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00737d0 con 0x7fdcf006cc80 2024-04-25T12:37:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fdce7400640 1 --2- 172.21.15.123:0/3634463852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00743b0 0x7fdcf00ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fcef2200640 1 -- start start 2024-04-25T12:37:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.935+0000 7f258d600640 1 Processor -- start 2024-04-25T12:37:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.934+0000 7fa931200640 1 -- 172.21.15.123:0/2838873213 wait complete. 2024-04-25T12:37:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.935+0000 7fdf39e00640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0c3740 0x7fdf3c14ccc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009f40 tx=0x7fdf34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.935+0000 7f258d600640 1 -- start start 2024-04-25T12:37:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.935+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34041030 con 0x7fdf3c0c3740 2024-04-25T12:37:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1e3c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0743b0 0x7ff1dc157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1e3c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0ca130 0x7ff1dc151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1e3c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1dc156220 0x7ff1dc156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1e3c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc073a70 con 0x7ff1dc156220 2024-04-25T12:37:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1e3c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc0c91f0 con 0x7ff1dc0ca130 2024-04-25T12:37:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1e3c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc073e80 con 0x7ff1dc0743b0 2024-04-25T12:37:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7fdcefe00640 1 --2- 172.21.15.123:0/3634463852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf006cc80 0x7fdcf00c8900 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00739a0 tx=0x7fdce002e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7e7603d115ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf34037040 con 0x7fdf3c0c3740 2024-04-25T12:37:22.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34036070 con 0x7fdf3c0c3740 2024-04-25T12:37:22.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c14d410 con 0x7fdf3c0c3740 2024-04-25T12:37:22.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c155840 con 0x7fdf3c0c3740 2024-04-25T12:37:22.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1dbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0ca130 0x7ff1dc151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1dbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0ca130 0x7ff1dc151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48250/0 (socket says 172.21.15.123:48250) 2024-04-25T12:37:22.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7ff1dbe00640 1 -- 172.21.15.123:0/1355235180 learned_addr learned my addr 172.21.15.123:0/1355235180 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7f3124a00640 1 Processor -- start 2024-04-25T12:37:22.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.936+0000 7f3124a00640 1 -- start start 2024-04-25T12:37:22.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fb285000640 1 Processor -- start 2024-04-25T12:37:22.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fdcef400640 1 -- 172.21.15.123:0/3634463852 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00345a0 con 0x7fdcf006cc80 2024-04-25T12:37:22.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fdcef400640 1 -- 172.21.15.123:0/3634463852 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce003a070 con 0x7fdcf006cc80 2024-04-25T12:37:22.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fdcef400640 1 -- 172.21.15.123:0/3634463852 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0034960 con 0x7fdcf006cc80 2024-04-25T12:37:22.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fcef2200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c9f00 0x7fceec0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fcef2200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec074b20 con 0x7fceec154650 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fcef2200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec0ca780 con 0x7fceec0730e0 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fcef2200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fceec0caa60 con 0x7fceec0c9f00 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fcee2a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c9f00 0x7fceec0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fcee2a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c9f00 0x7fceec0c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56810/0 (socket says 172.21.15.123:56810) 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fcee2a00640 1 -- 172.21.15.123:0/4079601925 learned_addr learned my addr 172.21.15.123:0/4079601925 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3634463852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf006cc80 msgr2=0x7fdcf00c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7fdcf6c00640 1 --2- 172.21.15.123:0/3634463852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf006cc80 0x7fdcf00c8900 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00739a0 tx=0x7fdce002e8a0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7ff1e1200640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0743b0 0x7ff1dc157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7ff1e1c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1dc156220 0x7ff1dc156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7ff1e1200640 1 -- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0ca130 msgr2=0x7ff1dc151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7ff1e1200640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0ca130 0x7ff1dc151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7ff1e1200640 1 -- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1dc156220 msgr2=0x7ff1dc156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7ff1e1200640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1dc156220 0x7ff1dc156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7ff1e1200640 1 -- 172.21.15.123:0/1355235180 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc157b50 con 0x7ff1dc0743b0 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7f3124a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f3120154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7f3124a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3120155e80 0x7f312014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7f3124a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f312015ae50 0x7f31201495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7f3124a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f312015d010 con 0x7f3120155e80 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7f3124a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f312015cd30 con 0x7f312015ae50 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.938+0000 7f3124a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f31200c4500 con 0x7f31040041c0 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdf14001030 con 0x7fdf3c0c3740 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3634463852 shutdown_connections 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fdcf6c00640 1 --2- 172.21.15.123:0/3634463852 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00743b0 0x7fdcf00ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fdcf6c00640 1 --2- 172.21.15.123:0/3634463852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf006cc80 0x7fdcf00c8900 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919558874 0 0) 0x7fceec0ca780 con 0x7fceec0730e0 2024-04-25T12:37:22.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3634463852 >> 172.21.15.123:0/3634463852 conn(0x7fdcf006f440 msgr2=0x7fdcf006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fced0003520 con 0x7fceec0730e0 2024-04-25T12:37:22.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3634463852 shutdown_connections 2024-04-25T12:37:22.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3634463852 wait complete. 2024-04-25T12:37:22.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7ff1dbe00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0ca130 0x7ff1dc151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:22.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7f311de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3120155e80 0x7f312014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1585707713 0 0) 0x7fced0003520 con 0x7fceec0730e0 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c9f00 msgr2=0x7fceec0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c9f00 0x7fceec0c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 >> v1:172.21.15.123:6789/0 conn(0x7fceec154650 legacy=0x7fceec15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec15da50 con 0x7fceec0730e0 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7fcee4001670 con 0x7fceec0730e0 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 >> v1:172.21.15.153:6789/0 conn(0x7fceec0730e0 legacy=0x7fceec164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7fceeaa00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced00039b0 0x7fced0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.939+0000 7f311de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3120155e80 0x7f312014ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41032/0 (socket says 172.21.15.123:41032) 2024-04-25T12:37:22.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7f311de00640 1 -- 172.21.15.123:0/3523443208 learned_addr learned my addr 172.21.15.123:0/3523443208 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fceeaa00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fced0004800 0x7fced0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fceeaa00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fced0005150 0x7fced000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.937+0000 7fb285000640 1 -- start start 2024-04-25T12:37:22.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fceec0caa60 con 0x7fced00039b0 2024-04-25T12:37:22.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fced000c070 con 0x7fced0005150 2024-04-25T12:37:22.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fced000c270 con 0x7fced0004800 2024-04-25T12:37:22.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7f311e800640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f3120154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7f311f200640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f312015ae50 0x7f31201495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fceebe00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fced0005150 0x7fced000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fa931200640 1 Processor -- start 2024-04-25T12:37:22.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.940+0000 7fb285000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb2800748a0 0x7fb28015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb27f400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb2800c2720 0x7fb28006c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:55928/0 (socket says 172.21.15.123:55928) 2024-04-25T12:37:22.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb27f400640 1 -- 172.21.15.123:0/3813975140 learned_addr learned my addr 172.21.15.123:0/3813975140 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:22.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fcee2a00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fced0004800 0x7fced0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7ff1e1200640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0743b0 0x7ff1dc157610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d9c0 tx=0x7ff1c800de80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fceebe00640 1 -- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fced0004800 msgr2=0x7fced0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fceebe00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fced0004800 0x7fced0004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fceebe00640 1 -- 172.21.15.123:0/4079601925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced00039b0 msgr2=0x7fced0003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T12:37:22.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fceebe00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced00039b0 0x7fced0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fceebe00640 1 -- 172.21.15.123:0/4079601925 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec15da50 con 0x7fced0005150 2024-04-25T12:37:22.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb285000640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800c81a0 0x7fb2800c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb280161450 con 0x7fb2800c2720 2024-04-25T12:37:22.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7f311f200640 1 -- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 msgr2=0x7f3120154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7f311f200640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31040041c0 0x7f3120154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7f311f200640 1 -- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3120155e80 msgr2=0x7f312014ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb27ea00640 1 --2- 172.21.15.123:0/3813975140 >> v2:172.21.15.153:3300/0 conn(0x7fb2800748a0 0x7fb28015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb2801616f0 con 0x7fb2800748a0 2024-04-25T12:37:22.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb27fe00640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800c81a0 0x7fb2800c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7f311f200640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3120155e80 0x7f312014ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7f311f200640 1 -- 172.21.15.123:0/3523443208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200c9220 con 0x7f312015ae50 2024-04-25T12:37:22.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7f311de00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3120155e80 0x7f312014ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:22.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb27e000640 1 -- 172.21.15.123:0/3813975140 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324875394 0 0) 0x7fb280161450 con 0x7fb2800c2720 2024-04-25T12:37:22.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb2800c4030 con 0x7fb2800c81a0 2024-04-25T12:37:22.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fb27e000640 1 -- 172.21.15.123:0/3813975140 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb264003520 con 0x7fb2800c2720 2024-04-25T12:37:22.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7f311f200640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f312015ae50 0x7f31201495c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3120070680 tx=0x7f3118004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fb27ea00640 1 -- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800c81a0 msgr2=0x7fb2800c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:22.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fb27ea00640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800c81a0 0x7fb2800c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fb27e000640 1 -- 172.21.15.123:0/3813975140 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1649276485 0 0) 0x7fb264003520 con 0x7fb2800c2720 2024-04-25T12:37:22.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fb27ea00640 1 -- 172.21.15.123:0/3813975140 >> v1:172.21.15.153:6789/0 conn(0x7fb2800c2720 legacy=0x7fb28006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fb27ea00640 1 -- 172.21.15.123:0/3813975140 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2800c4310 con 0x7fb2800748a0 2024-04-25T12:37:22.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fdcf6c00640 1 Processor -- start 2024-04-25T12:37:22.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fb27fe00640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800c81a0 0x7fb2800c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:22.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf34044070 con 0x7fdf3c0c3740 2024-04-25T12:37:22.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118045070 con 0x7f312015ae50 2024-04-25T12:37:22.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311802d950 con 0x7f312015ae50 2024-04-25T12:37:22.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118009c80 con 0x7f312015ae50 2024-04-25T12:37:22.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.942+0000 7fdf32000640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf1c074500 0x7fdf1c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fdcf6c00640 1 -- start start 2024-04-25T12:37:22.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fdf340b5ff0 con 0x7fdf3c0c3740 2024-04-25T12:37:22.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fceebe00640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fced0005150 0x7fced000b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c190 tx=0x7fced0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba1097ea468bb555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31201544d0 con 0x7f312015ae50 2024-04-25T12:37:22.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31200c9780 con 0x7f312015ae50 2024-04-25T12:37:22.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fdf39400640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf1c074500 0x7fdf1c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fdf32000640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdf1c07e260 0x7fdf1c080750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf1c080e20 con 0x7fdf1c07e260 2024-04-25T12:37:22.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fdf340bd050 con 0x7fdf3c0c3740 2024-04-25T12:37:22.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4036070 con 0x7fced0005150 2024-04-25T12:37:22.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fdf3a800640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdf1c07e260 0x7fdf1c080750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee4007270 con 0x7fced0005150 2024-04-25T12:37:22.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fceeaa00640 1 -- 172.21.15.123:0/4079601925 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4007c30 con 0x7fced0005150 2024-04-25T12:37:22.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800ab80 con 0x7ff1dc0743b0 2024-04-25T12:37:22.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7fdf39400640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf1c074500 0x7fdf1c076a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c1fb660 tx=0x7fdf2c0090e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c800ad20 con 0x7ff1dc0743b0 2024-04-25T12:37:22.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8010670 con 0x7ff1dc0743b0 2024-04-25T12:37:22.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7fcef2200640 1 -- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fced0005150 msgr2=0x7fced000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7fcef2200640 1 --2- 172.21.15.123:0/4079601925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fced0005150 0x7fced000b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c190 tx=0x7fced0005830 comp rx=0 tx=0).stop 2024-04-25T12:37:22.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc1f75a0 con 0x7ff1dc0743b0 2024-04-25T12:37:22.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7fceebe00640 1 -- 172.21.15.123:0/4079601925 reap_dead start 2024-04-25T12:37:22.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2580036780 0x7f25800cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258d600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f258004df90 0x7f25800515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258d600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25800dde50 0x7f25800c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fcef2200640 1 -- 172.21.15.123:0/4079601925 shutdown_connections 2024-04-25T12:37:22.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fcef2200640 1 -- 172.21.15.123:0/4079601925 >> 172.21.15.123:0/4079601925 conn(0x7fceec06f440 msgr2=0x7fceec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fb27ea00640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800748a0 0x7fb28015de60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0097a0 tx=0x7fb26c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f03dd92b181096e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf6c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf006cc80 0x7fdcf00c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf6c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcf00743b0 0x7fdcf01fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf6c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00ca670 0x7fdcf00cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf6c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00715a0 con 0x7fdcf006cc80 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf6c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0073a70 con 0x7fdcf00743b0 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf6c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00c91f0 con 0x7fdcf00ca670 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc1f7b60 con 0x7ff1dc0743b0 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf4c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00ca670 0x7fdcf00cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf4c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00ca670 0x7fdcf00cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56852/0 (socket says 172.21.15.123:56852) 2024-04-25T12:37:22.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fdcf4c00640 1 -- 172.21.15.123:0/3355618705 learned_addr learned my addr 172.21.15.123:0/3355618705 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fb27e000640 1 -- 172.21.15.123:0/3813975140 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c037070 con 0x7fb2800748a0 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2580047590 con 0x7f2580036780 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258d600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f25800e11c0 con 0x7f25800dde50 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258d600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f25800de260 con 0x7f258004df90 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f2587e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2580036780 0x7f25800cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25800dde50 0x7f25800c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25800dde50 0x7f25800c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48300/0 (socket says 172.21.15.123:48300) 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f258cc00640 1 -- 172.21.15.123:0/5228761 learned_addr learned my addr 172.21.15.123:0/5228761 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7f2587400640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f258004df90 0x7f25800515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f258cc00640 1 -- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f258004df90 msgr2=0x7f25800515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f258cc00640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f258004df90 0x7f25800515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f258cc00640 1 -- 172.21.15.123:0/5228761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2580036780 msgr2=0x7f25800cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f258cc00640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2580036780 0x7f25800cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f258cc00640 1 -- 172.21.15.123:0/5228761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25800cbc90 con 0x7f25800dde50 2024-04-25T12:37:22.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f2587400640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f258004df90 0x7f25800515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:22.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f258cc00640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25800dde50 0x7f25800c5e40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f257400c5e0 tx=0x7f257400caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fdcf4c00640 1 -- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcf00743b0 msgr2=0x7fdcf01fb0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T12:37:22.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fdcf4c00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcf00743b0 0x7fdcf01fb0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fdcf4c00640 1 -- 172.21.15.123:0/3355618705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf006cc80 msgr2=0x7fdcf00c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fdcf4c00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf006cc80 0x7fdcf00c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fdcf4c00640 1 -- 172.21.15.123:0/3355618705 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf015b6a0 con 0x7fdcf00ca670 2024-04-25T12:37:22.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fb27e000640 1 -- 172.21.15.123:0/3813975140 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26c02aba0 con 0x7fb2800748a0 2024-04-25T12:37:22.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fb27e000640 1 -- 172.21.15.123:0/3813975140 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c037070 con 0x7fb2800748a0 2024-04-25T12:37:22.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fdf3a800640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdf1c07e260 0x7fdf1c080750 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1b8001030 con 0x7ff1dc0743b0 2024-04-25T12:37:22.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fcef2200640 1 -- 172.21.15.123:0/4079601925 shutdown_connections 2024-04-25T12:37:22.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574015070 con 0x7f25800dde50 2024-04-25T12:37:22.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574004240 con 0x7f25800dde50 2024-04-25T12:37:22.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258017a660 con 0x7f25800dde50 2024-04-25T12:37:22.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.941+0000 7fa931200640 1 -- start start 2024-04-25T12:37:22.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fa931200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fa931200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92c0c7e60 0x7fa92c1fa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fa931200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa92c160b30 0x7fa92c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.943+0000 7fa931200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c161480 con 0x7fa92c160b30 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.946+0000 7fa931200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c0c6dc0 con 0x7fa92c0c7e60 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fa931200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c0c8270 con 0x7fa92c0c4960 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.944+0000 7fa929400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92c0c7e60 0x7fa92c1fa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.945+0000 7fa92a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa92c160b30 0x7fa92c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fa929400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92c0c7e60 0x7fa92c1fa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48290/0 (socket says 172.21.15.123:48290) 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fa929400640 1 -- 172.21.15.123:0/1053291212 learned_addr learned my addr 172.21.15.123:0/1053291212 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258017ac20 con 0x7f25800dde50 2024-04-25T12:37:22.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574003610 con 0x7f25800dde50 2024-04-25T12:37:22.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800748a0 msgr2=0x7fb28015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fb285000640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800748a0 0x7fb28015de60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0097a0 tx=0x7fb26c02a110 comp rx=0 tx=0).stop 2024-04-25T12:37:22.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fdcf4c00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00ca670 0x7fdcf00cc400 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c410 tx=0x7fdce800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf1c080e20 con 0x7fdf1c07e260 2024-04-25T12:37:22.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8017070 con 0x7fdcf00ca670 2024-04-25T12:37:22.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fa929e00640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce80035f0 con 0x7fdcf00ca670 2024-04-25T12:37:22.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80039c0 con 0x7fdcf00ca670 2024-04-25T12:37:22.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.948+0000 7fa929400640 1 -- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 msgr2=0x7fa92c160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.947+0000 7fcef2200640 1 -- 172.21.15.123:0/4079601925 wait complete. 2024-04-25T12:37:22.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.948+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3120076da0 con 0x7f312015ae50 2024-04-25T12:37:22.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.948+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f311803e070 con 0x7f312015ae50 2024-04-25T12:37:22.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.948+0000 7fa929400640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa92c0c4960 0x7fa92c160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.948+0000 7fa929400640 1 -- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa92c160b30 msgr2=0x7fa92c0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:22.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 shutdown_connections 2024-04-25T12:37:22.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fb285000640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800c81a0 0x7fb2800c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fb285000640 1 --2- 172.21.15.123:0/3813975140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800748a0 0x7fb28015de60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 >> 172.21.15.123:0/3813975140 conn(0x7fb280070b90 msgr2=0x7fb280070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fa929400640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa92c160b30 0x7fa92c0ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 shutdown_connections 2024-04-25T12:37:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2564001030 con 0x7f25800dde50 2024-04-25T12:37:22.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fa929400640 1 -- 172.21.15.123:0/1053291212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c1fab60 con 0x7fa92c0c7e60 2024-04-25T12:37:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fa92a800640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa92c160b30 0x7fa92c0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf00cc940 con 0x7fdcf00ca670 2024-04-25T12:37:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf00c5670 con 0x7fdcf00ca670 2024-04-25T12:37:22.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fb285000640 1 -- 172.21.15.123:0/3813975140 wait complete. 2024-04-25T12:37:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7f3116a00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3100074500 0x7f3100076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f31180b5b20 con 0x7f312015ae50 2024-04-25T12:37:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.949+0000 7fa929400640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92c0c7e60 0x7fa92c1fa620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa91800c3f0 tx=0x7fa91800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7f3116a00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f310007e260 0x7f3100080750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3118034e80 con 0x7f310007e260 2024-04-25T12:37:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f31180ba2c0 con 0x7f312015ae50 2024-04-25T12:37:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff1c8019400 con 0x7ff1dc0743b0 2024-04-25T12:37:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcf0076700 con 0x7fdcf00ca670 2024-04-25T12:37:22.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.951+0000 7f311de00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f310007e260 0x7f3100080750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.951+0000 7fcef2200640 1 Processor -- start 2024-04-25T12:37:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.950+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180074e0 con 0x7fa92c0c7e60 2024-04-25T12:37:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.951+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa918007680 con 0x7fa92c0c7e60 2024-04-25T12:37:22.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.951+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918004a20 con 0x7fa92c0c7e60 2024-04-25T12:37:23.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.951+0000 7fcef2200640 1 -- start start 2024-04-25T12:37:23.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.951+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c0c8c10 con 0x7fa92c0c7e60 2024-04-25T12:37:23.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.952+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c161cd0 con 0x7fa92c0c7e60 2024-04-25T12:37:23.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.952+0000 7f311de00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f310007e260 0x7f3100080750 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.952+0000 7ff1d8c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff1bc074580 0x7ff1bc076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.952+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7ff1c8096170 con 0x7ff1dc0743b0 2024-04-25T12:37:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.952+0000 7f311e800640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3100074500 0x7f3100076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.953+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f25740037b0 con 0x7f25800dde50 2024-04-25T12:37:23.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.953+0000 7ff1d8c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff1bc07e2e0 0x7ff1bc0807d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.953+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa92c076da0 con 0x7fa92c0c7e60 2024-04-25T12:37:23.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.953+0000 7fb285000640 1 Processor -- start 2024-04-25T12:37:23.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.953+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1c8025ec0 con 0x7ff1bc07e2e0 2024-04-25T12:37:23.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.953+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7ff1c8096560 con 0x7ff1dc0743b0 2024-04-25T12:37:23.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7f257be00640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2568074580 0x7f2568076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3118034e80 con 0x7f310007e260 2024-04-25T12:37:23.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7f311e800640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3100074500 0x7f3100076a10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3110005160 tx=0x7f3110009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7ff1e1c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff1bc07e2e0 0x7ff1bc0807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7ff1dbe00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff1bc074580 0x7ff1bc076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7fb285000640 1 -- start start 2024-04-25T12:37:23.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdce801a070 con 0x7fdcf00ca670 2024-04-25T12:37:23.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f2574095c30 con 0x7f25800dde50 2024-04-25T12:37:23.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7f257be00640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f256807e2e0 0x7f25680807d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f257400bfa0 con 0x7f256807e2e0 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f2574096020 con 0x7f25800dde50 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7fcef2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0005150 0x7fceec14e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7fcef2200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceec153c50 0x7fceec151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.954+0000 7fcef2200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c5de0 0x7fceec0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fcef2200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fceec15e2f0 con 0x7fced0005150 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fcef2200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fceec15e010 con 0x7fceec153c50 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fcef2200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fceec154a60 con 0x7fceec0c5de0 2024-04-25T12:37:23.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fcee2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceec153c50 0x7fceec151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7f2587e00640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2568074580 0x7f2568076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fceebe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c5de0 0x7fceec0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fceebe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c5de0 0x7fceec0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56862/0 (socket says 172.21.15.123:56862) 2024-04-25T12:37:23.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fceebe00640 1 -- 172.21.15.123:0/1407555778 learned_addr learned my addr 172.21.15.123:0/1407555778 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7f2587400640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f256807e2e0 0x7f25680807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.955+0000 7fceeb400640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0005150 0x7fceec14e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7ff1e1c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff1bc07e2e0 0x7ff1bc0807d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7f2587e00640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2568074580 0x7f2568076a90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f25800e0b30 tx=0x7f2588070030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fdcecc00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdccc074500 0x7fdccc076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa91801a020 con 0x7fa92c0c7e60 2024-04-25T12:37:23.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fdce8094e20 con 0x7fdcf00ca670 2024-04-25T12:37:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fceebe00640 1 -- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceec153c50 msgr2=0x7fceec151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fceebe00640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceec153c50 0x7fceec151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fceebe00640 1 -- 172.21.15.123:0/1407555778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0005150 msgr2=0x7fceec14e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fceebe00640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0005150 0x7fceec14e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fceebe00640 1 -- 172.21.15.123:0/1407555778 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec1f6aa0 con 0x7fceec0c5de0 2024-04-25T12:37:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fdcecc00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fdccc07e260 0x7fdccc080750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdccc080e20 con 0x7fdccc07e260 2024-04-25T12:37:23.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.956+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fdce8095300 con 0x7fdcf00ca670 2024-04-25T12:37:23.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7fceebe00640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c5de0 0x7fceec0cc7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c190 tx=0x7fcee4001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7ff1dbe00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff1bc074580 0x7ff1bc076a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc1f1970 tx=0x7ff1d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7fb285000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800748a0 0x7fb2801f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7fb285000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800c2720 0x7fb28015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7fdcefe00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdccc074500 0x7fdccc076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7fdce7400640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fdccc07e260 0x7fdccc080750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fdcefe00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdccc074500 0x7fdccc076a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdce0039000 tx=0x7fdce0033000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7f2587400640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f256807e2e0 0x7f25680807d0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1c8025ec0 con 0x7ff1bc07e2e0 2024-04-25T12:37:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4036040 con 0x7fceec0c5de0 2024-04-25T12:37:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee4007420 con 0x7fceec0c5de0 2024-04-25T12:37:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4005070 con 0x7fceec0c5de0 2024-04-25T12:37:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec15cf90 con 0x7fceec0c5de0 2024-04-25T12:37:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f257400bfa0 con 0x7f256807e2e0 2024-04-25T12:37:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.957+0000 7fb285000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2800c81a0 0x7fb280162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb285000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb280073b40 con 0x7fb2800c81a0 2024-04-25T12:37:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb285000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb28006cec0 con 0x7fb2800c2720 2024-04-25T12:37:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb285000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb280162fc0 con 0x7fb2800748a0 2024-04-25T12:37:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fa922000640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa90c074580 0x7fa90c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fa918094d80 con 0x7fa92c0c7e60 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fa922000640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa90c07e2e0 0x7fa90c0807d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa90c080ea0 con 0x7fa90c07e2e0 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fa9180973e0 con 0x7fa92c0c7e60 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.959+0000 7fa929e00640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa90c074580 0x7fa90c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb27fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2800c81a0 0x7fb280162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb27fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2800c81a0 0x7fb280162160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41086/0 (socket says 172.21.15.123:41086) 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb27fe00640 1 -- 172.21.15.123:0/799326796 learned_addr learned my addr 172.21.15.123:0/799326796 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb27ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800c2720 0x7fb28015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec15d550 con 0x7fceec0c5de0 2024-04-25T12:37:23.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.959+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fceec152630 con 0x7fceec0c5de0 2024-04-25T12:37:23.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.959+0000 7fb27fe00640 1 -- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800748a0 msgr2=0x7fb2801f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.958+0000 7fb27f400640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800748a0 0x7fb2801f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.959+0000 7fdce7400640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fdccc07e260 0x7fdccc080750 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.959+0000 7fa92a800640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa90c07e2e0 0x7fa90c0807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.960+0000 7fa929e00640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa90c074580 0x7fa90c076a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa91c034000 tx=0x7fa91c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdccc080e20 con 0x7fdccc07e260 2024-04-25T12:37:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fb27fe00640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800748a0 0x7fb2801f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fb27fe00640 1 -- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800c2720 msgr2=0x7fb28015adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fb27fe00640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800c2720 0x7fb28015adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fb27fe00640 1 -- 172.21.15.123:0/799326796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2801f64e0 con 0x7fb2800c81a0 2024-04-25T12:37:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fb27ea00640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2800c2720 0x7fb28015adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fb27f400640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb2800748a0 0x7fb2801f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fa92a800640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa90c07e2e0 0x7fa90c0807d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.961+0000 7fb27fe00640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2800c81a0 0x7fb280162160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d180 tx=0x7fb27800d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.962+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27801a070 con 0x7fb2800c81a0 2024-04-25T12:37:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.962+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2801fa4d0 con 0x7fb2800c81a0 2024-04-25T12:37:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.962+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa90c080ea0 con 0x7fa90c07e2e0 2024-04-25T12:37:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.962+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb278015600 con 0x7fb2800c81a0 2024-04-25T12:37:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.962+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2801f6a60 con 0x7fb2800c81a0 2024-04-25T12:37:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.963+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2780159f0 con 0x7fb2800c81a0 2024-04-25T12:37:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.962+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcee4045020 con 0x7fceec0c5de0 2024-04-25T12:37:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.963+0000 7fcee3e00640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcec8074470 0x7fcec8076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.966+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fcee40b5820 con 0x7fceec0c5de0 2024-04-25T12:37:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.966+0000 7fcee3e00640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcec807e1d0 0x7fcec80806c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.966+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcee4032e50 con 0x7fcec807e1d0 2024-04-25T12:37:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.967+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fcee40b5bd0 con 0x7fceec0c5de0 2024-04-25T12:37:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.967+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb258001030 con 0x7fb2800c81a0 2024-04-25T12:37:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.967+0000 7fcee2a00640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcec807e1d0 0x7fcec80806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.967+0000 7fceeb400640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcec8074470 0x7fcec8076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.967+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb27801d070 con 0x7fb2800c81a0 2024-04-25T12:37:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.968+0000 7fceeb400640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcec8074470 0x7fcec8076980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0012e0 tx=0x7fcedc001670 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.968+0000 7fcee2a00640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcec807e1d0 0x7fcec80806c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.969+0000 7fb277400640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb2600744b0 0x7fb2600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.969+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fb27809a060 con 0x7fb2800c81a0 2024-04-25T12:37:23.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.969+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcee4032e50 con 0x7fcec807e1d0 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.969+0000 7fb277400640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb26007e190 0x7fb260080680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.969+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb278019ec0 con 0x7fb26007e190 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.969+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fb2780958c0 con 0x7fb2800c81a0 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.970+0000 7fb27f400640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb2600744b0 0x7fb2600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.970+0000 7fb27ea00640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb26007e190 0x7fb260080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.971+0000 7fb27ea00640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb26007e190 0x7fb260080680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.971+0000 7fb27f400640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb2600744b0 0x7fb2600769c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb270000f60 tx=0x7fb270009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.972+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb278019ec0 con 0x7fb26007e190 2024-04-25T12:37:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.987+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3120155970 con 0x7f310007e260 2024-04-25T12:37:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.987+0000 7f3116a00640 1 -- 172.21.15.123:0/3523443208 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3120155970 con 0x7f310007e260 2024-04-25T12:37:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.989+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f310007e260 msgr2=0x7f3100080750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.989+0000 7f3124a00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7f310007e260 0x7f3100080750 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.989+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1b8002dd0 con 0x7ff1bc07e2e0 2024-04-25T12:37:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.990+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3100074500 msgr2=0x7f3100076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.990+0000 7ff1d8c00640 1 -- 172.21.15.123:0/1355235180 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff1b8002dd0 con 0x7ff1bc07e2e0 2024-04-25T12:37:23.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.990+0000 7f3124a00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3100074500 0x7f3100076a10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3110005160 tx=0x7f3110009040 comp rx=0 tx=0).stop 2024-04-25T12:37:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.990+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f312015ae50 msgr2=0x7f31201495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.990+0000 7f3124a00640 1 --2- 172.21.15.123:0/3523443208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f312015ae50 0x7f31201495c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3120070680 tx=0x7f3118004ab0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.990+0000 7f311f200640 1 -- 172.21.15.123:0/3523443208 reap_dead start 2024-04-25T12:37:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.991+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 shutdown_connections 2024-04-25T12:37:23.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.991+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 >> 172.21.15.123:0/3523443208 conn(0x7f3120068810 msgr2=0x7f3120075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.991+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff1bc07e2e0 msgr2=0x7ff1bc0807d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.991+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 shutdown_connections 2024-04-25T12:37:23.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.991+0000 7ff1e3c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7ff1bc07e2e0 0x7ff1bc0807d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.992+0000 7f3124a00640 1 -- 172.21.15.123:0/3523443208 wait complete. 2024-04-25T12:37:23.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.992+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff1bc074580 msgr2=0x7ff1bc076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.992+0000 7ff1e3c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff1bc074580 0x7ff1bc076a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc1f1970 tx=0x7ff1d002e000 comp rx=0 tx=0).stop 2024-04-25T12:37:23.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.992+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0743b0 msgr2=0x7ff1dc157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.992+0000 7ff1e3c00640 1 --2- 172.21.15.123:0/1355235180 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1dc0743b0 0x7ff1dc157610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d9c0 tx=0x7ff1c800de80 comp rx=0 tx=0).stop 2024-04-25T12:37:23.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.992+0000 7ff1e1c00640 1 -- 172.21.15.123:0/1355235180 reap_dead start 2024-04-25T12:37:23.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.992+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf14002dd0 con 0x7fdf1c07e260 2024-04-25T12:37:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.993+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 shutdown_connections 2024-04-25T12:37:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.993+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 >> 172.21.15.123:0/1355235180 conn(0x7ff1dc06f440 msgr2=0x7ff1dc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.993+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 shutdown_connections 2024-04-25T12:37:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.993+0000 7ff1e3c00640 1 -- 172.21.15.123:0/1355235180 wait complete. 2024-04-25T12:37:23.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.993+0000 7fdf32000640 1 -- 172.21.15.123:0/3434227578 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdf14002dd0 con 0x7fdf1c07e260 2024-04-25T12:37:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.994+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdf1c07e260 msgr2=0x7fdf1c080750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.994+0000 7fdf40e00640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7fdf1c07e260 0x7fdf1c080750 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.995+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf1c074500 msgr2=0x7fdf1c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.995+0000 7fdf40e00640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdf1c074500 0x7fdf1c076a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c1fb660 tx=0x7fdf2c0090e0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.995+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0c3740 msgr2=0x7fdf3c14ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.995+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdcf00c5140 con 0x7fdccc07e260 2024-04-25T12:37:23.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.995+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa92c1556c0 con 0x7fa90c07e2e0 2024-04-25T12:37:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.995+0000 7fdf40e00640 1 --2- 172.21.15.123:0/3434227578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0c3740 0x7fdf3c14ccc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009f40 tx=0x7fdf34004a40 comp rx=0 tx=0).stop 2024-04-25T12:37:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.996+0000 7fdf3a800640 1 -- 172.21.15.123:0/3434227578 reap_dead start 2024-04-25T12:37:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.996+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 shutdown_connections 2024-04-25T12:37:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.996+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 >> 172.21.15.123:0/3434227578 conn(0x7fdf3c06f440 msgr2=0x7fdf3c1596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.996+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2564002dd0 con 0x7f256807e2e0 2024-04-25T12:37:23.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.996+0000 7fa922000640 1 -- 172.21.15.123:0/1053291212 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa92c1556c0 con 0x7fa90c07e2e0 2024-04-25T12:37:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.996+0000 7fdcecc00640 1 -- 172.21.15.123:0/3355618705 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdcf00c5140 con 0x7fdccc07e260 2024-04-25T12:37:23.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7f257be00640 1 -- 172.21.15.123:0/5228761 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2564002dd0 con 0x7f256807e2e0 2024-04-25T12:37:23.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 shutdown_connections 2024-04-25T12:37:23.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa90c07e2e0 msgr2=0x7fa90c0807d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fa931200640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fa90c07e2e0 0x7fa90c0807d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fdccc07e260 msgr2=0x7fdccc080750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdcf6c00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7fdccc07e260 0x7fdccc080750 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdf40e00640 1 -- 172.21.15.123:0/3434227578 wait complete. 2024-04-25T12:37:23.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdccc074500 msgr2=0x7fdccc076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdcf6c00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fdccc074500 0x7fdccc076a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdce0039000 tx=0x7fdce0033000 comp rx=0 tx=0).stop 2024-04-25T12:37:23.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00ca670 msgr2=0x7fdcf00cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdcf6c00640 1 --2- 172.21.15.123:0/3355618705 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdcf00ca670 0x7fdcf00cc400 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c410 tx=0x7fdce800c8d0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.997+0000 7fdcf4c00640 1 -- 172.21.15.123:0/3355618705 reap_dead start 2024-04-25T12:37:23.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa90c074580 msgr2=0x7fa90c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fa931200640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fa90c074580 0x7fa90c076a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa91c034000 tx=0x7fa91c02e000 comp rx=0 tx=0).stop 2024-04-25T12:37:23.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92c0c7e60 msgr2=0x7fa92c1fa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fa931200640 1 --2- 172.21.15.123:0/1053291212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa92c0c7e60 0x7fa92c1fa620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa91800c3f0 tx=0x7fa91800c8b0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 shutdown_connections 2024-04-25T12:37:23.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 >> 172.21.15.123:0/3355618705 conn(0x7fdcf006f440 msgr2=0x7fdcf0151370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fa92a800640 1 -- 172.21.15.123:0/1053291212 reap_dead start 2024-04-25T12:37:23.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 shutdown_connections 2024-04-25T12:37:23.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 >> 172.21.15.123:0/1053291212 conn(0x7fa92c0710e0 msgr2=0x7fa92c159140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 shutdown_connections 2024-04-25T12:37:23.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fdcf6c00640 1 -- 172.21.15.123:0/3355618705 wait complete. 2024-04-25T12:37:23.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f256807e2e0 msgr2=0x7f25680807d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7f258d600640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f256807e2e0 0x7f25680807d0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.998+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fceec1523e0 con 0x7fcec807e1d0 2024-04-25T12:37:23.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2568074580 msgr2=0x7f2568076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2568074580 0x7f2568076a90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f25800e0b30 tx=0x7f2588070030 comp rx=0 tx=0).stop 2024-04-25T12:37:23.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 shutdown_connections 2024-04-25T12:37:23.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25800dde50 msgr2=0x7f25800c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 --2- 172.21.15.123:0/5228761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25800dde50 0x7f25800c5e40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f257400c5e0 tx=0x7f257400caa0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7fa931200640 1 -- 172.21.15.123:0/1053291212 wait complete. 2024-04-25T12:37:23.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258cc00640 1 -- 172.21.15.123:0/5228761 reap_dead start 2024-04-25T12:37:23.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 shutdown_connections 2024-04-25T12:37:23.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 >> 172.21.15.123:0/5228761 conn(0x7f258000e2e0 msgr2=0x7f25800479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 shutdown_connections 2024-04-25T12:37:23.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7fcee3e00640 1 -- 172.21.15.123:0/1407555778 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fceec1523e0 con 0x7fcec807e1d0 2024-04-25T12:37:23.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:22.999+0000 7f258d600640 1 -- 172.21.15.123:0/5228761 wait complete. 2024-04-25T12:37:23.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.000+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcec807e1d0 msgr2=0x7fcec80806c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.000+0000 7fcef2200640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7fcec807e1d0 0x7fcec80806c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.001+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcec8074470 msgr2=0x7fcec8076980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.001+0000 7fcef2200640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcec8074470 0x7fcec8076980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0012e0 tx=0x7fcedc001670 comp rx=0 tx=0).stop 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.001+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c5de0 msgr2=0x7fceec0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.001+0000 7fcef2200640 1 --2- 172.21.15.123:0/1407555778 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fceec0c5de0 0x7fceec0cc7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c190 tx=0x7fcee4001970 comp rx=0 tx=0).stop 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.001+0000 7fceebe00640 1 -- 172.21.15.123:0/1407555778 reap_dead start 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.002+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 shutdown_connections 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.002+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 >> 172.21.15.123:0/1407555778 conn(0x7fceec06f440 msgr2=0x7fceec0c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.002+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 shutdown_connections 2024-04-25T12:37:23.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.002+0000 7fcef2200640 1 -- 172.21.15.123:0/1407555778 wait complete. 2024-04-25T12:37:23.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.004+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb258002dd0 con 0x7fb26007e190 2024-04-25T12:37:23.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.005+0000 7fb277400640 1 -- 172.21.15.123:0/799326796 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb258002dd0 con 0x7fb26007e190 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.007+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb26007e190 msgr2=0x7fb260080680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.007+0000 7fb285000640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7fb26007e190 0x7fb260080680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.007+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb2600744b0 msgr2=0x7fb2600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.007+0000 7fb285000640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb2600744b0 0x7fb2600769c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb270000f60 tx=0x7fb270009040 comp rx=0 tx=0).stop 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.007+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2800c81a0 msgr2=0x7fb280162160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.007+0000 7fb285000640 1 --2- 172.21.15.123:0/799326796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2800c81a0 0x7fb280162160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d180 tx=0x7fb27800d640 comp rx=0 tx=0).stop 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.007+0000 7fb27fe00640 1 -- 172.21.15.123:0/799326796 reap_dead start 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.008+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 shutdown_connections 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.008+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 >> 172.21.15.123:0/799326796 conn(0x7fb280070b90 msgr2=0x7fb28006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.009+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 shutdown_connections 2024-04-25T12:37:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.009+0000 7fb285000640 1 -- 172.21.15.123:0/799326796 wait complete. 2024-04-25T12:37:23.045 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:23.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T12:37:23.046 INFO:teuthology.orchestra.run.smithi123.stdout:64424509483 2024-04-25T12:37:23.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T12:37:23.047 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:23.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T12:37:23.048 INFO:teuthology.orchestra.run.smithi123.stdout:201863462930 2024-04-25T12:37:23.049 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T12:37:23.049 INFO:teuthology.orchestra.run.smithi123.stdout:64424509483 2024-04-25T12:37:23.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T12:37:23.050 INFO:teuthology.orchestra.run.smithi123.stdout:60129542186 2024-04-25T12:37:23.051 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T12:37:23.051 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:23.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T12:37:23.053 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:23.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T12:37:23.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.491+0000 7f6731000640 1 Processor -- start 2024-04-25T12:37:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.491+0000 7f8ded000640 1 Processor -- start 2024-04-25T12:37:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.491+0000 7f8ded000640 1 -- start start 2024-04-25T12:37:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.492+0000 7f1a7f200640 1 Processor -- start 2024-04-25T12:37:23.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.491+0000 7f6731000640 1 -- start start 2024-04-25T12:37:23.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.494+0000 7f6731000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f67240334e0 0x7f6724006a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.494+0000 7f6731000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67240d56f0 0x7f672404c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f672b400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f67240334e0 0x7f6724006a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f672be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67240d56f0 0x7f672404c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f672be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67240d56f0 0x7f672404c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56892/0 (socket says 172.21.15.123:56892) 2024-04-25T12:37:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f672aa00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f6724045e70 0x7f6724046d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:55946/0 (socket says 172.21.15.123:55946) 2024-04-25T12:37:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.493+0000 7f1a7f200640 1 -- start start 2024-04-25T12:37:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f8ded000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8de0045fd0 0x7f8de00062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f8ded000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de00d9a20 con 0x7f8de0006810 2024-04-25T12:37:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f8ded000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de00e0260 con 0x7f8de004c8c0 2024-04-25T12:37:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.495+0000 7f8ded000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8de004c520 con 0x7f8de0045fd0 2024-04-25T12:37:23.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f672b400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f67240334e0 0x7f6724006a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48336/0 (socket says 172.21.15.123:48336) 2024-04-25T12:37:23.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f672be00640 1 -- 172.21.15.123:0/1714340980 learned_addr learned my addr 172.21.15.123:0/1714340980 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f8de7400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8de004c8c0 0x7f8de00dfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:55936/0 (socket says 172.21.15.123:55936) 2024-04-25T12:37:23.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f1a7f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a78007610 0x7f1a780e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f8de7400640 1 -- 172.21.15.123:0/3700599364 learned_addr learned my addr 172.21.15.123:0/3700599364 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:23.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f8de6a00640 1 --2- 172.21.15.123:0/3700599364 >> v2:172.21.15.153:3300/0 conn(0x7f8de0045fd0 0x7f8de00062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6724047540 con 0x7f6724045e70 2024-04-25T12:37:23.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f6581800640 1 Processor -- start 2024-04-25T12:37:23.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6724034fe0 con 0x7f67240334e0 2024-04-25T12:37:23.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6724035840 con 0x7f67240d56f0 2024-04-25T12:37:23.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3589463223 0 0) 0x7f8de00e0260 con 0x7f8de004c8c0 2024-04-25T12:37:23.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dd0003520 con 0x7f8de004c8c0 2024-04-25T12:37:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f1a7f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a7804b6e0 0x7f1a7804baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f1a7f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a780062b0 con 0x7f1a780457c0 2024-04-25T12:37:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f1a7f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1a78006080 con 0x7f1a78007610 2024-04-25T12:37:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.496+0000 7f1a7f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1a78007180 con 0x7f1a7804b6e0 2024-04-25T12:37:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3540339423 0 0) 0x7f8dd0003520 con 0x7f8de004c8c0 2024-04-25T12:37:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 >> v2:172.21.15.153:3300/0 conn(0x7f8de0045fd0 msgr2=0x7f8de00062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f1a7de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a78007610 0x7f1a780e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f1a7e800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a7804b6e0 0x7f1a7804baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f8de6000640 1 --2- 172.21.15.123:0/3700599364 >> v2:172.21.15.153:3300/0 conn(0x7f8de0045fd0 0x7f8de00062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 >> v1:172.21.15.123:6789/0 conn(0x7f8de0006810 legacy=0x7f8de00e9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f1a7de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a78007610 0x7f1a780e36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41102/0 (socket says 172.21.15.123:41102) 2024-04-25T12:37:23.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f1a7de00640 1 -- 172.21.15.123:0/2806976474 learned_addr learned my addr 172.21.15.123:0/2806976474 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f6581800640 1 -- start start 2024-04-25T12:37:23.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f672a000640 1 -- 172.21.15.123:0/1714340980 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 943435446 0 0) 0x7f6724047540 con 0x7f6724045e70 2024-04-25T12:37:23.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f672b400640 1 -- 172.21.15.123:0/1714340980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67240d56f0 msgr2=0x7f672404c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f672b400640 1 --2- 172.21.15.123:0/1714340980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67240d56f0 0x7f672404c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f672b400640 1 -- 172.21.15.123:0/1714340980 >> v1:172.21.15.153:6789/0 conn(0x7f6724045e70 legacy=0x7f6724046d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.497+0000 7f672b400640 1 -- 172.21.15.123:0/1714340980 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6724046280 con 0x7f67240334e0 2024-04-25T12:37:23.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.498+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0007030 con 0x7f8de004c8c0 2024-04-25T12:37:23.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.498+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f8ddc001670 con 0x7f8de004c8c0 2024-04-25T12:37:23.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.498+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 >> v1:172.21.15.153:6789/0 conn(0x7f8de004c8c0 legacy=0x7f8de00dfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f1a7de00640 1 -- 172.21.15.123:0/2806976474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a7804b6e0 msgr2=0x7f1a7804baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f1a7de00640 1 --2- 172.21.15.123:0/2806976474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a7804b6e0 0x7f1a7804baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f1a7de00640 1 -- 172.21.15.123:0/2806976474 >> v1:172.21.15.123:6789/0 conn(0x7f1a780457c0 legacy=0x7f1a780dd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.498+0000 7f8de6000640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f6581800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f657c154840 0x7f657c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f6581800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0ccb90 0x7f657c0d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f6581800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f657c0cbb80 0x7f657c15da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.498+0000 7f8de6000640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd0004800 0x7f8dd0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f8de6000640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8dd00051a0 0x7f8dd00141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f657aa00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f657c154840 0x7f657c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f1a7de00640 1 -- 172.21.15.123:0/2806976474 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a78045bd0 con 0x7f1a78007610 2024-04-25T12:37:23.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8de004c520 con 0x7f8dd0004800 2024-04-25T12:37:23.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0014920 con 0x7f8dd00039b0 2024-04-25T12:37:23.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.499+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd0014b20 con 0x7f8dd00051a0 2024-04-25T12:37:23.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f8de7400640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f8de6a00640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd0004800 0x7f8dd0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f657a000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0ccb90 0x7f657c0d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f6581800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f657c0c5500 con 0x7f657c0cbb80 2024-04-25T12:37:23.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f6581800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f657c0caf50 con 0x7f657c154840 2024-04-25T12:37:23.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f6581800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f657c0cb230 con 0x7f657c0ccb90 2024-04-25T12:37:23.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f657a000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0ccb90 0x7f657c0d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56906/0 (socket says 172.21.15.123:56906) 2024-04-25T12:37:23.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.500+0000 7f657a000640 1 -- 172.21.15.123:0/4044056467 learned_addr learned my addr 172.21.15.123:0/4044056467 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f672b400640 1 --2- 172.21.15.123:0/1714340980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f6724006a90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67200071d0 tx=0x7f672002b1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38bb8a255de7dcd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f657b400640 1 --2- 172.21.15.123:0/4044056467 >> v2:172.21.15.123:3300/0 conn(0x7f657c0cbb80 0x7f657c15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f8de7e00640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8dd00051a0 0x7f8dd00141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f8de7400640 1 -- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8dd00051a0 msgr2=0x7f8dd00141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f8de7400640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8dd00051a0 0x7f8dd00141d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f8de7400640 1 -- 172.21.15.123:0/3700599364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd0004800 msgr2=0x7f8dd0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f8de7400640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd0004800 0x7f8dd0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f8de7400640 1 -- 172.21.15.123:0/3700599364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0007030 con 0x7f8dd00039b0 2024-04-25T12:37:23.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.502+0000 7f672a000640 1 -- 172.21.15.123:0/1714340980 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200089e0 con 0x7f67240334e0 2024-04-25T12:37:23.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.502+0000 7f672a000640 1 -- 172.21.15.123:0/1714340980 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6720008b80 con 0x7f67240334e0 2024-04-25T12:37:23.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.502+0000 7f672a000640 1 -- 172.21.15.123:0/1714340980 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000a0d0 con 0x7f67240334e0 2024-04-25T12:37:23.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.502+0000 7f1a7de00640 1 --2- 172.21.15.123:0/2806976474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e36c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009870 tx=0x7f1a7002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949414d4b9796885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f657a000640 1 -- 172.21.15.123:0/4044056467 >> v2:172.21.15.153:3300/0 conn(0x7f657c154840 msgr2=0x7f657c0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f657a000640 1 --2- 172.21.15.123:0/4044056467 >> v2:172.21.15.153:3300/0 conn(0x7f657c154840 0x7f657c0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f657a000640 1 -- 172.21.15.123:0/4044056467 >> v2:172.21.15.123:3300/0 conn(0x7f657c0cbb80 msgr2=0x7f657c15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f657a000640 1 --2- 172.21.15.123:0/4044056467 >> v2:172.21.15.123:3300/0 conn(0x7f657c0cbb80 0x7f657c15da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f657a000640 1 -- 172.21.15.123:0/4044056467 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c0c4eb0 con 0x7f657c0ccb90 2024-04-25T12:37:23.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.501+0000 7f8de6a00640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd0004800 0x7f8dd0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:23.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.502+0000 7f8de7e00640 1 -- 172.21.15.123:0/3700599364 reap_dead start 2024-04-25T12:37:23.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 msgr2=0x7f6724006a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f6731000640 1 --2- 172.21.15.123:0/1714340980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f6724006a90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67200071d0 tx=0x7f672002b1c0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f1a7ca00640 1 -- 172.21.15.123:0/2806976474 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70037070 con 0x7f1a78007610 2024-04-25T12:37:23.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f1a7ca00640 1 -- 172.21.15.123:0/2806976474 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7002ae50 con 0x7f1a78007610 2024-04-25T12:37:23.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f1a7ca00640 1 -- 172.21.15.123:0/2806976474 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70037070 con 0x7f1a78007610 2024-04-25T12:37:23.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 shutdown_connections 2024-04-25T12:37:23.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f6731000640 1 --2- 172.21.15.123:0/1714340980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67240d56f0 0x7f672404c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f6731000640 1 --2- 172.21.15.123:0/1714340980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f6724006a90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.503+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 >> 172.21.15.123:0/1714340980 conn(0x7f67240503b0 msgr2=0x7f67240507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 shutdown_connections 2024-04-25T12:37:23.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f6731000640 1 -- 172.21.15.123:0/1714340980 wait complete. 2024-04-25T12:37:23.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f657a000640 1 --2- 172.21.15.123:0/4044056467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0ccb90 0x7f657c0d0050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f657c0cb440 tx=0x7f656c02a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e83114043ccd93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f8de7400640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003e40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00c370 tx=0x7f8dd0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40014bf2aa3e53d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f6579600640 1 -- 172.21.15.123:0/4044056467 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c037070 con 0x7f657c0ccb90 2024-04-25T12:37:23.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f1a7f200640 1 -- 172.21.15.123:0/2806976474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 msgr2=0x7f1a780e36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f1a7f200640 1 --2- 172.21.15.123:0/2806976474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e36c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009870 tx=0x7f1a7002a300 comp rx=0 tx=0).stop 2024-04-25T12:37:23.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc036070 con 0x7f8dd00039b0 2024-04-25T12:37:23.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ddc005ad0 con 0x7f8dd00039b0 2024-04-25T12:37:23.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.504+0000 7f8de6000640 1 -- 172.21.15.123:0/3700599364 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc007d90 con 0x7f8dd00039b0 2024-04-25T12:37:23.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.505+0000 7f6579600640 1 -- 172.21.15.123:0/4044056467 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c032440 con 0x7f657c0ccb90 2024-04-25T12:37:23.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.505+0000 7f6579600640 1 -- 172.21.15.123:0/4044056467 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c032810 con 0x7f657c0ccb90 2024-04-25T12:37:23.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7faf1f600640 1 Processor -- start 2024-04-25T12:37:23.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f9c82c00640 1 Processor -- start 2024-04-25T12:37:23.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f8ded000640 1 -- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dd00039b0 msgr2=0x7f8dd0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f8ded000640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003e40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00c370 tx=0x7f8dd0005830 comp rx=0 tx=0).stop 2024-04-25T12:37:23.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f1a7f200640 1 -- 172.21.15.123:0/2806976474 shutdown_connections 2024-04-25T12:37:23.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f1a7f200640 1 --2- 172.21.15.123:0/2806976474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a7804b6e0 0x7f1a7804baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f1a7f200640 1 --2- 172.21.15.123:0/2806976474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e36c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f1a7f200640 1 -- 172.21.15.123:0/2806976474 >> 172.21.15.123:0/2806976474 conn(0x7f1a780503a0 msgr2=0x7f1a780507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7faf1f600640 1 -- start start 2024-04-25T12:37:23.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f8ded000640 1 -- 172.21.15.123:0/3700599364 shutdown_connections 2024-04-25T12:37:23.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f8ded000640 1 --2- 172.21.15.123:0/3700599364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003e40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f8ded000640 1 -- 172.21.15.123:0/3700599364 >> 172.21.15.123:0/3700599364 conn(0x7f8de00503b0 msgr2=0x7f8de00507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f1a7f200640 1 -- 172.21.15.123:0/2806976474 shutdown_connections 2024-04-25T12:37:23.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f6731000640 1 Processor -- start 2024-04-25T12:37:23.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f6581800640 1 -- 172.21.15.123:0/4044056467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0ccb90 msgr2=0x7f657c0d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f6581800640 1 --2- 172.21.15.123:0/4044056467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0ccb90 0x7f657c0d0050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f657c0cb440 tx=0x7f656c02a280 comp rx=0 tx=0).stop 2024-04-25T12:37:23.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.508+0000 7fde22a00640 1 Processor -- start 2024-04-25T12:37:23.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.508+0000 7f6731000640 1 -- start start 2024-04-25T12:37:23.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.508+0000 7f8ded000640 1 -- 172.21.15.123:0/3700599364 shutdown_connections 2024-04-25T12:37:23.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.508+0000 7f8ded000640 1 -- 172.21.15.123:0/3700599364 wait complete. 2024-04-25T12:37:23.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.508+0000 7fde22a00640 1 -- start start 2024-04-25T12:37:23.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.506+0000 7f9c82c00640 1 -- start start 2024-04-25T12:37:23.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.507+0000 7f1a7f200640 1 -- 172.21.15.123:0/2806976474 wait complete. 2024-04-25T12:37:23.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7fb422800640 1 Processor -- start 2024-04-25T12:37:23.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f6581800640 1 -- 172.21.15.123:0/4044056467 shutdown_connections 2024-04-25T12:37:23.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f6581800640 1 --2- 172.21.15.123:0/4044056467 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0ccb90 0x7f657c0d0050 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f6581800640 1 --2- 172.21.15.123:0/4044056467 >> v2:172.21.15.123:3300/0 conn(0x7f657c0cbb80 0x7f657c15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f6581800640 1 --2- 172.21.15.123:0/4044056467 >> v2:172.21.15.153:3300/0 conn(0x7f657c154840 0x7f657c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f6581800640 1 -- 172.21.15.123:0/4044056467 >> 172.21.15.123:0/4044056467 conn(0x7f657c06e960 msgr2=0x7f657c06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7fb422800640 1 -- start start 2024-04-25T12:37:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f9c82c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c7c0c90d0 0x7f9c7c0c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f9c82c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9c7c0c4240 0x7f9c7c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7faf1f600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faf180c2870 0x7faf18073db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7faf1f600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf180c4810 con 0x7faf180c90d0 2024-04-25T12:37:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7faf1f600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf180c83f0 con 0x7faf180c4240 2024-04-25T12:37:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7faf1f600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7faf180ce380 con 0x7faf180c2870 2024-04-25T12:37:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f9c82c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7c0c4810 con 0x7f9c7c15a730 2024-04-25T12:37:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.509+0000 7f9c82c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7c0c83f0 con 0x7f9c7c0c4240 2024-04-25T12:37:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f9c82c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7c0c86d0 con 0x7f9c7c0c90d0 2024-04-25T12:37:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f6581800640 1 -- 172.21.15.123:0/4044056467 shutdown_connections 2024-04-25T12:37:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7faf17e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7faf180c4240 0x7faf1815a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:55954/0 (socket says 172.21.15.123:55954) 2024-04-25T12:37:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7faf17e00640 1 -- 172.21.15.123:0/213544318 learned_addr learned my addr 172.21.15.123:0/213544318 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7faf1d600640 1 --2- 172.21.15.123:0/213544318 >> v2:172.21.15.123:3300/0 conn(0x7faf180c2870 0x7faf18073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f9c7be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c7c0c90d0 0x7f9c7c0c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f9c7be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c7c0c90d0 0x7f9c7c0c2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48374/0 (socket says 172.21.15.123:48374) 2024-04-25T12:37:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f9c7be00640 1 -- 172.21.15.123:0/2302466170 learned_addr learned my addr 172.21.15.123:0/2302466170 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f9c7be00640 1 -- 172.21.15.123:0/2302466170 >> v2:172.21.15.123:3300/0 conn(0x7f9c7c0c4240 msgr2=0x7f9c7c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f6731000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f67240db790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f6731000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6724045e70 0x7f67240d63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f9c7be00640 1 --2- 172.21.15.123:0/2302466170 >> v2:172.21.15.123:3300/0 conn(0x7f9c7c0c4240 0x7f9c7c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f9c7be00640 1 -- 172.21.15.123:0/2302466170 >> v1:172.21.15.123:6789/0 conn(0x7f9c7c15a730 legacy=0x7f9c7c073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.511+0000 7f6731000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67240d56f0 0x7f67240cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7fde22a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde1404da60 0x7fde140d6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.511+0000 7fde22a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde14047930 con 0x7fde14046db0 2024-04-25T12:37:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.511+0000 7fde22a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde14047bd0 con 0x7fde140464b0 2024-04-25T12:37:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.511+0000 7fde22a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fde14047eb0 con 0x7fde1404da60 2024-04-25T12:37:23.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.511+0000 7f672aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6724045e70 0x7f67240d63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.511+0000 7f672b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f67240db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f6731000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6724006570 con 0x7f67240d56f0 2024-04-25T12:37:23.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7fde22000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fde14046db0 0x7fde14006c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58598/0 (socket says 172.21.15.123:58598) 2024-04-25T12:37:23.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7fde22000640 1 -- 172.21.15.123:0/2194182048 learned_addr learned my addr 172.21.15.123:0/2194182048 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f6731000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6724035560 con 0x7f67240334e0 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f6731000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6724045350 con 0x7f6724045e70 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f672b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f67240db790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48388/0 (socket says 172.21.15.123:48388) 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f672b400640 1 -- 172.21.15.123:0/2220150994 learned_addr learned my addr 172.21.15.123:0/2220150994 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f672b400640 1 -- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6724045e70 msgr2=0x7f67240d63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f672b400640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6724045e70 0x7f67240d63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f672b400640 1 -- 172.21.15.123:0/2220150994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67240d56f0 msgr2=0x7f67240cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f672b400640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67240d56f0 0x7f67240cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f672b400640 1 -- 172.21.15.123:0/2220150994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67240dbcd0 con 0x7f67240334e0 2024-04-25T12:37:23.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f1a7f200640 1 Processor -- start 2024-04-25T12:37:23.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7faf17400640 1 -- 172.21.15.123:0/213544318 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 289168201 0 0) 0x7faf180c83f0 con 0x7faf180c4240 2024-04-25T12:37:23.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7faf17400640 1 -- 172.21.15.123:0/213544318 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faefc003520 con 0x7faf180c4240 2024-04-25T12:37:23.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.510+0000 7f6581800640 1 -- 172.21.15.123:0/4044056467 wait complete. 2024-04-25T12:37:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7f8ded000640 1 Processor -- start 2024-04-25T12:37:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fb422800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb41c0cde90 0x7fb41c167f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fb422800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c0cc3b0 con 0x7fb41c0c6970 2024-04-25T12:37:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fb422800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c1609b0 con 0x7fb41c156ad0 2024-04-25T12:37:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fb422800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb41c0cc650 con 0x7fb41c0cde90 2024-04-25T12:37:23.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7f1a7f200640 1 -- start start 2024-04-25T12:37:23.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589683464 0 0) 0x7fde14047930 con 0x7fde14046db0 2024-04-25T12:37:23.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde04003520 con 0x7fde14046db0 2024-04-25T12:37:23.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7faf17400640 1 -- 172.21.15.123:0/213544318 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617515040 0 0) 0x7faf180c4810 con 0x7faf180c90d0 2024-04-25T12:37:23.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7faf17400640 1 -- 172.21.15.123:0/213544318 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf180c83f0 con 0x7faf180c90d0 2024-04-25T12:37:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fb41b400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb41c0c6970 0x7fb41c0cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58606/0 (socket says 172.21.15.123:58606) 2024-04-25T12:37:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fb41b400640 1 -- 172.21.15.123:0/932243601 learned_addr learned my addr 172.21.15.123:0/932243601 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.512+0000 7f9c7be00640 1 -- 172.21.15.123:0/2302466170 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c0c89b0 con 0x7f9c7c0c90d0 2024-04-25T12:37:23.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7f6581800640 1 Processor -- start 2024-04-25T12:37:23.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fb41be00640 1 --2- 172.21.15.123:0/932243601 >> v2:172.21.15.153:3300/0 conn(0x7fb41c0cde90 0x7fb41c167f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7f672b400640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f67240db790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67200071d0 tx=0x7f67200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7f6581800640 1 -- start start 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.513+0000 7fde21600640 1 --2- 172.21.15.123:0/2194182048 >> v2:172.21.15.123:3300/0 conn(0x7fde1404da60 0x7fde140d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1134985531 0 0) 0x7fde14047bd0 con 0x7fde140464b0 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde14047930 con 0x7fde140464b0 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 454684511 0 0) 0x7fde14047930 con 0x7fde140464b0 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 >> v2:172.21.15.123:3300/0 conn(0x7fde1404da60 msgr2=0x7fde140d6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 --2- 172.21.15.123:0/2194182048 >> v2:172.21.15.123:3300/0 conn(0x7fde1404da60 0x7fde140d6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 >> v1:172.21.15.123:6789/0 conn(0x7fde14046db0 legacy=0x7fde14006c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde14048190 con 0x7fde140464b0 2024-04-25T12:37:23.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7fde08001670 con 0x7fde140464b0 2024-04-25T12:37:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7f8ded000640 1 -- start start 2024-04-25T12:37:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 >> v1:172.21.15.153:6789/0 conn(0x7fde140464b0 legacy=0x7fde140dc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7faf1d600640 1 -- 172.21.15.123:0/213544318 >> v1:172.21.15.153:6789/0 conn(0x7faf180c4240 legacy=0x7faf1815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7f671f400640 1 -- 172.21.15.123:0/2220150994 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720042020 con 0x7f67240334e0 2024-04-25T12:37:23.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7f671f400640 1 -- 172.21.15.123:0/2220150994 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672000a690 con 0x7f67240334e0 2024-04-25T12:37:23.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7faf1d600640 1 -- 172.21.15.123:0/213544318 >> v1:172.21.15.123:6789/0 conn(0x7faf180c90d0 legacy=0x7faf180cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7faf1d600640 1 -- 172.21.15.123:0/213544318 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf180c86d0 con 0x7faf180c2870 2024-04-25T12:37:23.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.514+0000 7f671f400640 1 -- 172.21.15.123:0/2220150994 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720008260 con 0x7f67240334e0 2024-04-25T12:37:23.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7f9c7be00640 1 --2- 172.21.15.123:0/2302466170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 0x7f9c7c0c2870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009870 tx=0x7f9c6c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c6b1b144604978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6724172260 con 0x7f67240334e0 2024-04-25T12:37:23.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fb41a000640 1 -- 172.21.15.123:0/932243601 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2675962254 0 0) 0x7fb41c0cc3b0 con 0x7fb41c0c6970 2024-04-25T12:37:23.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fb41a000640 1 -- 172.21.15.123:0/932243601 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb400003520 con 0x7fb41c0c6970 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fde1be00640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04003b40 0x7fde04003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fde1be00640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde04004800 0x7fde04004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fde1be00640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde04005150 0x7fde0400b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde14047eb0 con 0x7fde04003b40 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde0400c0e0 con 0x7fde04005150 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.515+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fde0400c2e0 con 0x7fde04004800 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7fde22000640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde04005150 0x7fde0400b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7fde21600640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04003b40 0x7fde04003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f1a7f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7fb41be00640 1 -- 172.21.15.123:0/932243601 >> v1:172.21.15.153:6789/0 conn(0x7fb41c156ad0 legacy=0x7fb41c06d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7fb41be00640 1 -- 172.21.15.123:0/932243601 >> v1:172.21.15.123:6789/0 conn(0x7fb41c0c6970 legacy=0x7fb41c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7fb41be00640 1 -- 172.21.15.123:0/932243601 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c0cc930 con 0x7fb41c0cde90 2024-04-25T12:37:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7fb41a000640 1 -- 172.21.15.123:0/932243601 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764930860 0 0) 0x7fb41c1609b0 con 0x7fb41c156ad0 2024-04-25T12:37:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f1a7f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a780457c0 0x7f1a780d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f1a7f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a7804b6e0 0x7f1a780347e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7fde20c00640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde04004800 0x7fde04004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f8ded000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd00039b0 0x7f8de00d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f8ded000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8de0175450 0x7f8de00465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f8ded000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de00e2180 0x7f8de00cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f8ded000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8de00e34a0 con 0x7f8dd00039b0 2024-04-25T12:37:23.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f8ded000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8de004bfa0 con 0x7f8de00e2180 2024-04-25T12:37:23.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7f8ded000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8de00e3740 con 0x7f8de0175450 2024-04-25T12:37:23.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.516+0000 7faf1d600640 1 --2- 172.21.15.123:0/213544318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf18073db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf100097d0 tx=0x7faf1002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440ca94ea64d6897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78006b30 con 0x7f1a78007610 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78006850 con 0x7f1a7804b6e0 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1a78006e10 con 0x7f1a780457c0 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a780457c0 0x7f1a780d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41182/0 (socket says 172.21.15.123:41182) 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7de00640 1 -- 172.21.15.123:0/2009249988 learned_addr learned my addr 172.21.15.123:0/2009249988 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f1a7e800640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a7804b6e0 0x7f1a780347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f6581800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0cbb80 0x7f657c14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f9c7aa00640 1 -- 172.21.15.123:0/2302466170 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c037070 con 0x7f9c7c0c90d0 2024-04-25T12:37:23.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7faf17400640 1 -- 172.21.15.123:0/213544318 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10037070 con 0x7faf180c2870 2024-04-25T12:37:23.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f9c7aa00640 1 -- 172.21.15.123:0/2302466170 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6c032440 con 0x7f9c7c0c90d0 2024-04-25T12:37:23.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f9c7aa00640 1 -- 172.21.15.123:0/2302466170 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c032810 con 0x7f9c7c0c90d0 2024-04-25T12:37:23.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7fb41be00640 1 --2- 172.21.15.123:0/932243601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c167f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb4080097a0 tx=0x7fb40802a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca02934bb657b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7faf17400640 1 -- 172.21.15.123:0/213544318 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf10032440 con 0x7faf180c2870 2024-04-25T12:37:23.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7faf17400640 1 -- 172.21.15.123:0/213544318 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10032810 con 0x7faf180c2870 2024-04-25T12:37:23.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f6581800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f657c0ccb90 0x7f657c1e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a7e800640 1 -- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a780457c0 msgr2=0x7f1a780d31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f6581800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f657c06cf30 0x7f657c145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f6581800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f657c0c4720 con 0x7f657c0ccb90 2024-04-25T12:37:23.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f6581800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f657c0cb670 con 0x7f657c06cf30 2024-04-25T12:37:23.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f6581800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f657c0c48a0 con 0x7f657c0cbb80 2024-04-25T12:37:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7fde22000640 1 -- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde04004800 msgr2=0x7fde04004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7fde22000640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde04004800 0x7fde04004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7fde22000640 1 -- 172.21.15.123:0/2194182048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04003b40 msgr2=0x7fde04003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7fde22000640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04003b40 0x7fde04003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7fde22000640 1 -- 172.21.15.123:0/2194182048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde14048190 con 0x7fde04005150 2024-04-25T12:37:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7fde21600640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04003b40 0x7fde04003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a7e800640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a780457c0 0x7f1a780d31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f8de7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de00e2180 0x7f8de00cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f8de6a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8de0175450 0x7f8de00465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f8de7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de00e2180 0x7f8de00cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48422/0 (socket says 172.21.15.123:48422) 2024-04-25T12:37:23.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.517+0000 7f8de7e00640 1 -- 172.21.15.123:0/706970335 learned_addr learned my addr 172.21.15.123:0/706970335 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f657aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0cbb80 0x7f657c14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f657aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0cbb80 0x7f657c14d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56958/0 (socket says 172.21.15.123:56958) 2024-04-25T12:37:23.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f657aa00640 1 -- 172.21.15.123:0/285693978 learned_addr learned my addr 172.21.15.123:0/285693978 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7fb41a000640 1 -- 172.21.15.123:0/932243601 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408037070 con 0x7fb41c0cde90 2024-04-25T12:37:23.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7fde22000640 1 -- 172.21.15.123:0/2194182048 reap_dead start 2024-04-25T12:37:23.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f657b400640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f657c06cf30 0x7f657c145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7fb41a000640 1 -- 172.21.15.123:0/932243601 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb408032640 con 0x7fb41c0cde90 2024-04-25T12:37:23.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7fb41a000640 1 -- 172.21.15.123:0/932243601 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408032a10 con 0x7fb41c0cde90 2024-04-25T12:37:23.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a7e800640 1 -- 172.21.15.123:0/2009249988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 msgr2=0x7f1a780e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a7e800640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a7e800640 1 -- 172.21.15.123:0/2009249988 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a780e0950 con 0x7f1a7804b6e0 2024-04-25T12:37:23.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a7de00640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:23.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a7e800640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a7804b6e0 0x7f1a780347e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1a80066010 tx=0x7f1a80069cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7fde22000640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde04005150 0x7fde0400b9b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0771e0 tx=0x7fde04005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9463e24181a94f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6724171ca0 con 0x7f67240334e0 2024-04-25T12:37:23.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f1a76000640 1 -- 172.21.15.123:0/2009249988 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8006db00 con 0x7f1a7804b6e0 2024-04-25T12:37:23.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f1a76000640 1 -- 172.21.15.123:0/2009249988 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8006dca0 con 0x7f1a7804b6e0 2024-04-25T12:37:23.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f1a76000640 1 -- 172.21.15.123:0/2009249988 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80096620 con 0x7f1a7804b6e0 2024-04-25T12:37:23.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.518+0000 7f657a000640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f657c0ccb90 0x7f657c1e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0b3070 con 0x7fde04005150 2024-04-25T12:37:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1c06e270 con 0x7fde04005150 2024-04-25T12:37:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7fde1be00640 1 -- 172.21.15.123:0/2194182048 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1000b5e0 con 0x7fde04005150 2024-04-25T12:37:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f9c82c00640 1 -- 172.21.15.123:0/2302466170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 msgr2=0x7f9c7c0c2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f9c82c00640 1 --2- 172.21.15.123:0/2302466170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 0x7f9c7c0c2870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009870 tx=0x7f9c6c02a2c0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f1a7f200640 1 -- 172.21.15.123:0/2009249988 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a780d39c0 con 0x7f1a7804b6e0 2024-04-25T12:37:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f1a7f200640 1 -- 172.21.15.123:0/2009249988 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a780df930 con 0x7f1a7804b6e0 2024-04-25T12:37:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7faf1f600640 1 -- 172.21.15.123:0/213544318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 msgr2=0x7faf18073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7faf1f600640 1 --2- 172.21.15.123:0/213544318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf18073db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf100097d0 tx=0x7faf1002a250 comp rx=0 tx=0).stop 2024-04-25T12:37:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f9c82c00640 1 -- 172.21.15.123:0/2302466170 shutdown_connections 2024-04-25T12:37:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f9c82c00640 1 --2- 172.21.15.123:0/2302466170 >> v2:172.21.15.123:3300/0 conn(0x7f9c7c0c4240 0x7f9c7c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f9c82c00640 1 --2- 172.21.15.123:0/2302466170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 0x7f9c7c0c2870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f9c82c00640 1 -- 172.21.15.123:0/2302466170 >> 172.21.15.123:0/2302466170 conn(0x7f9c7c06c750 msgr2=0x7f9c7c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f8de7e00640 1 -- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8de0175450 msgr2=0x7f8de00465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f8de7e00640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8de0175450 0x7f8de00465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f8de7e00640 1 -- 172.21.15.123:0/706970335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd00039b0 msgr2=0x7f8de00d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f8de7400640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd00039b0 0x7f8de00d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f657aa00640 1 -- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f657c06cf30 msgr2=0x7f657c145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f657aa00640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f657c06cf30 0x7f657c145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f657aa00640 1 -- 172.21.15.123:0/285693978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f657c0ccb90 msgr2=0x7f657c1e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f657aa00640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f657c0ccb90 0x7f657c1e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f657aa00640 1 -- 172.21.15.123:0/285693978 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c146b60 con 0x7f657c0cbb80 2024-04-25T12:37:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f657b400640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f657c06cf30 0x7f657c145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7faf1f600640 1 -- 172.21.15.123:0/213544318 shutdown_connections 2024-04-25T12:37:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7faf1f600640 1 --2- 172.21.15.123:0/213544318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf18073db0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7faf1f600640 1 -- 172.21.15.123:0/213544318 >> 172.21.15.123:0/213544318 conn(0x7faf1806c750 msgr2=0x7faf1806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7f9c82c00640 1 -- 172.21.15.123:0/2302466170 shutdown_connections 2024-04-25T12:37:23.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f8de7e00640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd00039b0 0x7f8de00d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7f8de7e00640 1 -- 172.21.15.123:0/706970335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de00e2f90 con 0x7f8de00e2180 2024-04-25T12:37:23.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7f9c82c00640 1 -- 172.21.15.123:0/2302466170 wait complete. 2024-04-25T12:37:23.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7faf1f600640 1 -- 172.21.15.123:0/213544318 shutdown_connections 2024-04-25T12:37:23.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7f8de6a00640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8de0175450 0x7f8de00465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:23.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7fb422800640 1 -- 172.21.15.123:0/932243601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 msgr2=0x7fb41c167f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.519+0000 7fb422800640 1 --2- 172.21.15.123:0/932243601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c167f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb4080097a0 tx=0x7fb40802a2e0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7fde22a00640 1 -- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde04005150 msgr2=0x7fde0400b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7fde22a00640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde04005150 0x7fde0400b9b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0771e0 tx=0x7fde04005830 comp rx=0 tx=0).stop 2024-04-25T12:37:23.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7f657aa00640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0cbb80 0x7f657c14d000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f656400e870 tx=0x7f656400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6724005570 con 0x7f67240334e0 2024-04-25T12:37:23.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7fb422800640 1 -- 172.21.15.123:0/932243601 shutdown_connections 2024-04-25T12:37:23.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7fb422800640 1 --2- 172.21.15.123:0/932243601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c167f20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7fb422800640 1 -- 172.21.15.123:0/932243601 >> 172.21.15.123:0/932243601 conn(0x7fb41c0d1a30 msgr2=0x7fb41c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7faf1f600640 1 -- 172.21.15.123:0/213544318 wait complete. 2024-04-25T12:37:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7fb422800640 1 -- 172.21.15.123:0/932243601 shutdown_connections 2024-04-25T12:37:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7fb422800640 1 -- 172.21.15.123:0/932243601 wait complete. 2024-04-25T12:37:23.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7f6572a00640 1 -- 172.21.15.123:0/285693978 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564017070 con 0x7f657c0cbb80 2024-04-25T12:37:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7f1a7f200640 1 -- 172.21.15.123:0/2009249988 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a78005560 con 0x7f1a7804b6e0 2024-04-25T12:37:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7f8de7e00640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de00e2180 0x7f8de00cdb90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8de8065b40 tx=0x7f8de8069cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657c14c8d0 con 0x7f657c0cbb80 2024-04-25T12:37:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.522+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f657c0cea20 con 0x7f657c0cbb80 2024-04-25T12:37:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.522+0000 7f6572a00640 1 -- 172.21.15.123:0/285693978 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656400c940 con 0x7f657c0cbb80 2024-04-25T12:37:23.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.522+0000 7f6572a00640 1 -- 172.21.15.123:0/285693978 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564017070 con 0x7f657c0cbb80 2024-04-25T12:37:23.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7fde22a00640 1 -- 172.21.15.123:0/2194182048 shutdown_connections 2024-04-25T12:37:23.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7fde22a00640 1 --2- 172.21.15.123:0/2194182048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde04005150 0x7fde0400b9b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.520+0000 7fde22a00640 1 -- 172.21.15.123:0/2194182048 >> 172.21.15.123:0/2194182048 conn(0x7fde1404df90 msgr2=0x7fde1404e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7fde22a00640 1 -- 172.21.15.123:0/2194182048 shutdown_connections 2024-04-25T12:37:23.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.521+0000 7fde22a00640 1 -- 172.21.15.123:0/2194182048 wait complete. 2024-04-25T12:37:23.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.522+0000 7f1a76000640 1 -- 172.21.15.123:0/2009249988 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a800967c0 con 0x7f1a7804b6e0 2024-04-25T12:37:23.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.523+0000 7f671f400640 1 -- 172.21.15.123:0/2220150994 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6720045020 con 0x7f67240334e0 2024-04-25T12:37:23.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.523+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6554005290 con 0x7f657c0cbb80 2024-04-25T12:37:23.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.524+0000 7fde22a00640 1 Processor -- start 2024-04-25T12:37:23.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.524+0000 7faf1f600640 1 Processor -- start 2024-04-25T12:37:23.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.524+0000 7f1a76000640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1a600744b0 0x7f1a600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.524+0000 7f1a76000640 1 -- 172.21.15.123:0/2009249988 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f1a80112310 con 0x7f1a7804b6e0 2024-04-25T12:37:23.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.524+0000 7fde22a00640 1 -- start start 2024-04-25T12:37:23.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7faf1f600640 1 -- start start 2024-04-25T12:37:23.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7fb422800640 1 Processor -- start 2024-04-25T12:37:23.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7f1a7de00640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1a600744b0 0x7f1a600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7f671f400640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f670c0745b0 0x7f670c076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7fb422800640 1 -- start start 2024-04-25T12:37:23.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7f9c82c00640 1 Processor -- start 2024-04-25T12:37:23.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7f671f400640 1 -- 172.21.15.123:0/2220150994 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f67200b6300 con 0x7f67240334e0 2024-04-25T12:37:23.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7f672aa00640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f670c0745b0 0x7f670c076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.525+0000 7f9c82c00640 1 -- start start 2024-04-25T12:37:23.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7f672aa00640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f670c0745b0 0x7f670c076ac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f672c06b050 tx=0x7f672c0696d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7f1a7de00640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1a600744b0 0x7f1a600769c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009870 tx=0x7f1a7002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7fde22a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04005150 0x7fde140e0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7fde22a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1404f620 0x7fde140ea360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7fde22a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde1417cd30 0x7fde1417d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7fde22a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde140356b0 con 0x7fde04005150 2024-04-25T12:37:23.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7fde22a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde140dc1e0 con 0x7fde1404f620 2024-04-25T12:37:23.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7fde22a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fde14034130 con 0x7fde1417cd30 2024-04-25T12:37:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.526+0000 7f6572a00640 1 -- 172.21.15.123:0/285693978 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f656401a070 con 0x7f657c0cbb80 2024-04-25T12:37:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f8ddb400640 1 -- 172.21.15.123:0/706970335 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de806db00 con 0x7f8de00e2180 2024-04-25T12:37:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de00e1900 con 0x7f8de00e2180 2024-04-25T12:37:23.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7fde22000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde1417cd30 0x7fde1417d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de00e1ba0 con 0x7f8de00e2180 2024-04-25T12:37:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7fde22000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde1417cd30 0x7fde1417d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56964/0 (socket says 172.21.15.123:56964) 2024-04-25T12:37:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7fde22000640 1 -- 172.21.15.123:0/2645476674 learned_addr learned my addr 172.21.15.123:0/2645476674 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7fde21600640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04005150 0x7fde140e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7faf1f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf181f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb422800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0c6970 0x7fb41c200000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb422800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb422800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb41c156ad0 0x7fb41c203260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb422800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c06e6e0 con 0x7fb41c0c6970 2024-04-25T12:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb422800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0ccef0 con 0x7fb41c0cde90 2024-04-25T12:37:23.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb422800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb41c06dcd0 con 0x7fb41c156ad0 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f9c82c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0c4240 0x7f9c7c1fa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f9c82c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 0x7f9c7c14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f9c82c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c7c15a730 0x7f9c7c14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f9c82c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c0ce330 con 0x7f9c7c0c4240 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f9c82c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c0ce1b0 con 0x7f9c7c0c90d0 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.527+0000 7f9c82c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c0744c0 con 0x7f9c7c15a730 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0c4240 0x7f9c7c1fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0c4240 0x7f9c7c1fa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41222/0 (socket says 172.21.15.123:41222) 2024-04-25T12:37:23.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 -- 172.21.15.123:0/3586213781 learned_addr learned my addr 172.21.15.123:0/3586213781 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf181f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1f600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf180c4240 0x7faf180c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1f600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf180c90d0 0x7faf181faed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faf180c2c80 con 0x7faf180c2870 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1f600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faf180c8c90 con 0x7faf180c4240 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1f600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faf1815ae40 con 0x7faf180c90d0 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf181f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41214/0 (socket says 172.21.15.123:41214) 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7faf1cc00640 1 -- 172.21.15.123:0/2968766112 learned_addr learned my addr 172.21.15.123:0/2968766112 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7faf1d600640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf180c90d0 0x7faf181faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7faf17e00640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf180c4240 0x7faf180c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7faf1cc00640 1 -- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf180c90d0 msgr2=0x7faf181faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7faf1cc00640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf180c90d0 0x7faf181faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7faf1cc00640 1 -- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf180c4240 msgr2=0x7faf180c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7faf1cc00640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf180c4240 0x7faf180c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de00dcd20 con 0x7f8de00e2180 2024-04-25T12:37:23.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb41be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb41c156ad0 0x7fb41c203260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7faf1cc00640 1 -- 172.21.15.123:0/2968766112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf181f6f50 con 0x7faf180c2870 2024-04-25T12:37:23.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fb41be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb41c156ad0 0x7fb41c203260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:56994/0 (socket says 172.21.15.123:56994) 2024-04-25T12:37:23.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde20c00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1404f620 0x7fde140ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde21600640 1 -- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde1417cd30 msgr2=0x7fde1417d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde21600640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde1417cd30 0x7fde1417d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde21600640 1 -- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1404f620 msgr2=0x7fde140ea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde21600640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1404f620 0x7fde140ea360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde21600640 1 -- 172.21.15.123:0/2645476674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde140ddba0 con 0x7fde04005150 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde20c00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1404f620 0x7fde140ea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7fde21600640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04005150 0x7fde140e0020 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b550 tx=0x7fde1000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7fde19600640 1 -- 172.21.15.123:0/2645476674 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10018070 con 0x7fde04005150 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7faf1cc00640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf181f83c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c3f0 tx=0x7faf0400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1404b860 con 0x7fde04005150 2024-04-25T12:37:23.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7fde19600640 1 -- 172.21.15.123:0/2645476674 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde100107a0 con 0x7fde04005150 2024-04-25T12:37:23.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7fde19600640 1 -- 172.21.15.123:0/2645476674 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10010b50 con 0x7fde04005150 2024-04-25T12:37:23.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7f671f400640 1 -- 172.21.15.123:0/2220150994 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f672004c070 con 0x7f67240334e0 2024-04-25T12:37:23.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde140de120 con 0x7fde04005150 2024-04-25T12:37:23.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7fb41be00640 1 -- 172.21.15.123:0/2499205330 learned_addr learned my addr 172.21.15.123:0/2499205330 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c80c00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c7c15a730 0x7f9c7c14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 -- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c7c15a730 msgr2=0x7f9c7c14a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c7c15a730 0x7f9c7c14a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 -- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 msgr2=0x7f9c7c14e0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 0x7f9c7c14e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.528+0000 7f9c7be00640 1 -- 172.21.15.123:0/3586213781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c1fa8b0 con 0x7f9c7c0c4240 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.529+0000 7f9c7be00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0c4240 0x7f9c7c1fa370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009870 tx=0x7f9c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7f9c73e00640 1 -- 172.21.15.123:0/3586213781 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c041070 con 0x7f9c7c0c4240 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7f9c73e00640 1 -- 172.21.15.123:0/3586213781 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6c03c930 con 0x7f9c7c0c4240 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c14e8c0 con 0x7f9c7c0c4240 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c0c6cd0 con 0x7f9c7c0c4240 2024-04-25T12:37:23.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7f9c73e00640 1 -- 172.21.15.123:0/3586213781 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c037040 con 0x7f9c7c0c4240 2024-04-25T12:37:23.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde140059c0 con 0x7fde04005150 2024-04-25T12:37:23.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7f6572a00640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f65580746e0 0x7f6558076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.530+0000 7fb41aa00640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7fb41b400640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0c6970 0x7fb41c200000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7faf14c00640 1 -- 172.21.15.123:0/2968766112 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04017070 con 0x7faf180c2870 2024-04-25T12:37:23.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7faf14c00640 1 -- 172.21.15.123:0/2968766112 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf040035f0 con 0x7faf180c2870 2024-04-25T12:37:23.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7faf14c00640 1 -- 172.21.15.123:0/2968766112 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf040039e0 con 0x7faf180c2870 2024-04-25T12:37:23.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7f6572a00640 1 -- 172.21.15.123:0/285693978 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f6564095310 con 0x7f657c0cbb80 2024-04-25T12:37:23.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7f657a000640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f65580746e0 0x7f6558076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7fb41aa00640 1 -- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb41c156ad0 msgr2=0x7fb41c203260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7fb41aa00640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb41c156ad0 0x7fb41c203260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7fb41aa00640 1 -- 172.21.15.123:0/2499205330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0c6970 msgr2=0x7fb41c200000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7fb41aa00640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0c6970 0x7fb41c200000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7fb41aa00640 1 -- 172.21.15.123:0/2499205330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c15ae10 con 0x7fb41c0cde90 2024-04-25T12:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf181f7210 con 0x7faf180c2870 2024-04-25T12:37:23.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.531+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf181f77d0 con 0x7faf180c2870 2024-04-25T12:37:23.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.532+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c076700 con 0x7f9c7c0c4240 2024-04-25T12:37:23.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.532+0000 7fb41aa00640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c15ce40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb41c069630 tx=0x7fb40c00c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.532+0000 7f657a000640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f65580746e0 0x7f6558076bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f656c034000 tx=0x7f656c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.533+0000 7fde19600640 1 -- 172.21.15.123:0/2645476674 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde1001b070 con 0x7fde04005150 2024-04-25T12:37:23.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.533+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf18076700 con 0x7faf180c2870 2024-04-25T12:37:23.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.534+0000 7fde19600640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fddfc0744b0 0x7fddfc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.535+0000 7faf14c00640 1 -- 172.21.15.123:0/2968766112 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf0401a070 con 0x7faf180c2870 2024-04-25T12:37:23.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.535+0000 7fde19600640 1 -- 172.21.15.123:0/2645476674 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fde100958c0 con 0x7fde04005150 2024-04-25T12:37:23.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.535+0000 7fde20c00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fddfc0744b0 0x7fddfc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.535+0000 7f9c73e00640 1 -- 172.21.15.123:0/3586213781 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c6c044070 con 0x7f9c7c0c4240 2024-04-25T12:37:23.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.536+0000 7fde20c00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fddfc0744b0 0x7fddfc0769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fde08001820 tx=0x7fde08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.536+0000 7f9c73e00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9c5c0744e0 0x7f9c5c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.536+0000 7f9c73e00640 1 -- 172.21.15.123:0/3586213781 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f9c6c0b6390 con 0x7f9c7c0c4240 2024-04-25T12:37:23.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.536+0000 7faf14c00640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faef8074530 0x7faef8076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.536+0000 7f9c7b400640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9c5c0744e0 0x7f9c5c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.536+0000 7faf14c00640 1 -- 172.21.15.123:0/2968766112 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7faf04094eb0 con 0x7faf180c2870 2024-04-25T12:37:23.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.536+0000 7faf17e00640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faef8074530 0x7faef8076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7f8ddb400640 1 -- 172.21.15.123:0/706970335 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de8069e20 con 0x7f8de00e2180 2024-04-25T12:37:23.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7f8ddb400640 1 -- 172.21.15.123:0/706970335 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de809b060 con 0x7f8de00e2180 2024-04-25T12:37:23.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7fb413400640 1 -- 172.21.15.123:0/2499205330 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c007030 con 0x7fb41c0cde90 2024-04-25T12:37:23.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c200540 con 0x7fb41c0cde90 2024-04-25T12:37:23.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7fb413400640 1 -- 172.21.15.123:0/2499205330 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c0071d0 con 0x7fb41c0cde90 2024-04-25T12:37:23.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c15b030 con 0x7fb41c0cde90 2024-04-25T12:37:23.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7fb413400640 1 -- 172.21.15.123:0/2499205330 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c007580 con 0x7fb41c0cde90 2024-04-25T12:37:23.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7f1a76000640 1 -- 172.21.15.123:0/2009249988 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a800dc030 con 0x7f1a7804b6e0 2024-04-25T12:37:23.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.537+0000 7f9c7b400640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9c5c0744e0 0x7f9c5c0769f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c1608e0 tx=0x7f9c68006b80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.538+0000 7faf17e00640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faef8074530 0x7faef8076a40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf180ccab0 tx=0x7faf08001a30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.539+0000 7f6572a00640 1 -- 172.21.15.123:0/285693978 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f656405f040 con 0x7f657c0cbb80 2024-04-25T12:37:23.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.539+0000 7f8ddb400640 1 -- 172.21.15.123:0/706970335 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8de80a0480 con 0x7f8de00e2180 2024-04-25T12:37:23.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.539+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c06ef60 con 0x7fb41c0cde90 2024-04-25T12:37:23.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.540+0000 7f8ddb400640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8dc80744b0 0x7f8dc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.541+0000 7f8ddb400640 1 -- 172.21.15.123:0/706970335 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f8de8112370 con 0x7f8de00e2180 2024-04-25T12:37:23.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.541+0000 7f9c73e00640 1 -- 172.21.15.123:0/3586213781 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c6c080790 con 0x7f9c7c0c4240 2024-04-25T12:37:23.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.541+0000 7f8de7400640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8dc80744b0 0x7f8dc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.542+0000 7f8de7400640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8dc80744b0 0x7f8dc80769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc033000 tx=0x7f8ddc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.543+0000 7f8ddb400640 1 -- 172.21.15.123:0/706970335 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8de00e2f90 con 0x7f8de00e2180 2024-04-25T12:37:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.545+0000 7fb413400640 1 -- 172.21.15.123:0/2499205330 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb40c01a020 con 0x7fb41c0cde90 2024-04-25T12:37:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.547+0000 7fb413400640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3fc0744b0 0x7fb3fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.548+0000 7fb413400640 1 -- 172.21.15.123:0/2499205330 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fb40c095510 con 0x7fb41c0cde90 2024-04-25T12:37:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.548+0000 7fb41b400640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3fc0744b0 0x7fb3fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.549+0000 7fb413400640 1 -- 172.21.15.123:0/2499205330 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb41c06ef60 con 0x7fb41c0cde90 2024-04-25T12:37:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.549+0000 7fb41b400640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3fc0744b0 0x7fb3fc0769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb4140012e0 tx=0x7fb414009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.551+0000 7fde19600640 1 -- 172.21.15.123:0/2645476674 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde1005f680 con 0x7fde04005150 2024-04-25T12:37:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.562+0000 7faf14c00640 1 -- 172.21.15.123:0/2968766112 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf04021020 con 0x7faf180c2870 2024-04-25T12:37:23.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.967+0000 7f1a7f200640 1 -- 172.21.15.123:0/2009249988 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1a7804c240 con 0x7f1a7804b6e0 2024-04-25T12:37:23.970 INFO:teuthology.orchestra.run.smithi123.stdout:60129542186 2024-04-25T12:37:23.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.968+0000 7f1a76000640 1 -- 172.21.15.123:0/2009249988 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1a800e0330 con 0x7f1a7804b6e0 2024-04-25T12:37:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 -- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1a600744b0 msgr2=0x7f1a600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1a600744b0 0x7f1a600769c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009870 tx=0x7f1a7002e000 comp rx=0 tx=0).stop 2024-04-25T12:37:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 -- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a7804b6e0 msgr2=0x7f1a780347e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a7804b6e0 0x7f1a780347e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1a80066010 tx=0x7f1a80069cf0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 -- 172.21.15.123:0/2009249988 shutdown_connections 2024-04-25T12:37:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1a600744b0 0x7f1a600769c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a7804b6e0 0x7f1a780347e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1a780457c0 0x7f1a780d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 --2- 172.21.15.123:0/2009249988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a78007610 0x7f1a780e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.972+0000 7f1a6b400640 1 -- 172.21.15.123:0/2009249988 >> 172.21.15.123:0/2009249988 conn(0x7f1a780503a0 msgr2=0x7f1a780db0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.973+0000 7f1a6b400640 1 -- 172.21.15.123:0/2009249988 shutdown_connections 2024-04-25T12:37:23.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.974+0000 7f1a6b400640 1 -- 172.21.15.123:0/2009249988 wait complete. 2024-04-25T12:37:23.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.977+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f672404db40 con 0x7f67240334e0 2024-04-25T12:37:23.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.978+0000 7f671f400640 1 -- 172.21.15.123:0/2220150994 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6720080720 con 0x7f67240334e0 2024-04-25T12:37:23.979 INFO:teuthology.orchestra.run.smithi123.stdout:64424509481 2024-04-25T12:37:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.981+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f670c0745b0 msgr2=0x7f670c076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.981+0000 7f6731000640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f670c0745b0 0x7f670c076ac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f672c06b050 tx=0x7f672c0696d0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.981+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 msgr2=0x7f67240db790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.981+0000 7f6731000640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f67240db790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67200071d0 tx=0x7f67200046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.982+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 shutdown_connections 2024-04-25T12:37:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.982+0000 7f6731000640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f670c0745b0 0x7f670c076ac0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.982+0000 7f6731000640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67240d56f0 0x7f67240cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.982+0000 7f6731000640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6724045e70 0x7f67240d63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.982+0000 7f6731000640 1 --2- 172.21.15.123:0/2220150994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67240334e0 0x7f67240db790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.982+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 >> 172.21.15.123:0/2220150994 conn(0x7f67240503b0 msgr2=0x7f67240ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.982+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9c7c0c3a60 con 0x7f9c7c0c4240 2024-04-25T12:37:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.983+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 shutdown_connections 2024-04-25T12:37:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.983+0000 7f6731000640 1 -- 172.21.15.123:0/2220150994 wait complete. 2024-04-25T12:37:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.983+0000 7f9c73e00640 1 -- 172.21.15.123:0/3586213781 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c6c080090 con 0x7f9c7c0c4240 2024-04-25T12:37:23.985 INFO:teuthology.orchestra.run.smithi123.stdout:60129542186 2024-04-25T12:37:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.987+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9c5c0744e0 msgr2=0x7f9c5c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.987+0000 7f9c82c00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9c5c0744e0 0x7f9c5c0769f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c1608e0 tx=0x7f9c68006b80 comp rx=0 tx=0).stop 2024-04-25T12:37:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.987+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0c4240 msgr2=0x7f9c7c1fa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.987+0000 7f9c82c00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0c4240 0x7f9c7c1fa370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009870 tx=0x7f9c6c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6554005850 con 0x7f657c0cbb80 2024-04-25T12:37:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 shutdown_connections 2024-04-25T12:37:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9c5c0744e0 0x7f9c5c0769f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c7c15a730 0x7f9c7c14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7c0c90d0 0x7f9c7c14e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 --2- 172.21.15.123:0/3586213781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0c4240 0x7f9c7c1fa370 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 >> 172.21.15.123:0/3586213781 conn(0x7f9c7c06c750 msgr2=0x7f9c7c075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 shutdown_connections 2024-04-25T12:37:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.988+0000 7f9c82c00640 1 -- 172.21.15.123:0/3586213781 wait complete. 2024-04-25T12:37:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.989+0000 7f6572a00640 1 -- 172.21.15.123:0/285693978 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6564063340 con 0x7f657c0cbb80 2024-04-25T12:37:23.989 INFO:teuthology.orchestra.run.smithi123.stdout:64424509481 2024-04-25T12:37:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.990+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb41c1609b0 con 0x7fb41c0cde90 2024-04-25T12:37:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.990+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8de004c6a0 con 0x7f8de00e2180 2024-04-25T12:37:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.989+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fde14047720 con 0x7fde04005150 2024-04-25T12:37:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.990+0000 7fde19600640 1 -- 172.21.15.123:0/2645476674 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fde10063980 con 0x7fde04005150 2024-04-25T12:37:23.990 INFO:teuthology.orchestra.run.smithi123.stdout:60129542186 2024-04-25T12:37:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.991+0000 7fb413400640 1 -- 172.21.15.123:0/2499205330 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb41c1609b0 con 0x7fb41c0cde90 2024-04-25T12:37:23.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.991+0000 7f8ddb400640 1 -- 172.21.15.123:0/706970335 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8de8092070 con 0x7f8de00e2180 2024-04-25T12:37:23.991 INFO:teuthology.orchestra.run.smithi123.stdout:201863462929 2024-04-25T12:37:23.991 INFO:teuthology.orchestra.run.smithi123.stdout:60129542185 2024-04-25T12:37:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.993+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f65580746e0 msgr2=0x7f6558076bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.993+0000 7f6581800640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f65580746e0 0x7f6558076bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f656c034000 tx=0x7f656c02e000 comp rx=0 tx=0).stop 2024-04-25T12:37:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.993+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0cbb80 msgr2=0x7f657c14d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.993+0000 7f6581800640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0cbb80 0x7f657c14d000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f656400e870 tx=0x7f656400ed30 comp rx=0 tx=0).stop 2024-04-25T12:37:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 shutdown_connections 2024-04-25T12:37:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7f6581800640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f65580746e0 0x7f6558076bf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7f6581800640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f657c06cf30 0x7f657c145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7f6581800640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f657c0ccb90 0x7f657c1e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7f6581800640 1 --2- 172.21.15.123:0/285693978 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f657c0cbb80 0x7f657c14d000 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 >> 172.21.15.123:0/285693978 conn(0x7f657c06e960 msgr2=0x7f657c0cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fddfc0744b0 msgr2=0x7fddfc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7fde22a00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fddfc0744b0 0x7fddfc0769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fde08001820 tx=0x7fde08009040 comp rx=0 tx=0).stop 2024-04-25T12:37:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04005150 msgr2=0x7fde140e0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.994+0000 7fde22a00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04005150 0x7fde140e0020 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b550 tx=0x7fde1000ba10 comp rx=0 tx=0).stop 2024-04-25T12:37:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 shutdown_connections 2024-04-25T12:37:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 shutdown_connections 2024-04-25T12:37:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f6581800640 1 -- 172.21.15.123:0/285693978 wait complete. 2024-04-25T12:37:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8dc80744b0 msgr2=0x7f8dc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8dc80744b0 0x7f8dc80769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc033000 tx=0x7f8ddc02e000 comp rx=0 tx=0).stop 2024-04-25T12:37:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de00e2180 msgr2=0x7f8de00cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de00e2180 0x7f8de00cdb90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8de8065b40 tx=0x7f8de8069cf0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fde22a00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fddfc0744b0 0x7fddfc0769c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 shutdown_connections 2024-04-25T12:37:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8dc80744b0 0x7f8dc80769c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de00e2180 0x7f8de00cdb90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8de0175450 0x7f8de00465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 --2- 172.21.15.123:0/706970335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dd00039b0 0x7f8de00d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 >> 172.21.15.123:0/706970335 conn(0x7f8de00503b0 msgr2=0x7f8de004edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fde22a00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fde1417cd30 0x7fde1417d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 shutdown_connections 2024-04-25T12:37:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7f8ded000640 1 -- 172.21.15.123:0/706970335 wait complete. 2024-04-25T12:37:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3fc0744b0 msgr2=0x7fb3fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3fc0744b0 0x7fb3fc0769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb4140012e0 tx=0x7fb414009040 comp rx=0 tx=0).stop 2024-04-25T12:37:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fde22a00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1404f620 0x7fde140ea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 msgr2=0x7fb41c15ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c15ce40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb41c069630 tx=0x7fb40c00c870 comp rx=0 tx=0).stop 2024-04-25T12:37:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fde22a00640 1 --2- 172.21.15.123:0/2645476674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04005150 0x7fde140e0020 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 >> 172.21.15.123:0/2645476674 conn(0x7fde1404df90 msgr2=0x7fde140d9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.995+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 shutdown_connections 2024-04-25T12:37:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fde22a00640 1 -- 172.21.15.123:0/2645476674 wait complete. 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 shutdown_connections 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fb3fc0744b0 0x7fb3fc0769c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb41c156ad0 0x7fb41c203260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb41c0cde90 0x7fb41c15ce40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 --2- 172.21.15.123:0/2499205330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0c6970 0x7fb41c200000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.996+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 >> 172.21.15.123:0/2499205330 conn(0x7fb41c0d1a30 msgr2=0x7fb41c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.997+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 shutdown_connections 2024-04-25T12:37:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:23.997+0000 7fb422800640 1 -- 172.21.15.123:0/2499205330 wait complete. 2024-04-25T12:37:24.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.001+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7faf180cd570 con 0x7faf180c2870 2024-04-25T12:37:24.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.002+0000 7faf14c00640 1 -- 172.21.15.123:0/2968766112 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf0405ec80 con 0x7faf180c2870 2024-04-25T12:37:24.002 INFO:teuthology.orchestra.run.smithi123.stdout:60129542186 2024-04-25T12:37:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.006+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faef8074530 msgr2=0x7faef8076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.006+0000 7faf1f600640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faef8074530 0x7faef8076a40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf180ccab0 tx=0x7faf08001a30 comp rx=0 tx=0).stop 2024-04-25T12:37:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.006+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 msgr2=0x7faf181f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.006+0000 7faf1f600640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf181f83c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c3f0 tx=0x7faf0400c8b0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.007+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 shutdown_connections 2024-04-25T12:37:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.007+0000 7faf1f600640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7faef8074530 0x7faef8076a40 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf180ccab0 tx=0x7faf08001a30 comp rx=0 tx=0).stop 2024-04-25T12:37:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.007+0000 7faf1f600640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf180c90d0 0x7faf181faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.007+0000 7faf1f600640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf180c4240 0x7faf180c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.007+0000 7faf1f600640 1 --2- 172.21.15.123:0/2968766112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf180c2870 0x7faf181f83c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.007+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 >> 172.21.15.123:0/2968766112 conn(0x7faf1806c750 msgr2=0x7faf180ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:24.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.008+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 shutdown_connections 2024-04-25T12:37:24.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:24.008+0000 7faf1f600640 1 -- 172.21.15.123:0/2968766112 wait complete. 2024-04-25T12:37:24.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.2 2024-04-25T12:37:24.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509481 for osd.0 2024-04-25T12:37:24.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.6 2024-04-25T12:37:24.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.7 2024-04-25T12:37:24.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 201863462930 got 201863462929 for osd.5 2024-04-25T12:37:24.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509481 for osd.3 2024-04-25T12:37:24.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.1 2024-04-25T12:37:24.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.4 2024-04-25T12:37:25.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T12:37:25.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T12:37:25.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T12:37:25.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T12:37:25.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T12:37:25.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T12:37:25.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T12:37:25.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T12:37:25.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.415+0000 7f1e78c00640 1 Processor -- start 2024-04-25T12:37:25.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.415+0000 7f1e78c00640 1 -- start start 2024-04-25T12:37:25.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.419+0000 7f1e78c00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1e74071520 0x7f1e740c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.419+0000 7f1e78c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e7406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.419+0000 7f1e78c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1e7406cab0 0x7f1e740cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.419+0000 7f1e78c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1e741583e0 con 0x7f1e7406cab0 2024-04-25T12:37:25.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.419+0000 7f1e78c00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1e741586a0 con 0x7f1e74071520 2024-04-25T12:37:25.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.419+0000 7f1e78c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1e74158980 con 0x7f1e7413e0d0 2024-04-25T12:37:25.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.420+0000 7f1e71400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e7406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.420+0000 7f1e71400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e7406c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:57000/0 (socket says 172.21.15.123:57000) 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.420+0000 7f1e71400640 1 -- 172.21.15.123:0/3688615846 learned_addr learned my addr 172.21.15.123:0/3688615846 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.420+0000 7f1e71e00640 1 --2- 172.21.15.123:0/3688615846 >> v2:172.21.15.153:3300/0 conn(0x7f1e74071520 0x7f1e740c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.420+0000 7f1e72800640 1 --2- 172.21.15.123:0/3688615846 >> v2:172.21.15.123:3300/0 conn(0x7f1e7406cab0 0x7f1e740cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.421+0000 7f1e72800640 1 -- 172.21.15.123:0/3688615846 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 msgr2=0x7f1e7406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.421+0000 7f1e72800640 1 --2- 172.21.15.123:0/3688615846 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e7406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.421+0000 7f1e72800640 1 -- 172.21.15.123:0/3688615846 >> v2:172.21.15.153:3300/0 conn(0x7f1e74071520 msgr2=0x7f1e740c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.421+0000 7f1e72800640 1 --2- 172.21.15.123:0/3688615846 >> v2:172.21.15.153:3300/0 conn(0x7f1e74071520 0x7f1e740c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.421+0000 7f1e72800640 1 -- 172.21.15.123:0/3688615846 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e74158c60 con 0x7f1e7406cab0 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.423+0000 7f1e72800640 1 --2- 172.21.15.123:0/3688615846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e740cc850 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e74072290 tx=0x7f1e6402a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5598869dcafb2212 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.424+0000 7f1e70a00640 1 -- 172.21.15.123:0/3688615846 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64037070 con 0x7f1e7406cab0 2024-04-25T12:37:25.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.424+0000 7f1e70a00640 1 -- 172.21.15.123:0/3688615846 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e64032440 con 0x7f1e7406cab0 2024-04-25T12:37:25.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.425+0000 7f1e78c00640 1 -- 172.21.15.123:0/3688615846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 msgr2=0x7f1e740cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.425+0000 7f1e70a00640 1 -- 172.21.15.123:0/3688615846 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64037070 con 0x7f1e7406cab0 2024-04-25T12:37:25.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.425+0000 7f1e78c00640 1 --2- 172.21.15.123:0/3688615846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e740cc850 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e74072290 tx=0x7f1e6402a3b0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.426+0000 7f1e78c00640 1 -- 172.21.15.123:0/3688615846 shutdown_connections 2024-04-25T12:37:25.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.426+0000 7f1e78c00640 1 --2- 172.21.15.123:0/3688615846 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e7406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.426+0000 7f1e78c00640 1 --2- 172.21.15.123:0/3688615846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e740cc850 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.426+0000 7f1e78c00640 1 --2- 172.21.15.123:0/3688615846 >> v2:172.21.15.153:3300/0 conn(0x7f1e74071520 0x7f1e740c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.426+0000 7f1e78c00640 1 -- 172.21.15.123:0/3688615846 >> 172.21.15.123:0/3688615846 conn(0x7f1e740710e0 msgr2=0x7f1e740cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.426+0000 7f1e78c00640 1 -- 172.21.15.123:0/3688615846 shutdown_connections 2024-04-25T12:37:25.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.426+0000 7f1e78c00640 1 -- 172.21.15.123:0/3688615846 wait complete. 2024-04-25T12:37:25.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.430+0000 7f1e78c00640 1 Processor -- start 2024-04-25T12:37:25.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.430+0000 7f1e78c00640 1 -- start start 2024-04-25T12:37:25.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.433+0000 7f1e78c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e7415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.434+0000 7f1e71e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e7415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.434+0000 7f1e71e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e7415b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41244/0 (socket says 172.21.15.123:41244) 2024-04-25T12:37:25.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.435+0000 7f1e71e00640 1 -- 172.21.15.123:0/1081728530 learned_addr learned my addr 172.21.15.123:0/1081728530 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.435+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e74071520 0x7f1e7413bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.435+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e74074130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.435+0000 7f1e71400640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e74071520 0x7f1e7413bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.436+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e740ccd90 con 0x7f1e7406cab0 2024-04-25T12:37:25.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.437+0000 7f1e72800640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e74074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.440+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7413e4e0 con 0x7f1e74071520 2024-04-25T12:37:25.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.440+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1e740ccf10 con 0x7f1e7413e0d0 2024-04-25T12:37:25.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.441+0000 7f1e72800640 1 -- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e74071520 msgr2=0x7f1e7413bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.442+0000 7f1e72800640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e74071520 0x7f1e7413bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.442+0000 7f1e72800640 1 -- 172.21.15.123:0/1081728530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 msgr2=0x7f1e7415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.442+0000 7f1e72800640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e7415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.443+0000 7f1e72800640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7415b930 con 0x7f1e7413e0d0 2024-04-25T12:37:25.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.443+0000 7f1e71400640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e74071520 0x7f1e7413bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:25.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.444+0000 7f1e72800640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e74074130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e740ce520 tx=0x7f1e640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.446+0000 7f1e6a000640 1 -- 172.21.15.123:0/1081728530 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64037050 con 0x7f1e7413e0d0 2024-04-25T12:37:25.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.446+0000 7f1e6a000640 1 -- 172.21.15.123:0/1081728530 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e64038040 con 0x7f1e7413e0d0 2024-04-25T12:37:25.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.447+0000 7f1e6a000640 1 -- 172.21.15.123:0/1081728530 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64032810 con 0x7f1e7413e0d0 2024-04-25T12:37:25.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.447+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7415a930 con 0x7f1e7413e0d0 2024-04-25T12:37:25.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.447+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7413cbb0 con 0x7f1e7413e0d0 2024-04-25T12:37:25.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.449+0000 7f1e5f400640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7415bb00 con 0x7f1e7413e0d0 2024-04-25T12:37:25.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.452+0000 7f1e6a000640 1 -- 172.21.15.123:0/1081728530 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e64045020 con 0x7f1e7413e0d0 2024-04-25T12:37:25.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.454+0000 7f1e6a000640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1e54074580 0x7f1e54076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.455+0000 7f1e6a000640 1 -- 172.21.15.123:0/1081728530 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f1e640b5f80 con 0x7f1e7413e0d0 2024-04-25T12:37:25.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.455+0000 7f1e71e00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1e54074580 0x7f1e54076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.456+0000 7f1e6a000640 1 -- 172.21.15.123:0/1081728530 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e64080250 con 0x7f1e7413e0d0 2024-04-25T12:37:25.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.457+0000 7f1e71e00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1e54074580 0x7f1e54076a90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0094c0 tx=0x7f1e6c00d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.468+0000 7f8c45a00640 1 Processor -- start 2024-04-25T12:37:25.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.468+0000 7f8c45a00640 1 -- start start 2024-04-25T12:37:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.471+0000 7f8c45a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8c4015cd10 0x7f8c4015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.472+0000 7f8c45a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c40074c30 con 0x7f8c40160620 2024-04-25T12:37:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.472+0000 7f8c3f400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8c4015cd10 0x7f8c4015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.472+0000 7f8c3f400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8c4015cd10 0x7f8c4015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48494/0 (socket says 172.21.15.123:48494) 2024-04-25T12:37:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.472+0000 7f8c3f400640 1 -- 172.21.15.123:0/2502032402 learned_addr learned my addr 172.21.15.123:0/2502032402 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.473+0000 7f8c45a00640 1 -- 172.21.15.123:0/2502032402 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400c8d40 con 0x7f8c4006c850 2024-04-25T12:37:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.473+0000 7f8c45a00640 1 -- 172.21.15.123:0/2502032402 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8c4015fbd0 con 0x7f8c4015cd10 2024-04-25T12:37:25.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.473+0000 7f1c97e00640 1 Processor -- start 2024-04-25T12:37:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.473+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920738821 0 0) 0x7f8c400c8d40 con 0x7f8c4006c850 2024-04-25T12:37:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.473+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c24003520 con 0x7f8c4006c850 2024-04-25T12:37:25.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.473+0000 7f1c97e00640 1 -- start start 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.473+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2811721064 0 0) 0x7f8c24003520 con 0x7f8c4006c850 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 >> v2:172.21.15.153:3300/0 conn(0x7f8c4015cd10 msgr2=0x7f8c4015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 --2- 172.21.15.123:0/2502032402 >> v2:172.21.15.153:3300/0 conn(0x7f8c4015cd10 0x7f8c4015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 >> v1:172.21.15.123:6789/0 conn(0x7f8c40160620 legacy=0x7f8c400c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3f400640 1 --2- 172.21.15.123:0/2502032402 >> v2:172.21.15.153:3300/0 conn(0x7f8c4015cd10 0x7f8c4015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c40073b00 con 0x7f8c4006c850 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f8c2c001670 con 0x7f8c4006c850 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 >> v1:172.21.15.153:6789/0 conn(0x7f8c4006c850 legacy=0x7f8c40166f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c240039b0 0x7f8c24003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c24004800 0x7f8c24004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c24005270 0x7f8c2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2400c1a0 con 0x7f8c24004800 2024-04-25T12:37:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2400c350 con 0x7f8c24005270 2024-04-25T12:37:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.474+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2400c550 con 0x7f8c240039b0 2024-04-25T12:37:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.475+0000 7f8c45000640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c24005270 0x7f8c2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.475+0000 7f8c3f400640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c24004800 0x7f8c24004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.476+0000 7f8c3fe00640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c240039b0 0x7f8c24003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.476+0000 7f1c97e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c9015c380 0x7f1c900ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f8c45000640 1 -- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c240039b0 msgr2=0x7f8c24003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f1c97e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9015ca80 con 0x7f1c900ceb80 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f1c97e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9015cd60 con 0x7f1c900c67c0 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f1c97e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1c900b4600 con 0x7f1c9015c380 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f1c95e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1c900ceb80 0x7f1c90166d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58620/0 (socket says 172.21.15.123:58620) 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f1c95e00640 1 -- 172.21.15.123:0/1103793300 learned_addr learned my addr 172.21.15.123:0/1103793300 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f8c45000640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c240039b0 0x7f8c24003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f8c45000640 1 -- 172.21.15.123:0/2502032402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c24004800 msgr2=0x7f8c24004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f8c45000640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c24004800 0x7f8c24004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f8c45000640 1 -- 172.21.15.123:0/2502032402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c24003520 con 0x7f8c24005270 2024-04-25T12:37:25.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f8c3fe00640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c240039b0 0x7f8c24003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:25.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.477+0000 7f8c3f400640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c24004800 0x7f8c24004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:25.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.479+0000 7f8c45000640 1 -- 172.21.15.123:0/2502032402 reap_dead start 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.478+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2039196828 0 0) 0x7f1c9015ca80 con 0x7f1c900ceb80 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.478+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c78003520 con 0x7f1c900ceb80 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.478+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 912992444 0 0) 0x7f1c78003520 con 0x7f1c900ceb80 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.478+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 >> v2:172.21.15.123:3300/0 conn(0x7f1c9015c380 msgr2=0x7f1c900ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.479+0000 7f1c94a00640 1 --2- 172.21.15.123:0/1103793300 >> v2:172.21.15.123:3300/0 conn(0x7f1c9015c380 0x7f1c900ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.479+0000 7f1c8fe00640 1 --2- 172.21.15.123:0/1103793300 >> v2:172.21.15.123:3300/0 conn(0x7f1c9015c380 0x7f1c900ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.479+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 >> v1:172.21.15.153:6789/0 conn(0x7f1c900c67c0 legacy=0x7f1c901602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.479+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9015d040 con 0x7f1c900ceb80 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f1c80001670 con 0x7f1c900ceb80 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 >> v1:172.21.15.123:6789/0 conn(0x7f1c900ceb80 legacy=0x7f1c90166d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c780039b0 0x7f1c78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c78004800 0x7f1c78004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c78005350 0x7f1c7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900b4600 con 0x7f1c78004800 2024-04-25T12:37:25.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7800c250 con 0x7f1c780039b0 2024-04-25T12:37:25.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7800c450 con 0x7f1c78005350 2024-04-25T12:37:25.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.480+0000 7f1c94a00640 1 --2- 172.21.15.123:0/1103793300 >> v2:172.21.15.123:3300/0 conn(0x7f1c9015c380 0x7f1c900ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:25.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.481+0000 7f1c95400640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c780039b0 0x7f1c78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.481+0000 7f1c95e00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c78005350 0x7f1c7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.481+0000 7f1c94a00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c78004800 0x7f1c78004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.481+0000 7f45d1400640 1 Processor -- start 2024-04-25T12:37:25.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f45d1400640 1 -- start start 2024-04-25T12:37:25.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f8c45000640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c24005270 0x7f8c2400b9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c160 tx=0x7f8c24005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe6aef1c434ca54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f1c95400640 1 -- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c78005350 msgr2=0x7f1c7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f1c95400640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c78005350 0x7f1c7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f1c95400640 1 -- 172.21.15.123:0/1103793300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c78004800 msgr2=0x7f1c78004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c036070 con 0x7f8c24005270 2024-04-25T12:37:25.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2c007270 con 0x7f8c24005270 2024-04-25T12:37:25.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.482+0000 7f8c3ea00640 1 -- 172.21.15.123:0/2502032402 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38007d30 con 0x7f8c24005270 2024-04-25T12:37:25.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.483+0000 7f1c95400640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c78004800 0x7f1c78004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.483+0000 7f1c95400640 1 -- 172.21.15.123:0/1103793300 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9015d040 con 0x7f1c780039b0 2024-04-25T12:37:25.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.483+0000 7f1c94a00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c78004800 0x7f1c78004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:25.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.484+0000 7f45d1400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c404fb60 0x7f45c40e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.484+0000 7f45d1400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c40e0f90 con 0x7f45c4037840 2024-04-25T12:37:25.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.484+0000 7f45d1400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c40e1270 con 0x7f45c4037d90 2024-04-25T12:37:25.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.484+0000 7f45d1400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f45c40e1550 con 0x7f45c404fb60 2024-04-25T12:37:25.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.484+0000 7f8c45a00640 1 -- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c24005270 msgr2=0x7f8c2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.484+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c24005270 0x7f8c2400b9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c160 tx=0x7f8c24005830 comp rx=0 tx=0).stop 2024-04-25T12:37:25.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f8c45a00640 1 -- 172.21.15.123:0/2502032402 shutdown_connections 2024-04-25T12:37:25.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.484+0000 7f1c95e00640 1 -- 172.21.15.123:0/1103793300 reap_dead start 2024-04-25T12:37:25.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f45d0a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c404fb60 0x7f45c40e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2502032402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c24005270 0x7f8c2400b9c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f8c45a00640 1 -- 172.21.15.123:0/2502032402 >> 172.21.15.123:0/2502032402 conn(0x7f8c400cd600 msgr2=0x7f8c400cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f45cb400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f45c4037840 0x7f45c4052570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58636/0 (socket says 172.21.15.123:58636) 2024-04-25T12:37:25.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f45cb400640 1 -- 172.21.15.123:0/3036294503 learned_addr learned my addr 172.21.15.123:0/3036294503 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:25.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298133173 0 0) 0x7f45c40e0f90 con 0x7f45c4037840 2024-04-25T12:37:25.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45b4003520 con 0x7f45c4037840 2024-04-25T12:37:25.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f8c45a00640 1 -- 172.21.15.123:0/2502032402 shutdown_connections 2024-04-25T12:37:25.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609347312 0 0) 0x7f45c40e1270 con 0x7f45c4037d90 2024-04-25T12:37:25.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45c40e0f90 con 0x7f45c4037d90 2024-04-25T12:37:25.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f1c95400640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c780039b0 0x7f1c78003de0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c84001f60 tx=0x7f1c78005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=953b4c51fb346a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.485+0000 7f8c45a00640 1 -- 172.21.15.123:0/2502032402 wait complete. 2024-04-25T12:37:25.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f45d0a00640 1 -- 172.21.15.123:0/3036294503 >> v1:172.21.15.153:6789/0 conn(0x7f45c4037d90 legacy=0x7f45c40ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 210083201 0 0) 0x7f45c40e0f90 con 0x7f45c4037d90 2024-04-25T12:37:25.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f45d0a00640 1 -- 172.21.15.123:0/3036294503 >> v1:172.21.15.123:6789/0 conn(0x7f45c4037840 legacy=0x7f45c4052570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f45d0a00640 1 -- 172.21.15.123:0/3036294503 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c40dd530 con 0x7f45c404fb60 2024-04-25T12:37:25.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.486+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84001b30 con 0x7f1c780039b0 2024-04-25T12:37:25.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.487+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c84031620 con 0x7f1c780039b0 2024-04-25T12:37:25.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.487+0000 7f1c8fe00640 1 -- 172.21.15.123:0/1103793300 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84031a50 con 0x7f1c780039b0 2024-04-25T12:37:25.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.487+0000 7f45d0a00640 1 --2- 172.21.15.123:0/3036294503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c404fb60 0x7f45c40e0900 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45cc02e0e0 tx=0x7f45cc0a77f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=376608fd3a190de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.487+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0b3070 con 0x7f45c404fb60 2024-04-25T12:37:25.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.487+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc0ae400 con 0x7f45c404fb60 2024-04-25T12:37:25.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.487+0000 7f45caa00640 1 -- 172.21.15.123:0/3036294503 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b80015a0 con 0x7f45c404fb60 2024-04-25T12:37:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.488+0000 7f1c97e00640 1 -- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c780039b0 msgr2=0x7f1c78003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.488+0000 7f1c97e00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c780039b0 0x7f1c78003de0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c84001f60 tx=0x7f1c78005830 comp rx=0 tx=0).stop 2024-04-25T12:37:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.488+0000 7f7d0d400640 1 Processor -- start 2024-04-25T12:37:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f45d1400640 1 -- 172.21.15.123:0/3036294503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c404fb60 msgr2=0x7f45c40e0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f45d1400640 1 --2- 172.21.15.123:0/3036294503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c404fb60 0x7f45c40e0900 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45cc02e0e0 tx=0x7f45cc0a77f0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f1c97e00640 1 -- 172.21.15.123:0/1103793300 shutdown_connections 2024-04-25T12:37:25.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f1c97e00640 1 --2- 172.21.15.123:0/1103793300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c780039b0 0x7f1c78003de0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f1c97e00640 1 -- 172.21.15.123:0/1103793300 >> 172.21.15.123:0/1103793300 conn(0x7f1c9008d840 msgr2=0x7f1c9008dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.488+0000 7f7d0d400640 1 -- start start 2024-04-25T12:37:25.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f45d1400640 1 -- 172.21.15.123:0/3036294503 shutdown_connections 2024-04-25T12:37:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f45d1400640 1 --2- 172.21.15.123:0/3036294503 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c404fb60 0x7f45c40e0900 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f45d1400640 1 -- 172.21.15.123:0/3036294503 >> 172.21.15.123:0/3036294503 conn(0x7f45c400e2e0 msgr2=0x7f45c400e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f1c97e00640 1 -- 172.21.15.123:0/1103793300 shutdown_connections 2024-04-25T12:37:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f45d1400640 1 -- 172.21.15.123:0/3036294503 shutdown_connections 2024-04-25T12:37:25.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f45d1400640 1 -- 172.21.15.123:0/3036294503 wait complete. 2024-04-25T12:37:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.490+0000 7f8c45a00640 1 Processor -- start 2024-04-25T12:37:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.489+0000 7f1c97e00640 1 -- 172.21.15.123:0/1103793300 wait complete. 2024-04-25T12:37:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.491+0000 7f2ca7400640 1 Processor -- start 2024-04-25T12:37:25.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.491+0000 7f8c45a00640 1 -- start start 2024-04-25T12:37:25.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.493+0000 7f45d1400640 1 Processor -- start 2024-04-25T12:37:25.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.493+0000 7f45d1400640 1 -- start start 2024-04-25T12:37:25.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.491+0000 7f2ca7400640 1 -- start start 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.492+0000 7f7d0d400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d080c90d0 0x7f7d080c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.492+0000 7f7d0d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d080c4240 0x7f7d080cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.492+0000 7f7d0d400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d0815a730 0x7f7d0815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.492+0000 7f7d0d400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0815b080 con 0x7f7d080c90d0 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.492+0000 7f7d0d400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7d080c4810 con 0x7f7d0815a730 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.492+0000 7f7d0d400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080c83f0 con 0x7f7d080c4240 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.493+0000 7f7d06800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d080c90d0 0x7f7d080c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.493+0000 7f7d06800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d080c90d0 0x7f7d080c2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41284/0 (socket says 172.21.15.123:41284) 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.493+0000 7f7d06800640 1 -- 172.21.15.123:0/3451846770 learned_addr learned my addr 172.21.15.123:0/3451846770 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.493+0000 7f7d07200640 1 --2- 172.21.15.123:0/3451846770 >> v2:172.21.15.153:3300/0 conn(0x7f7d0815a730 0x7f7d0815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f7d06800640 1 -- 172.21.15.123:0/3451846770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d080c4240 msgr2=0x7f7d080cdc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T12:37:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f8c45a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c24005270 0x7f8c4014c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f8c45a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c401516b0 0x7f8c40153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f8c45a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4015f5d0 0x7f8c40154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f8c45a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400738a0 con 0x7f8c401516b0 2024-04-25T12:37:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f8c45a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4015c880 con 0x7f8c4015f5d0 2024-04-25T12:37:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f8c45a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8c400740c0 con 0x7f8c24005270 2024-04-25T12:37:25.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.493+0000 7f1c97e00640 1 Processor -- start 2024-04-25T12:37:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f1c97e00640 1 -- start start 2024-04-25T12:37:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f7d06800640 1 --2- 172.21.15.123:0/3451846770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d080c4240 0x7f7d080cdc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f2ca7400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ca006d440 0x7f2ca0156670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f2ca7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2ca00c72c0 0x7f2ca00c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f2ca7400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca006d850 con 0x7f2ca01606e0 2024-04-25T12:37:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f7d06800640 1 -- 172.21.15.123:0/3451846770 >> v2:172.21.15.153:3300/0 conn(0x7f7d0815a730 msgr2=0x7f7d0815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f7d06800640 1 --2- 172.21.15.123:0/3451846770 >> v2:172.21.15.153:3300/0 conn(0x7f7d0815a730 0x7f7d0815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.494+0000 7f7d06800640 1 -- 172.21.15.123:0/3451846770 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d080c86d0 con 0x7f7d080c90d0 2024-04-25T12:37:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f2ca7400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca0160af0 con 0x7f2ca00c72c0 2024-04-25T12:37:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f2ca7400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca015fc40 con 0x7f2ca006d440 2024-04-25T12:37:25.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f8c3f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c401516b0 0x7f8c40153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f2ca4a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2ca01606e0 0x7f2ca015d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58652/0 (socket says 172.21.15.123:58652) 2024-04-25T12:37:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f2ca4a00640 1 -- 172.21.15.123:0/848339898 learned_addr learned my addr 172.21.15.123:0/848339898 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f2c9fe00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ca006d440 0x7f2ca0156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f2ca5400640 1 --2- 172.21.15.123:0/848339898 >> v2:172.21.15.123:3300/0 conn(0x7f2ca00c72c0 0x7f2ca00c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f45d1400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c4037840 0x7f45c40e0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f45d1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45c4037d90 0x7f45c40516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f2c9fe00640 1 -- 172.21.15.123:0/848339898 >> v2:172.21.15.123:3300/0 conn(0x7f2ca00c72c0 msgr2=0x7f2ca00c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f8c3f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c401516b0 0x7f8c40153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41296/0 (socket says 172.21.15.123:41296) 2024-04-25T12:37:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f8c3f400640 1 -- 172.21.15.123:0/2428744537 learned_addr learned my addr 172.21.15.123:0/2428744537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.495+0000 7f45d1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c404fb60 0x7f45c404b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45d1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4052e00 con 0x7f45c404fb60 2024-04-25T12:37:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45d1400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4052c80 con 0x7f45c4037d90 2024-04-25T12:37:25.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45d1400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f45c404edd0 con 0x7f45c4037840 2024-04-25T12:37:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f8c3f400640 1 -- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c24005270 msgr2=0x7f8c4014c280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:37:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f8c3f400640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c24005270 0x7f8c4014c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f8c3f400640 1 -- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4015f5d0 msgr2=0x7f8c40154430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T12:37:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f8c3f400640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4015f5d0 0x7f8c40154430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f8c3f400640 1 -- 172.21.15.123:0/2428744537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c401557b0 con 0x7f8c401516b0 2024-04-25T12:37:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c4037840 0x7f45c40e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c4037840 0x7f45c40e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:57064/0 (socket says 172.21.15.123:57064) 2024-04-25T12:37:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45cbe00640 1 -- 172.21.15.123:0/3983743310 learned_addr learned my addr 172.21.15.123:0/3983743310 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45cb400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45c4037d90 0x7f45c40516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f45d0a00640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c404fb60 0x7f45c404b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f2c9fe00640 1 --2- 172.21.15.123:0/848339898 >> v2:172.21.15.123:3300/0 conn(0x7f2ca00c72c0 0x7f2ca00c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f2c9fe00640 1 -- 172.21.15.123:0/848339898 >> v1:172.21.15.123:6789/0 conn(0x7f2ca01606e0 legacy=0x7f2ca015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f2c9fe00640 1 -- 172.21.15.123:0/848339898 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca00cc920 con 0x7f2ca006d440 2024-04-25T12:37:25.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f2ca5400640 1 --2- 172.21.15.123:0/848339898 >> v2:172.21.15.123:3300/0 conn(0x7f2ca00c72c0 0x7f2ca00c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:25.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.496+0000 7f7d06800640 1 --2- 172.21.15.123:0/3451846770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d080c2870 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d0806e6e0 tx=0x7f7cf002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbccbdb40e73f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f7d05400640 1 -- 172.21.15.123:0/3451846770 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0037070 con 0x7f7d080c90d0 2024-04-25T12:37:25.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f45cb400640 1 -- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c4037840 msgr2=0x7f45c40e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f7d05400640 1 -- 172.21.15.123:0/3451846770 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf0032440 con 0x7f7d080c90d0 2024-04-25T12:37:25.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f7d05400640 1 -- 172.21.15.123:0/3451846770 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0032810 con 0x7f7d080c90d0 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c97e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c780039b0 0x7f1c90157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c97e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c90202930 0x7f1c90202d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90203280 0x7f1c901ff590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c97e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9015fd80 con 0x7f1c90203280 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c97e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9015d320 con 0x7f1c90202930 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c97e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9015c790 con 0x7f1c780039b0 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c94a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c90202930 0x7f1c90202d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c95400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c780039b0 0x7f1c90157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f1c94a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c90202930 0x7f1c90202d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48554/0 (socket says 172.21.15.123:48554) 2024-04-25T12:37:25.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f1c94a00640 1 -- 172.21.15.123:0/2253842611 learned_addr learned my addr 172.21.15.123:0/2253842611 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f45cb400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c4037840 0x7f45c40e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f45cb400640 1 -- 172.21.15.123:0/3983743310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c404fb60 msgr2=0x7f45c404b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f45cb400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c404fb60 0x7f45c404b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.497+0000 7f45cb400640 1 -- 172.21.15.123:0/3983743310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c40e0ac0 con 0x7f45c4037d90 2024-04-25T12:37:25.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f1c95e00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90203280 0x7f1c901ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f8c3f400640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c401516b0 0x7f8c40153ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800fe90 tx=0x7f8c3800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f45cb400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45c4037d90 0x7f45c40516d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c5b0 tx=0x7f45b800ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f1c95400640 1 -- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c90202930 msgr2=0x7f1c90202d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f1c95400640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c90202930 0x7f1c90202d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f1c95400640 1 -- 172.21.15.123:0/2253842611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90203280 msgr2=0x7f1c901ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f1c95400640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90203280 0x7f1c901ff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f45bfe00640 1 -- 172.21.15.123:0/3983743310 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8018070 con 0x7f45c4037d90 2024-04-25T12:37:25.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f45bfe00640 1 -- 172.21.15.123:0/3983743310 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b8007500 con 0x7f45c4037d90 2024-04-25T12:37:25.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f8c37e00640 1 -- 172.21.15.123:0/2428744537 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38018020 con 0x7f8c401516b0 2024-04-25T12:37:25.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f8c37e00640 1 -- 172.21.15.123:0/2428744537 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c38014470 con 0x7f8c401516b0 2024-04-25T12:37:25.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f8c37e00640 1 -- 172.21.15.123:0/2428744537 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38014820 con 0x7f8c401516b0 2024-04-25T12:37:25.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f1c95400640 1 -- 172.21.15.123:0/2253842611 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c901528d0 con 0x7f1c780039b0 2024-04-25T12:37:25.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.498+0000 7f45bfe00640 1 -- 172.21.15.123:0/3983743310 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8004a20 con 0x7f45c4037d90 2024-04-25T12:37:25.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c4185c50 con 0x7f45c4037d90 2024-04-25T12:37:25.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c40dfaa0 con 0x7f45c4037d90 2024-04-25T12:37:25.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f7d0d400640 1 -- 172.21.15.123:0/3451846770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 msgr2=0x7f7d080c2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f7d0d400640 1 --2- 172.21.15.123:0/3451846770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d080c2870 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d0806e6e0 tx=0x7f7cf002a2a0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f1c95400640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c780039b0 0x7f1c90157c60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c420 tx=0x7f1c840015b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f1c8d600640 1 -- 172.21.15.123:0/2253842611 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84045410 con 0x7f1c780039b0 2024-04-25T12:37:25.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c40155a50 con 0x7f8c401516b0 2024-04-25T12:37:25.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f2c9fe00640 1 --2- 172.21.15.123:0/848339898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca0156670 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ca008ed70 tx=0x7f2c8c02a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebcf7a3e558b1596 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.499+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c400ccf80 con 0x7f8c401516b0 2024-04-25T12:37:25.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f1c97e00640 1 -- 172.21.15.123:0/2253842611 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c90157050 con 0x7f1c780039b0 2024-04-25T12:37:25.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f1c8d600640 1 -- 172.21.15.123:0/2253842611 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c840455b0 con 0x7f1c780039b0 2024-04-25T12:37:25.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f1c8d600640 1 -- 172.21.15.123:0/2253842611 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84045410 con 0x7f1c780039b0 2024-04-25T12:37:25.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f1c97e00640 1 -- 172.21.15.123:0/2253842611 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c90159880 con 0x7f1c780039b0 2024-04-25T12:37:25.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f2c9f400640 1 -- 172.21.15.123:0/848339898 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c037070 con 0x7f2ca006d440 2024-04-25T12:37:25.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f2c9f400640 1 -- 172.21.15.123:0/848339898 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8c032440 con 0x7f2ca006d440 2024-04-25T12:37:25.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f2c9f400640 1 -- 172.21.15.123:0/848339898 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c032810 con 0x7f2ca006d440 2024-04-25T12:37:25.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f7d0d400640 1 -- 172.21.15.123:0/3451846770 shutdown_connections 2024-04-25T12:37:25.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f7d0d400640 1 --2- 172.21.15.123:0/3451846770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d080c4240 0x7f7d080cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f7d0d400640 1 --2- 172.21.15.123:0/3451846770 >> v2:172.21.15.153:3300/0 conn(0x7f7d0815a730 0x7f7d0815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f7d0d400640 1 --2- 172.21.15.123:0/3451846770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d080c2870 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.500+0000 7f7d0d400640 1 -- 172.21.15.123:0/3451846770 >> 172.21.15.123:0/3451846770 conn(0x7f7d0806c750 msgr2=0x7f7d0806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.501+0000 7f7d0d400640 1 -- 172.21.15.123:0/3451846770 shutdown_connections 2024-04-25T12:37:25.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.501+0000 7f7d0d400640 1 -- 172.21.15.123:0/3451846770 wait complete. 2024-04-25T12:37:25.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.502+0000 7f8915000640 1 Processor -- start 2024-04-25T12:37:25.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.501+0000 7f2ca7400640 1 -- 172.21.15.123:0/848339898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 msgr2=0x7f2ca0156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.501+0000 7f2ca7400640 1 --2- 172.21.15.123:0/848339898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca0156670 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ca008ed70 tx=0x7f2c8c02a1f0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.503+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c90132f30 con 0x7f1c780039b0 2024-04-25T12:37:25.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.503+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c18005290 con 0x7f8c401516b0 2024-04-25T12:37:25.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.503+0000 7f2ca7400640 1 -- 172.21.15.123:0/848339898 shutdown_connections 2024-04-25T12:37:25.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.503+0000 7f2ca7400640 1 --2- 172.21.15.123:0/848339898 >> v2:172.21.15.123:3300/0 conn(0x7f2ca00c72c0 0x7f2ca00c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.503+0000 7f2ca7400640 1 --2- 172.21.15.123:0/848339898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca0156670 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.503+0000 7f2ca7400640 1 -- 172.21.15.123:0/848339898 >> 172.21.15.123:0/848339898 conn(0x7f2ca0068a50 msgr2=0x7f2ca00d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.504+0000 7f2ca7400640 1 -- 172.21.15.123:0/848339898 shutdown_connections 2024-04-25T12:37:25.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.504+0000 7f2ca7400640 1 -- 172.21.15.123:0/848339898 wait complete. 2024-04-25T12:37:25.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.504+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c4000d50 con 0x7f45c4037d90 2024-04-25T12:37:25.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.504+0000 7f8c37e00640 1 -- 172.21.15.123:0/2428744537 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8c38005d10 con 0x7f8c401516b0 2024-04-25T12:37:25.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.502+0000 7f8915000640 1 -- start start 2024-04-25T12:37:25.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.505+0000 7f8915000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89100c90d0 0x7f89100c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.505+0000 7f7d0d400640 1 Processor -- start 2024-04-25T12:37:25.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.505+0000 7f8915000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89100c4240 0x7f89100cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.505+0000 7f8c37e00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8c20074530 0x7f8c20076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.506+0000 7f8c37e00640 1 -- 172.21.15.123:0/2428744537 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f8c38096110 con 0x7f8c401516b0 2024-04-25T12:37:25.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.506+0000 7f890de00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89100c90d0 0x7f89100c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.506+0000 7f8915000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89100c4810 con 0x7f891015a730 2024-04-25T12:37:25.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.506+0000 7f8915000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f89100c83f0 con 0x7f89100c4240 2024-04-25T12:37:25.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.506+0000 7f8915000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f89100c86d0 con 0x7f89100c90d0 2024-04-25T12:37:25.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.506+0000 7f890de00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89100c90d0 0x7f89100c2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48568/0 (socket says 172.21.15.123:48568) 2024-04-25T12:37:25.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f890de00640 1 -- 172.21.15.123:0/275076512 learned_addr learned my addr 172.21.15.123:0/275076512 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f8c3fe00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8c20074530 0x7f8c20076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.506+0000 7f7d0d400640 1 -- start start 2024-04-25T12:37:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f1c8d600640 1 -- 172.21.15.123:0/2253842611 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1c84043020 con 0x7f1c780039b0 2024-04-25T12:37:25.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f2ca7400640 1 Processor -- start 2024-04-25T12:37:25.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f890de00640 1 -- 172.21.15.123:0/275076512 >> v2:172.21.15.123:3300/0 conn(0x7f89100c4240 msgr2=0x7f89100cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f890de00640 1 --2- 172.21.15.123:0/275076512 >> v2:172.21.15.123:3300/0 conn(0x7f89100c4240 0x7f89100cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f890de00640 1 -- 172.21.15.123:0/275076512 >> v1:172.21.15.123:6789/0 conn(0x7f891015a730 legacy=0x7f8910073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f890de00640 1 -- 172.21.15.123:0/275076512 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89100c89b0 con 0x7f89100c90d0 2024-04-25T12:37:25.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.505+0000 7f45bfe00640 1 -- 172.21.15.123:0/3983743310 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f45b801b070 con 0x7f45c4037d90 2024-04-25T12:37:25.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f45bfe00640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f45ac0744b0 0x7f45ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f45bfe00640 1 -- 172.21.15.123:0/3983743310 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f45b8094a80 con 0x7f45c4037d90 2024-04-25T12:37:25.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f4777000640 1 Processor -- start 2024-04-25T12:37:25.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f4777000640 1 -- start start 2024-04-25T12:37:25.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f7d0d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d080c4240 0x7f7d081fb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f7d0d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d081f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f7d0d400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815a730 0x7f7d0814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f7d0d400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080ce1b0 con 0x7f7d080c90d0 2024-04-25T12:37:25.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f7d0d400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080c8c90 con 0x7f7d080c4240 2024-04-25T12:37:25.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f7d0d400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080ce330 con 0x7f7d0815a730 2024-04-25T12:37:25.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.508+0000 7f45cbe00640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f45ac0744b0 0x7f45ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.507+0000 7f2ca7400640 1 -- start start 2024-04-25T12:37:25.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f8c3fe00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8c20074530 0x7f8c20076a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c30003d50 tx=0x7f8c30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f7d06800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d080c4240 0x7f7d081fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f1c8d600640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1c740748b0 0x7f1c74076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f7d06800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d080c4240 0x7f7d081fb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48584/0 (socket says 172.21.15.123:48584) 2024-04-25T12:37:25.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f7d06800640 1 -- 172.21.15.123:0/1174499165 learned_addr learned my addr 172.21.15.123:0/1174499165 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f7d07200640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815a730 0x7f7d0814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f1c8d600640 1 -- 172.21.15.123:0/2253842611 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f1c84032da0 con 0x7f1c780039b0 2024-04-25T12:37:25.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f1c94a00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1c740748b0 0x7f1c74076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f890de00640 1 --2- 172.21.15.123:0/275076512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89100c90d0 0x7f89100c2870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009870 tx=0x7f88fc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c320eedfc1c4c3bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.509+0000 7f7d05e00640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d081f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f45cbe00640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f45ac0744b0 0x7f45ac0769c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007b50 tx=0x7f45c00019d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f1c94a00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1c740748b0 0x7f1c74076dc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1c9015f8b0 tx=0x7f1c88008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f7d06800640 1 -- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815a730 msgr2=0x7f7d0814cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f4777000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4770152420 0x7f4770152830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f4777000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47700c8540 con 0x7f47700c46b0 2024-04-25T12:37:25.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f4777000640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47700c8820 con 0x7f47700b2fa0 2024-04-25T12:37:25.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f4777000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4770074ba0 con 0x7f4770152420 2024-04-25T12:37:25.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f7d06800640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815a730 0x7f7d0814cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f7d06800640 1 -- 172.21.15.123:0/1174499165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 msgr2=0x7f7d081f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f7d06800640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d081f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f7d06800640 1 -- 172.21.15.123:0/1174499165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d081f85d0 con 0x7f7d080c4240 2024-04-25T12:37:25.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.510+0000 7f7d05e00640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d081f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:37:25.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f4775000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4770152420 0x7f4770152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f476f400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f47700b2fa0 0x7f477015bc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:55992/0 (socket says 172.21.15.123:55992) 2024-04-25T12:37:25.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f476f400640 1 -- 172.21.15.123:0/3071557404 learned_addr learned my addr 172.21.15.123:0/3071557404 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:25.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f890ca00640 1 -- 172.21.15.123:0/275076512 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc037070 con 0x7f89100c90d0 2024-04-25T12:37:25.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f890ca00640 1 -- 172.21.15.123:0/275076512 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc032440 con 0x7f89100c90d0 2024-04-25T12:37:25.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f7d06800640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d080c4240 0x7f7d081fb250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000c00 tx=0x7f7cf00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f890ca00640 1 -- 172.21.15.123:0/275076512 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc032810 con 0x7f89100c90d0 2024-04-25T12:37:25.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f45bfe00640 1 -- 172.21.15.123:0/3983743310 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45b805e710 con 0x7f45c4037d90 2024-04-25T12:37:25.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f4775000640 1 -- 172.21.15.123:0/3071557404 >> v1:172.21.15.153:6789/0 conn(0x7f47700b2fa0 legacy=0x7f477015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca01f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ca00c72c0 0x7f2ca015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ca01606e0 0x7f2ca0155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f2ca7400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01601c0 con 0x7f2ca00c72c0 2024-04-25T12:37:25.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f2ca7400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca015fee0 con 0x7f2ca006d440 2024-04-25T12:37:25.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f2ca7400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca00cc620 con 0x7f2ca01606e0 2024-04-25T12:37:25.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.511+0000 7f7cfea00640 1 -- 172.21.15.123:0/1174499165 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0041030 con 0x7f7d080c4240 2024-04-25T12:37:25.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f4775000640 1 -- 172.21.15.123:0/3071557404 >> v1:172.21.15.123:6789/0 conn(0x7f47700c46b0 legacy=0x7f47700cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:25.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f7cfea00640 1 -- 172.21.15.123:0/1174499165 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf003d420 con 0x7f7d080c4240 2024-04-25T12:37:25.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f7cfea00640 1 -- 172.21.15.123:0/1174499165 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0037040 con 0x7f7d080c4240 2024-04-25T12:37:25.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0814d0e0 con 0x7f7d080c4240 2024-04-25T12:37:25.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d08156680 con 0x7f7d080c4240 2024-04-25T12:37:25.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f2ca5400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ca01606e0 0x7f2ca0155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f2ca4a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca01f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f2ca4a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca01f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48594/0 (socket says 172.21.15.123:48594) 2024-04-25T12:37:25.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f2ca4a00640 1 -- 172.21.15.123:0/3634349064 learned_addr learned my addr 172.21.15.123:0/3634349064 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f2c9fe00640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ca00c72c0 0x7f2ca015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f8915000640 1 -- 172.21.15.123:0/275076512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89100c90d0 msgr2=0x7f89100c2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.512+0000 7f8915000640 1 --2- 172.21.15.123:0/275076512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89100c90d0 0x7f89100c2870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009870 tx=0x7f88fc02a270 comp rx=0 tx=0).stop 2024-04-25T12:37:25.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f8915000640 1 -- 172.21.15.123:0/275076512 shutdown_connections 2024-04-25T12:37:25.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f8915000640 1 --2- 172.21.15.123:0/275076512 >> v2:172.21.15.123:3300/0 conn(0x7f89100c4240 0x7f89100cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f8915000640 1 --2- 172.21.15.123:0/275076512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89100c90d0 0x7f89100c2870 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f8915000640 1 -- 172.21.15.123:0/275076512 >> 172.21.15.123:0/275076512 conn(0x7f891006c750 msgr2=0x7f891006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f4775000640 1 -- 172.21.15.123:0/3071557404 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47700c8b00 con 0x7f4770152420 2024-04-25T12:37:25.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f8915000640 1 -- 172.21.15.123:0/275076512 shutdown_connections 2024-04-25T12:37:25.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f2c9fe00640 1 -- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ca01606e0 msgr2=0x7f2ca0155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f2c9fe00640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ca01606e0 0x7f2ca0155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f2c9fe00640 1 -- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 msgr2=0x7f2ca01f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f2c9fe00640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca01f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f1c8d600640 1 -- 172.21.15.123:0/2253842611 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c90132f30 con 0x7f1c780039b0 2024-04-25T12:37:25.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f2c9fe00640 1 -- 172.21.15.123:0/3634349064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca015ae40 con 0x7f2ca00c72c0 2024-04-25T12:37:25.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.514+0000 7f2c9fe00640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ca00c72c0 0x7f2ca015a900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c031000 tx=0x7f2c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.513+0000 7f8915000640 1 -- 172.21.15.123:0/275076512 wait complete. 2024-04-25T12:37:25.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.515+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce0005290 con 0x7f7d080c4240 2024-04-25T12:37:25.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.515+0000 7f7cfea00640 1 -- 172.21.15.123:0/1174499165 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7cf0044070 con 0x7f7d080c4240 2024-04-25T12:37:25.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.515+0000 7f4775000640 1 --2- 172.21.15.123:0/3071557404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f4770152830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f476800aed0 tx=0x7f476802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606ca5a0f82aad0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.516+0000 7f2c9cc00640 1 -- 172.21.15.123:0/3634349064 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c040020 con 0x7f2ca00c72c0 2024-04-25T12:37:25.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.516+0000 7f7cfea00640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744b0 0x7f7ce40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.517+0000 7f8c37e00640 1 -- 172.21.15.123:0/2428744537 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c38022030 con 0x7f8c401516b0 2024-04-25T12:37:25.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.517+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca0157090 con 0x7f2ca00c72c0 2024-04-25T12:37:25.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.516+0000 7f7cfea00640 1 -- 172.21.15.123:0/1174499165 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f7cf00b55b0 con 0x7f7d080c4240 2024-04-25T12:37:25.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.517+0000 7f7d05e00640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744b0 0x7f7ce40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.517+0000 7f8915000640 1 Processor -- start 2024-04-25T12:37:25.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.517+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca015c290 con 0x7f2ca00c72c0 2024-04-25T12:37:25.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.517+0000 7f2c9cc00640 1 -- 172.21.15.123:0/3634349064 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8c038040 con 0x7f2ca00c72c0 2024-04-25T12:37:25.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.517+0000 7f8915000640 1 -- start start 2024-04-25T12:37:25.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.519+0000 7f2c9cc00640 1 -- 172.21.15.123:0/3634349064 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c037070 con 0x7f2ca00c72c0 2024-04-25T12:37:25.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.519+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca0076700 con 0x7f2ca00c72c0 2024-04-25T12:37:25.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.519+0000 7f7d05e00640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744b0 0x7f7ce40769c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d080c7db0 tx=0x7f7cf80073e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.520+0000 7f8915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89100c4240 0x7f89101487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.521+0000 7f7cfea00640 1 -- 172.21.15.123:0/1174499165 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cf004c020 con 0x7f7d080c4240 2024-04-25T12:37:25.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.521+0000 7f8915000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89100c90d0 0x7f8910158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.521+0000 7f8915000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f891015a730 0x7f8910159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.521+0000 7f8915000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89100ce330 con 0x7f89100c4240 2024-04-25T12:37:25.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.521+0000 7f8915000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89100ce1b0 con 0x7f891015a730 2024-04-25T12:37:25.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.521+0000 7f8915000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f89100744c0 con 0x7f89100c90d0 2024-04-25T12:37:25.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.522+0000 7f890de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89100c4240 0x7f89101487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.522+0000 7f890de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89100c4240 0x7f89101487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41372/0 (socket says 172.21.15.123:41372) 2024-04-25T12:37:25.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.522+0000 7f2c9cc00640 1 -- 172.21.15.123:0/3634349064 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2c8c043020 con 0x7f2ca00c72c0 2024-04-25T12:37:25.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.522+0000 7f890de00640 1 -- 172.21.15.123:0/3602614335 learned_addr learned my addr 172.21.15.123:0/3602614335 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.523+0000 7f890e800640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f891015a730 0x7f8910159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.523+0000 7f2c9cc00640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c800744b0 0x7f2c800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f2c9cc00640 1 -- 172.21.15.123:0/3634349064 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f2c8c0b6280 con 0x7f2ca00c72c0 2024-04-25T12:37:25.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f890de00640 1 -- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89100c90d0 msgr2=0x7f8910158980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T12:37:25.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f890de00640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89100c90d0 0x7f8910158980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f890de00640 1 -- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f891015a730 msgr2=0x7f8910159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f890de00640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f891015a730 0x7f8910159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f890de00640 1 -- 172.21.15.123:0/3602614335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891014a4d0 con 0x7f89100c4240 2024-04-25T12:37:25.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f2ca4a00640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c800744b0 0x7f2c800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f476ea00640 1 -- 172.21.15.123:0/3071557404 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768037070 con 0x7f4770152420 2024-04-25T12:37:25.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f476ea00640 1 -- 172.21.15.123:0/3071557404 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4768008640 con 0x7f4770152420 2024-04-25T12:37:25.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.524+0000 7f476ea00640 1 -- 172.21.15.123:0/3071557404 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768008a10 con 0x7f4770152420 2024-04-25T12:37:25.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.525+0000 7f890de00640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89100c4240 0x7f89101487c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000c00 tx=0x7f88fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.525+0000 7f2ca4a00640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c800744b0 0x7f2c800769c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009370 tx=0x7f2c94009300 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.526+0000 7f4777000640 1 -- 172.21.15.123:0/3071557404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 msgr2=0x7f4770152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.526+0000 7f4777000640 1 --2- 172.21.15.123:0/3071557404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f4770152830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f476800aed0 tx=0x7f476802b820 comp rx=0 tx=0).stop 2024-04-25T12:37:25.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.526+0000 7f4777000640 1 -- 172.21.15.123:0/3071557404 shutdown_connections 2024-04-25T12:37:25.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.526+0000 7f4777000640 1 --2- 172.21.15.123:0/3071557404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f4770152830 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.526+0000 7f4777000640 1 -- 172.21.15.123:0/3071557404 >> 172.21.15.123:0/3071557404 conn(0x7f477006c750 msgr2=0x7f477006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.528+0000 7f4777000640 1 -- 172.21.15.123:0/3071557404 shutdown_connections 2024-04-25T12:37:25.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.528+0000 7f4777000640 1 -- 172.21.15.123:0/3071557404 wait complete. 2024-04-25T12:37:25.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.531+0000 7f4777000640 1 Processor -- start 2024-04-25T12:37:25.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.533+0000 7f8906000640 1 -- 172.21.15.123:0/3602614335 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc037040 con 0x7f89100c4240 2024-04-25T12:37:25.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.532+0000 7f4777000640 1 -- start start 2024-04-25T12:37:25.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.532+0000 7f2c9cc00640 1 -- 172.21.15.123:0/3634349064 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c8c04a030 con 0x7f2ca00c72c0 2024-04-25T12:37:25.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.534+0000 7f4777000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47700b2fa0 0x7f47701558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.534+0000 7f4777000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47700c46b0 0x7f4770156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.534+0000 7f4777000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f477014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.534+0000 7f4777000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4770074710 con 0x7f47700c46b0 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.534+0000 7f4777000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47700ce4f0 con 0x7f4770152420 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.534+0000 7f4777000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f47700b33b0 con 0x7f47700b2fa0 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.535+0000 7f476fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47700b2fa0 0x7f47701558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.535+0000 7f4775000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f477014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.535+0000 7f4775000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f477014b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48618/0 (socket says 172.21.15.123:48618) 2024-04-25T12:37:25.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.535+0000 7f4775000640 1 -- 172.21.15.123:0/2816393307 learned_addr learned my addr 172.21.15.123:0/2816393307 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.533+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89101f7140 con 0x7f89100c4240 2024-04-25T12:37:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.533+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101f7700 con 0x7f89100c4240 2024-04-25T12:37:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.533+0000 7f8906000640 1 -- 172.21.15.123:0/3602614335 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc004230 con 0x7f89100c4240 2024-04-25T12:37:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.533+0000 7f8906000640 1 -- 172.21.15.123:0/3602614335 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc03c670 con 0x7f89100c4240 2024-04-25T12:37:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.534+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8910076700 con 0x7f89100c4240 2024-04-25T12:37:25.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.537+0000 7f4775000640 1 -- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47700b2fa0 msgr2=0x7f47701558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.537+0000 7f8906000640 1 -- 172.21.15.123:0/3602614335 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88fc03a070 con 0x7f89100c4240 2024-04-25T12:37:25.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.537+0000 7f4775000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47700b2fa0 0x7f47701558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.538+0000 7f4775000640 1 -- 172.21.15.123:0/2816393307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47700c46b0 msgr2=0x7f4770156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.538+0000 7f4775000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47700c46b0 0x7f4770156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.538+0000 7f4775000640 1 -- 172.21.15.123:0/2816393307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47701f4ca0 con 0x7f4770152420 2024-04-25T12:37:25.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.538+0000 7f476fe00640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47700b2fa0 0x7f47701558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:25.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.539+0000 7f8906000640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f88f00744e0 0x7f88f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.540+0000 7f890d400640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f88f00744e0 0x7f88f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.540+0000 7f4775000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f477014b830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f476802be10 tx=0x7f4768004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.541+0000 7f4767e00640 1 -- 172.21.15.123:0/2816393307 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768037050 con 0x7f4770152420 2024-04-25T12:37:25.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.541+0000 7f4767e00640 1 -- 172.21.15.123:0/2816393307 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4768043070 con 0x7f4770152420 2024-04-25T12:37:25.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.541+0000 7f890d400640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f88f00744e0 0x7f88f00769f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f89100c7dd0 tx=0x7f8908009250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.541+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47701570c0 con 0x7f4770152420 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.541+0000 7f4767e00640 1 -- 172.21.15.123:0/2816393307 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768032b80 con 0x7f4770152420 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.541+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47701ee770 con 0x7f4770152420 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.543+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4770076700 con 0x7f4770152420 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.545+0000 7f4767e00640 1 -- 172.21.15.123:0/2816393307 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4768048460 con 0x7f4770152420 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.547+0000 7f4767e00640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f474c074580 0x7f474c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.547+0000 7f4767e00640 1 -- 172.21.15.123:0/2816393307 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f47680b65a0 con 0x7f4770152420 2024-04-25T12:37:25.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.548+0000 7f476fe00640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f474c074580 0x7f474c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:25.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.548+0000 7f8906000640 1 -- 172.21.15.123:0/3602614335 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f88fc0b3f30 con 0x7f89100c4240 2024-04-25T12:37:25.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.549+0000 7f476fe00640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f474c074580 0x7f474c076a90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4758006fd0 tx=0x7f4758009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:25.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.549+0000 7f8906000640 1 -- 172.21.15.123:0/3602614335 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88fc041020 con 0x7f89100c4240 2024-04-25T12:37:25.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.550+0000 7f4767e00640 1 -- 172.21.15.123:0/2816393307 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f476804d020 con 0x7f4770152420 2024-04-25T12:37:25.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.892+0000 7f1e5f400640 1 -- 172.21.15.123:0/1081728530 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1e74076da0 con 0x7f1e7413e0d0 2024-04-25T12:37:25.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.893+0000 7f1e6a000640 1 -- 172.21.15.123:0/1081728530 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1e6407fb50 con 0x7f1e7413e0d0 2024-04-25T12:37:25.895 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:25.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.896+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1e54074580 msgr2=0x7f1e54076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1e54074580 0x7f1e54076a90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0094c0 tx=0x7f1e6c00d040 comp rx=0 tx=0).stop 2024-04-25T12:37:25.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 msgr2=0x7f1e74074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e74074130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e740ce520 tx=0x7f1e640046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 shutdown_connections 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1e54074580 0x7f1e54076a90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e7413e0d0 0x7f1e74074130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e74071520 0x7f1e7413bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 --2- 172.21.15.123:0/1081728530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7406cab0 0x7f1e7415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.897+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 >> 172.21.15.123:0/1081728530 conn(0x7f1e740710e0 msgr2=0x7f1e740c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.898+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 shutdown_connections 2024-04-25T12:37:25.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.898+0000 7f1e78c00640 1 -- 172.21.15.123:0/1081728530 wait complete. 2024-04-25T12:37:25.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542187 for osd.2 2024-04-25T12:37:25.936 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:25.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.939+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f45c404e380 con 0x7f45c4037d90 2024-04-25T12:37:25.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.939+0000 7f45bfe00640 1 -- 172.21.15.123:0/3983743310 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f45b8062a10 con 0x7f45c4037d90 2024-04-25T12:37:25.938 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:25.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.942+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f45ac0744b0 msgr2=0x7f45ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f45ac0744b0 0x7f45ac0769c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007b50 tx=0x7f45c00019d0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45c4037d90 msgr2=0x7f45c40516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45c4037d90 0x7f45c40516d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c5b0 tx=0x7f45b800ca70 comp rx=0 tx=0).stop 2024-04-25T12:37:25.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 shutdown_connections 2024-04-25T12:37:25.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f45ac0744b0 0x7f45ac0769c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c404fb60 0x7f45c404b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45c4037d90 0x7f45c40516d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 --2- 172.21.15.123:0/3983743310 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f45c4037840 0x7f45c40e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 >> 172.21.15.123:0/3983743310 conn(0x7f45c400e2e0 msgr2=0x7f45c40dc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 shutdown_connections 2024-04-25T12:37:25.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.943+0000 7f45d1400640 1 -- 172.21.15.123:0/3983743310 wait complete. 2024-04-25T12:37:25.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.950+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2ca00cd160 con 0x7f2ca00c72c0 2024-04-25T12:37:25.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.951+0000 7f2c9cc00640 1 -- 172.21.15.123:0/3634349064 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ca006d850 con 0x7f2ca00c72c0 2024-04-25T12:37:25.950 INFO:teuthology.orchestra.run.smithi123.stdout:201863462931 2024-04-25T12:37:25.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.952+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1c900cdf70 con 0x7f1c780039b0 2024-04-25T12:37:25.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.953+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c800744b0 msgr2=0x7f2c800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.953+0000 7f2ca7400640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c800744b0 0x7f2c800769c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009370 tx=0x7f2c94009300 comp rx=0 tx=0).stop 2024-04-25T12:37:25.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.953+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ca00c72c0 msgr2=0x7f2ca015a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.953+0000 7f2ca7400640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ca00c72c0 0x7f2ca015a900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c031000 tx=0x7f2c8c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f1c8d600640 1 -- 172.21.15.123:0/2253842611 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1c900cdf70 con 0x7f1c780039b0 2024-04-25T12:37:25.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 shutdown_connections 2024-04-25T12:37:25.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f2c800744b0 0x7f2c800769c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ca01606e0 0x7f2ca0155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ca00c72c0 0x7f2ca015a900 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 --2- 172.21.15.123:0/3634349064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ca006d440 0x7f2ca01f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 >> 172.21.15.123:0/3634349064 conn(0x7f2ca0068a50 msgr2=0x7f2ca00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 shutdown_connections 2024-04-25T12:37:25.954 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:25.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.954+0000 7f2ca7400640 1 -- 172.21.15.123:0/3634349064 wait complete. 2024-04-25T12:37:25.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.958+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7ce0005580 con 0x7f7d080c4240 2024-04-25T12:37:25.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.958+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1c740748b0 msgr2=0x7f1c74076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.958+0000 7f1c7ea00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1c740748b0 0x7f1c74076dc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1c9015f8b0 tx=0x7f1c88008040 comp rx=0 tx=0).stop 2024-04-25T12:37:25.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.958+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c780039b0 msgr2=0x7f1c90157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.958+0000 7f1c7ea00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c780039b0 0x7f1c90157c60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c420 tx=0x7f1c840015b0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.958+0000 7f7cfea00640 1 -- 172.21.15.123:0/1174499165 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7cf007f840 con 0x7f7d080c4240 2024-04-25T12:37:25.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 shutdown_connections 2024-04-25T12:37:25.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f1c740748b0 0x7f1c74076dc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c90203280 0x7f1c901ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c90202930 0x7f1c90202d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 --2- 172.21.15.123:0/2253842611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c780039b0 0x7f1c90157c60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 >> 172.21.15.123:0/2253842611 conn(0x7f1c9008d840 msgr2=0x7f1c9015b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.960 INFO:teuthology.orchestra.run.smithi123.stdout:60129542186 2024-04-25T12:37:25.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 shutdown_connections 2024-04-25T12:37:25.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.959+0000 7f1c7ea00640 1 -- 172.21.15.123:0/2253842611 wait complete. 2024-04-25T12:37:25.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744b0 msgr2=0x7f7ce40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744b0 0x7f7ce40769c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d080c7db0 tx=0x7f7cf80073e0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d080c4240 msgr2=0x7f7d081fb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d080c4240 0x7f7d081fb250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000c00 tx=0x7f7cf00046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 shutdown_connections 2024-04-25T12:37:25.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7ce40744b0 0x7f7ce40769c0 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d080c7db0 tx=0x7f7cf80073e0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d0815a730 0x7f7d0814cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080c90d0 0x7f7d081f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 --2- 172.21.15.123:0/1174499165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d080c4240 0x7f7d081fb250 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 >> 172.21.15.123:0/1174499165 conn(0x7f7d0806c750 msgr2=0x7f7d0806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.962+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 shutdown_connections 2024-04-25T12:37:25.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.963+0000 7f7d0d400640 1 -- 172.21.15.123:0/1174499165 wait complete. 2024-04-25T12:37:25.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.963+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8c18005580 con 0x7f8c401516b0 2024-04-25T12:37:25.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.964+0000 7f8c37e00640 1 -- 172.21.15.123:0/2428744537 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8c38060500 con 0x7f8c401516b0 2024-04-25T12:37:25.964 INFO:teuthology.orchestra.run.smithi123.stdout:64424509483 2024-04-25T12:37:25.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.968+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8c20074530 msgr2=0x7f8c20076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.968+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8c20074530 0x7f8c20076a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c30003d50 tx=0x7f8c30009040 comp rx=0 tx=0).stop 2024-04-25T12:37:25.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c401516b0 msgr2=0x7f8c40153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c401516b0 0x7f8c40153ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800fe90 tx=0x7f8c3800d520 comp rx=0 tx=0).stop 2024-04-25T12:37:25.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 shutdown_connections 2024-04-25T12:37:25.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8c20074530 0x7f8c20076a40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4015f5d0 0x7f8c40154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c401516b0 0x7f8c40153ef0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 --2- 172.21.15.123:0/2428744537 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c24005270 0x7f8c4014c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 >> 172.21.15.123:0/2428744537 conn(0x7f8c400cd600 msgr2=0x7f8c400c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.969+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 shutdown_connections 2024-04-25T12:37:25.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.970+0000 7f8c45a00640 1 -- 172.21.15.123:0/2428744537 wait complete. 2024-04-25T12:37:25.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.978+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8910156d60 con 0x7f89100c4240 2024-04-25T12:37:25.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.980+0000 7f8906000640 1 -- 172.21.15.123:0/3602614335 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88fc080220 con 0x7f89100c4240 2024-04-25T12:37:25.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542187 for osd.7 2024-04-25T12:37:25.980 INFO:teuthology.orchestra.run.smithi123.stdout:64424509483 2024-04-25T12:37:25.980 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:25.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f88f00744e0 msgr2=0x7f88f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f88f00744e0 0x7f88f00769f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f89100c7dd0 tx=0x7f8908009250 comp rx=0 tx=0).stop 2024-04-25T12:37:25.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89100c4240 msgr2=0x7f89101487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89100c4240 0x7f89101487c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000c00 tx=0x7f88fc0046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 shutdown_connections 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f88f00744e0 0x7f88f00769f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f891015a730 0x7f8910159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89100c90d0 0x7f8910158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 --2- 172.21.15.123:0/3602614335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89100c4240 0x7f89101487c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 >> 172.21.15.123:0/3602614335 conn(0x7f891006c750 msgr2=0x7f8910156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.984+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 shutdown_connections 2024-04-25T12:37:25.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.985+0000 7f8915000640 1 -- 172.21.15.123:0/3602614335 wait complete. 2024-04-25T12:37:25.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.993+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4770073250 con 0x7f4770152420 2024-04-25T12:37:25.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 201863462930 got 201863462931 for osd.5 2024-04-25T12:37:25.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542187 for osd.6 2024-04-25T12:37:25.993 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:25.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.993+0000 7f4767e00640 1 -- 172.21.15.123:0/2816393307 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47680809c0 con 0x7f4770152420 2024-04-25T12:37:25.994 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:25.994 INFO:teuthology.orchestra.run.smithi123.stdout:60129542187 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f474c074580 msgr2=0x7f474c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f474c074580 0x7f474c076a90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4758006fd0 tx=0x7f4758009060 comp rx=0 tx=0).stop 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 msgr2=0x7f477014b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f477014b830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f476802be10 tx=0x7f4768004a40 comp rx=0 tx=0).stop 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 shutdown_connections 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f474c074580 0x7f474c076a90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4770152420 0x7f477014b830 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47700c46b0 0x7f4770156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 --2- 172.21.15.123:0/2816393307 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47700b2fa0 0x7f47701558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.996+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 >> 172.21.15.123:0/2816393307 conn(0x7f477006c750 msgr2=0x7f4770151650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.997+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 shutdown_connections 2024-04-25T12:37:25.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:25.997+0000 7f4777000640 1 -- 172.21.15.123:0/2816393307 wait complete. 2024-04-25T12:37:26.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.1 2024-04-25T12:37:26.005 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:26.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509483 for osd.0 2024-04-25T12:37:26.009 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:26.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509483 for osd.3 2024-04-25T12:37:26.014 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:26.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542187 for osd.4 2024-04-25T12:37:26.026 DEBUG:teuthology.parallel:result is None 2024-04-25T12:37:26.026 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-25T12:37:26.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:37:26.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.320+0000 7f0b37200640 1 Processor -- start 2024-04-25T12:37:26.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.321+0000 7f0b37200640 1 -- start start 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.322+0000 7f0b37200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b380cdc60 0x7f0b380b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.322+0000 7f0b37200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0b380d1ad0 0x7f0b380d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b37200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b3815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b37200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3815fdd0 con 0x7f0b380cdc60 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b37200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0b381600b0 con 0x7f0b380d1ad0 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b37200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0b38160390 con 0x7f0b3815c850 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b380cdc60 0x7f0b380b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b380cdc60 0x7f0b380b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41390/0 (socket says 172.21.15.123:41390) 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 -- 172.21.15.123:0/3671638461 learned_addr learned my addr 172.21.15.123:0/3671638461 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b2d400640 1 --2- 172.21.15.123:0/3671638461 >> v2:172.21.15.153:3300/0 conn(0x7f0b380d1ad0 0x7f0b380d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b36800640 1 --2- 172.21.15.123:0/3671638461 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b3815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 -- 172.21.15.123:0/3671638461 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 msgr2=0x7f0b3815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:26.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 --2- 172.21.15.123:0/3671638461 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b3815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 -- 172.21.15.123:0/3671638461 >> v2:172.21.15.153:3300/0 conn(0x7f0b380d1ad0 msgr2=0x7f0b380d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:26.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 --2- 172.21.15.123:0/3671638461 >> v2:172.21.15.153:3300/0 conn(0x7f0b380d1ad0 0x7f0b380d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.323+0000 7f0b35e00640 1 -- 172.21.15.123:0/3671638461 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b38160670 con 0x7f0b380cdc60 2024-04-25T12:37:26.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.324+0000 7f0b35e00640 1 --2- 172.21.15.123:0/3671638461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380b42c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b380c91c0 tx=0x7f0b2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3662673216d9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:26.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b35400640 1 -- 172.21.15.123:0/3671638461 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28037070 con 0x7f0b380cdc60 2024-04-25T12:37:26.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b35400640 1 -- 172.21.15.123:0/3671638461 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b28032440 con 0x7f0b380cdc60 2024-04-25T12:37:26.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b35400640 1 -- 172.21.15.123:0/3671638461 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28032810 con 0x7f0b380cdc60 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b37200640 1 -- 172.21.15.123:0/3671638461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 msgr2=0x7f0b380b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b37200640 1 --2- 172.21.15.123:0/3671638461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380b42c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b380c91c0 tx=0x7f0b2802a1e0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b37200640 1 -- 172.21.15.123:0/3671638461 shutdown_connections 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b37200640 1 --2- 172.21.15.123:0/3671638461 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b3815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b37200640 1 --2- 172.21.15.123:0/3671638461 >> v2:172.21.15.153:3300/0 conn(0x7f0b380d1ad0 0x7f0b380d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b37200640 1 --2- 172.21.15.123:0/3671638461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380b42c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.325+0000 7f0b37200640 1 -- 172.21.15.123:0/3671638461 >> 172.21.15.123:0/3671638461 conn(0x7f0b3808d840 msgr2=0x7f0b3808dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.326+0000 7f0b37200640 1 -- 172.21.15.123:0/3671638461 shutdown_connections 2024-04-25T12:37:26.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.326+0000 7f0b37200640 1 -- 172.21.15.123:0/3671638461 wait complete. 2024-04-25T12:37:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.328+0000 7f0b37200640 1 Processor -- start 2024-04-25T12:37:26.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.328+0000 7f0b37200640 1 -- start start 2024-04-25T12:37:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.329+0000 7f0b37200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.329+0000 7f0b37200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b380d1ad0 0x7f0b38158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.329+0000 7f0b37200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b38158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.329+0000 7f0b37200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380b4980 con 0x7f0b380cdc60 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.329+0000 7f0b37200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380b4800 con 0x7f0b380d1ad0 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.329+0000 7f0b37200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3815bbb0 con 0x7f0b3815c850 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41402/0 (socket says 172.21.15.123:41402) 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 -- 172.21.15.123:0/2186876417 learned_addr learned my addr 172.21.15.123:0/2186876417 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b36800640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b38158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b2d400640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b380d1ad0 0x7f0b38158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 -- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 msgr2=0x7f0b38158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b38158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 -- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b380d1ad0 msgr2=0x7f0b38158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b380d1ad0 0x7f0b38158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 -- 172.21.15.123:0/2186876417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b381f2b20 con 0x7f0b380cdc60 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b36800640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b38158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b35e00640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380ce3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802a220 tx=0x7f0b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.330+0000 7f0b2ea00640 1 -- 172.21.15.123:0/2186876417 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28041020 con 0x7f0b380cdc60 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.331+0000 7f0b2ea00640 1 -- 172.21.15.123:0/2186876417 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b2803d620 con 0x7f0b380cdc60 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.331+0000 7f0b2ea00640 1 -- 172.21.15.123:0/2186876417 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28037040 con 0x7f0b380cdc60 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.331+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3815e010 con 0x7f0b380cdc60 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.331+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381f30c0 con 0x7f0b380cdc60 2024-04-25T12:37:26.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.332+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b38076da0 con 0x7f0b380cdc60 2024-04-25T12:37:26.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.333+0000 7f0b2ea00640 1 -- 172.21.15.123:0/2186876417 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0b28044020 con 0x7f0b380cdc60 2024-04-25T12:37:26.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.334+0000 7f0b2ea00640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0b140744b0 0x7f0b140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.334+0000 7f0b2ea00640 1 -- 172.21.15.123:0/2186876417 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f0b280b5e80 con 0x7f0b380cdc60 2024-04-25T12:37:26.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.334+0000 7f0b2d400640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0b140744b0 0x7f0b140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.335+0000 7f0b2d400640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0b140744b0 0x7f0b140769c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b3815d130 tx=0x7f0b20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:26.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.338+0000 7f0b2ea00640 1 -- 172.21.15.123:0/2186876417 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b2804b030 con 0x7f0b380cdc60 2024-04-25T12:37:26.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.523+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b380d0a20 con 0x7f0b140744b0 2024-04-25T12:37:26.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.533+0000 7f0b2ea00640 1 -- 172.21.15.123:0/2186876417 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333131 (secure 0 0 0) 0x7f0b380d0a20 con 0x7f0b140744b0 2024-04-25T12:37:26.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:26.538 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.538+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0b140744b0 msgr2=0x7f0b140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.538+0000 7f0b37200640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0b140744b0 0x7f0b140769c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b3815d130 tx=0x7f0b20008040 comp rx=0 tx=0).stop 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.538+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 msgr2=0x7f0b380ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.538+0000 7f0b37200640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380ce3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802a220 tx=0x7f0b280046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 shutdown_connections 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0b140744b0 0x7f0b140769c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0b3815c850 0x7f0b38158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b380d1ad0 0x7f0b38158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 --2- 172.21.15.123:0/2186876417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cdc60 0x7f0b380ce3f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 >> 172.21.15.123:0/2186876417 conn(0x7f0b3808d840 msgr2=0x7f0b380b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:26.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 shutdown_connections 2024-04-25T12:37:26.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.539+0000 7f0b37200640 1 -- 172.21.15.123:0/2186876417 wait complete. 2024-04-25T12:37:26.563 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-04-25T12:37:24.840154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631420,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":269,"num_write_kb":1725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234616,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":225579,"kb_avail":754740104,"statfs":{"total":773094113280,"available":772853866496,"internally_reserved":0,"allocated":9093120,"data_stored":5889728,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":230993875},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001753"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569452+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:37:21.569452+0000","last_peered":"2024-04-25T12:37:21.569452+0000","last_clean":"2024-04-25T12:37:21.569452+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:21.569452+0000","last_undegraded":"2024-04-25T12:37:21.569452+0000","last_fullsized":"2024-04-25T12:37:21.569452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908252+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:37:21.908252+0000","last_peered":"2024-04-25T12:37:21.908252+0000","last_clean":"2024-04-25T12:37:21.908252+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:21.908252+0000","last_undegraded":"2024-04-25T12:37:21.908252+0000","last_fullsized":"2024-04-25T12:37:21.908252+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402490+0000","last_change":"2024-04-25T12:37:01.222445+0000","last_active":"2024-04-25T12:37:21.402490+0000","last_peered":"2024-04-25T12:37:21.402490+0000","last_clean":"2024-04-25T12:37:21.402490+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:21.402490+0000","last_undegraded":"2024-04-25T12:37:21.402490+0000","last_fullsized":"2024-04-25T12:37:21.402490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907880+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:37:21.907880+0000","last_peered":"2024-04-25T12:37:21.907880+0000","last_clean":"2024-04-25T12:37:21.907880+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:21.907880+0000","last_undegraded":"2024-04-25T12:37:21.907880+0000","last_fullsized":"2024-04-25T12:37:21.907880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908265+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:37:21.908265+0000","last_peered":"2024-04-25T12:37:21.908265+0000","last_clean":"2024-04-25T12:37:21.908265+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:21.908265+0000","last_undegraded":"2024-04-25T12:37:21.908265+0000","last_fullsized":"2024-04-25T12:37:21.908265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404380+0000","last_change":"2024-04-25T12:37:01.216562+0000","last_active":"2024-04-25T12:37:21.404380+0000","last_peered":"2024-04-25T12:37:21.404380+0000","last_clean":"2024-04-25T12:37:21.404380+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:21.404380+0000","last_undegraded":"2024-04-25T12:37:21.404380+0000","last_fullsized":"2024-04-25T12:37:21.404380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910309+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:37:21.910309+0000","last_peered":"2024-04-25T12:37:21.910309+0000","last_clean":"2024-04-25T12:37:21.910309+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:21.910309+0000","last_undegraded":"2024-04-25T12:37:21.910309+0000","last_fullsized":"2024-04-25T12:37:21.910309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912549+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:37:21.912549+0000","last_peered":"2024-04-25T12:37:21.912549+0000","last_clean":"2024-04-25T12:37:21.912549+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:21.912549+0000","last_undegraded":"2024-04-25T12:37:21.912549+0000","last_fullsized":"2024-04-25T12:37:21.912549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405114+0000","last_change":"2024-04-25T12:37:01.226633+0000","last_active":"2024-04-25T12:37:21.405114+0000","last_peered":"2024-04-25T12:37:21.405114+0000","last_clean":"2024-04-25T12:37:21.405114+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:21.405114+0000","last_undegraded":"2024-04-25T12:37:21.405114+0000","last_fullsized":"2024-04-25T12:37:21.405114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.771833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910815+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:37:21.910815+0000","last_peered":"2024-04-25T12:37:21.910815+0000","last_clean":"2024-04-25T12:37:21.910815+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:21.910815+0000","last_undegraded":"2024-04-25T12:37:21.910815+0000","last_fullsized":"2024-04-25T12:37:21.910815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423689+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:37:21.423689+0000","last_peered":"2024-04-25T12:37:21.423689+0000","last_clean":"2024-04-25T12:37:21.423689+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:21.423689+0000","last_undegraded":"2024-04-25T12:37:21.423689+0000","last_fullsized":"2024-04-25T12:37:21.423689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404697+0000","last_change":"2024-04-25T12:37:01.221650+0000","last_active":"2024-04-25T12:37:21.404697+0000","last_peered":"2024-04-25T12:37:21.404697+0000","last_clean":"2024-04-25T12:37:21.404697+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:21.404697+0000","last_undegraded":"2024-04-25T12:37:21.404697+0000","last_fullsized":"2024-04-25T12:37:21.404697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422862+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:37:21.422862+0000","last_peered":"2024-04-25T12:37:21.422862+0000","last_clean":"2024-04-25T12:37:21.422862+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:21.422862+0000","last_undegraded":"2024-04-25T12:37:21.422862+0000","last_fullsized":"2024-04-25T12:37:21.422862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423223+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:37:21.423223+0000","last_peered":"2024-04-25T12:37:21.423223+0000","last_clean":"2024-04-25T12:37:21.423223+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:21.423223+0000","last_undegraded":"2024-04-25T12:37:21.423223+0000","last_fullsized":"2024-04-25T12:37:21.423223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909732+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:37:21.909732+0000","last_peered":"2024-04-25T12:37:21.909732+0000","last_clean":"2024-04-25T12:37:21.909732+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:21.909732+0000","last_undegraded":"2024-04-25T12:37:21.909732+0000","last_fullsized":"2024-04-25T12:37:21.909732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402959+0000","last_change":"2024-04-25T12:37:01.209490+0000","last_active":"2024-04-25T12:37:21.402959+0000","last_peered":"2024-04-25T12:37:21.402959+0000","last_clean":"2024-04-25T12:37:21.402959+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:21.402959+0000","last_undegraded":"2024-04-25T12:37:21.402959+0000","last_fullsized":"2024-04-25T12:37:21.402959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:46.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908543+0000","last_change":"2024-04-25T12:37:01.230485+0000","last_active":"2024-04-25T12:37:21.908543+0000","last_peered":"2024-04-25T12:37:21.908543+0000","last_clean":"2024-04-25T12:37:21.908543+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:21.908543+0000","last_undegraded":"2024-04-25T12:37:21.908543+0000","last_fullsized":"2024-04-25T12:37:21.908543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:47.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909989+0000","last_change":"2024-04-25T12:37:01.222195+0000","last_active":"2024-04-25T12:37:21.909989+0000","last_peered":"2024-04-25T12:37:21.909989+0000","last_clean":"2024-04-25T12:37:21.909989+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:21.909989+0000","last_undegraded":"2024-04-25T12:37:21.909989+0000","last_fullsized":"2024-04-25T12:37:21.909989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:35.550835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568700+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:37:21.568700+0000","last_peered":"2024-04-25T12:37:21.568700+0000","last_clean":"2024-04-25T12:37:21.568700+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:21.568700+0000","last_undegraded":"2024-04-25T12:37:21.568700+0000","last_fullsized":"2024-04-25T12:37:21.568700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569070+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:37:21.569070+0000","last_peered":"2024-04-25T12:37:21.569070+0000","last_clean":"2024-04-25T12:37:21.569070+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:21.569070+0000","last_undegraded":"2024-04-25T12:37:21.569070+0000","last_fullsized":"2024-04-25T12:37:21.569070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403488+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:37:21.403488+0000","last_peered":"2024-04-25T12:37:21.403488+0000","last_clean":"2024-04-25T12:37:21.403488+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:21.403488+0000","last_undegraded":"2024-04-25T12:37:21.403488+0000","last_fullsized":"2024-04-25T12:37:21.403488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422941+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:37:21.422941+0000","last_peered":"2024-04-25T12:37:21.422941+0000","last_clean":"2024-04-25T12:37:21.422941+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:21.422941+0000","last_undegraded":"2024-04-25T12:37:21.422941+0000","last_fullsized":"2024-04-25T12:37:21.422941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422292+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:37:21.422292+0000","last_peered":"2024-04-25T12:37:21.422292+0000","last_clean":"2024-04-25T12:37:21.422292+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:37:21.422292+0000","last_undegraded":"2024-04-25T12:37:21.422292+0000","last_fullsized":"2024-04-25T12:37:21.422292+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405968+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:37:21.405968+0000","last_peered":"2024-04-25T12:37:21.405968+0000","last_clean":"2024-04-25T12:37:21.405968+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:21.405968+0000","last_undegraded":"2024-04-25T12:37:21.405968+0000","last_fullsized":"2024-04-25T12:37:21.405968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569094+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:37:21.569094+0000","last_peered":"2024-04-25T12:37:21.569094+0000","last_clean":"2024-04-25T12:37:21.569094+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:21.569094+0000","last_undegraded":"2024-04-25T12:37:21.569094+0000","last_fullsized":"2024-04-25T12:37:21.569094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424634+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:37:21.424634+0000","last_peered":"2024-04-25T12:37:21.424634+0000","last_clean":"2024-04-25T12:37:21.424634+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:21.424634+0000","last_undegraded":"2024-04-25T12:37:21.424634+0000","last_fullsized":"2024-04-25T12:37:21.424634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404952+0000","last_change":"2024-04-25T12:37:01.226160+0000","last_active":"2024-04-25T12:37:21.404952+0000","last_peered":"2024-04-25T12:37:21.404952+0000","last_clean":"2024-04-25T12:37:21.404952+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:21.404952+0000","last_undegraded":"2024-04-25T12:37:21.404952+0000","last_fullsized":"2024-04-25T12:37:21.404952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:43.475095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569434+0000","last_change":"2024-04-25T12:37:01.206319+0000","last_active":"2024-04-25T12:37:21.569434+0000","last_peered":"2024-04-25T12:37:21.569434+0000","last_clean":"2024-04-25T12:37:21.569434+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:21.569434+0000","last_undegraded":"2024-04-25T12:37:21.569434+0000","last_fullsized":"2024-04-25T12:37:21.569434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"74'12","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569349+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:37:21.569349+0000","last_peered":"2024-04-25T12:37:21.569349+0000","last_clean":"2024-04-25T12:37:21.569349+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:21.569349+0000","last_undegraded":"2024-04-25T12:37:21.569349+0000","last_fullsized":"2024-04-25T12:37:21.569349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568614+0000","last_change":"2024-04-25T12:37:01.225992+0000","last_active":"2024-04-25T12:37:21.568614+0000","last_peered":"2024-04-25T12:37:21.568614+0000","last_clean":"2024-04-25T12:37:21.568614+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:21.568614+0000","last_undegraded":"2024-04-25T12:37:21.568614+0000","last_fullsized":"2024-04-25T12:37:21.568614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:05.029213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908795+0000","last_change":"2024-04-25T12:37:01.208764+0000","last_active":"2024-04-25T12:37:21.908795+0000","last_peered":"2024-04-25T12:37:21.908795+0000","last_clean":"2024-04-25T12:37:21.908795+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:21.908795+0000","last_undegraded":"2024-04-25T12:37:21.908795+0000","last_fullsized":"2024-04-25T12:37:21.908795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:38.538923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424069+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:37:21.424069+0000","last_peered":"2024-04-25T12:37:21.424069+0000","last_clean":"2024-04-25T12:37:21.424069+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:21.424069+0000","last_undegraded":"2024-04-25T12:37:21.424069+0000","last_fullsized":"2024-04-25T12:37:21.424069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908596+0000","last_change":"2024-04-25T12:37:01.225006+0000","last_active":"2024-04-25T12:37:21.908596+0000","last_peered":"2024-04-25T12:37:21.908596+0000","last_clean":"2024-04-25T12:37:21.908596+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:21.908596+0000","last_undegraded":"2024-04-25T12:37:21.908596+0000","last_fullsized":"2024-04-25T12:37:21.908596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405700+0000","last_change":"2024-04-25T12:37:01.222179+0000","last_active":"2024-04-25T12:37:21.405700+0000","last_peered":"2024-04-25T12:37:21.405700+0000","last_clean":"2024-04-25T12:37:21.405700+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:21.405700+0000","last_undegraded":"2024-04-25T12:37:21.405700+0000","last_fullsized":"2024-04-25T12:37:21.405700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907696+0000","last_change":"2024-04-25T12:37:01.211908+0000","last_active":"2024-04-25T12:37:21.907696+0000","last_peered":"2024-04-25T12:37:21.907696+0000","last_clean":"2024-04-25T12:37:21.907696+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:21.907696+0000","last_undegraded":"2024-04-25T12:37:21.907696+0000","last_fullsized":"2024-04-25T12:37:21.907696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:23.523737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908482+0000","last_change":"2024-04-25T12:37:01.229773+0000","last_active":"2024-04-25T12:37:21.908482+0000","last_peered":"2024-04-25T12:37:21.908482+0000","last_clean":"2024-04-25T12:37:21.908482+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:21.908482+0000","last_undegraded":"2024-04-25T12:37:21.908482+0000","last_fullsized":"2024-04-25T12:37:21.908482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:55.680417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402818+0000","last_change":"2024-04-25T12:37:01.223560+0000","last_active":"2024-04-25T12:37:21.402818+0000","last_peered":"2024-04-25T12:37:21.402818+0000","last_clean":"2024-04-25T12:37:21.402818+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:21.402818+0000","last_undegraded":"2024-04-25T12:37:21.402818+0000","last_fullsized":"2024-04-25T12:37:21.402818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:18.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424227+0000","last_change":"2024-04-25T12:37:01.227566+0000","last_active":"2024-04-25T12:37:21.424227+0000","last_peered":"2024-04-25T12:37:21.424227+0000","last_clean":"2024-04-25T12:37:21.424227+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:37:21.424227+0000","last_undegraded":"2024-04-25T12:37:21.424227+0000","last_fullsized":"2024-04-25T12:37:21.424227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:43.265214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908845+0000","last_change":"2024-04-25T12:37:01.226991+0000","last_active":"2024-04-25T12:37:21.908845+0000","last_peered":"2024-04-25T12:37:21.908845+0000","last_clean":"2024-04-25T12:37:21.908845+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:21.908845+0000","last_undegraded":"2024-04-25T12:37:21.908845+0000","last_fullsized":"2024-04-25T12:37:21.908845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:23.065480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907219+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:37:21.907219+0000","last_peered":"2024-04-25T12:37:21.907219+0000","last_clean":"2024-04-25T12:37:21.907219+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:21.907219+0000","last_undegraded":"2024-04-25T12:37:21.907219+0000","last_fullsized":"2024-04-25T12:37:21.907219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'7","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402648+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:37:21.402648+0000","last_peered":"2024-04-25T12:37:21.402648+0000","last_clean":"2024-04-25T12:37:21.402648+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:21.402648+0000","last_undegraded":"2024-04-25T12:37:21.402648+0000","last_fullsized":"2024-04-25T12:37:21.402648+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907983+0000","last_change":"2024-04-25T12:37:01.222254+0000","last_active":"2024-04-25T12:37:21.907983+0000","last_peered":"2024-04-25T12:37:21.907983+0000","last_clean":"2024-04-25T12:37:21.907983+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:21.907983+0000","last_undegraded":"2024-04-25T12:37:21.907983+0000","last_fullsized":"2024-04-25T12:37:21.907983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908045+0000","last_change":"2024-04-25T12:37:01.224799+0000","last_active":"2024-04-25T12:37:21.908045+0000","last_peered":"2024-04-25T12:37:21.908045+0000","last_clean":"2024-04-25T12:37:21.908045+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:21.908045+0000","last_undegraded":"2024-04-25T12:37:21.908045+0000","last_fullsized":"2024-04-25T12:37:21.908045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:24.737805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423362+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:37:21.423362+0000","last_peered":"2024-04-25T12:37:21.423362+0000","last_clean":"2024-04-25T12:37:21.423362+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:21.423362+0000","last_undegraded":"2024-04-25T12:37:21.423362+0000","last_fullsized":"2024-04-25T12:37:21.423362+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908375+0000","last_change":"2024-04-25T12:37:01.226465+0000","last_active":"2024-04-25T12:37:21.908375+0000","last_peered":"2024-04-25T12:37:21.908375+0000","last_clean":"2024-04-25T12:37:21.908375+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:21.908375+0000","last_undegraded":"2024-04-25T12:37:21.908375+0000","last_fullsized":"2024-04-25T12:37:21.908375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:26.218787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568893+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:37:21.568893+0000","last_peered":"2024-04-25T12:37:21.568893+0000","last_clean":"2024-04-25T12:37:21.568893+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:21.568893+0000","last_undegraded":"2024-04-25T12:37:21.568893+0000","last_fullsized":"2024-04-25T12:37:21.568893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423614+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:37:21.423614+0000","last_peered":"2024-04-25T12:37:21.423614+0000","last_clean":"2024-04-25T12:37:21.423614+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:21.423614+0000","last_undegraded":"2024-04-25T12:37:21.423614+0000","last_fullsized":"2024-04-25T12:37:21.423614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910887+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:37:21.910887+0000","last_peered":"2024-04-25T12:37:21.910887+0000","last_clean":"2024-04-25T12:37:21.910887+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:21.910887+0000","last_undegraded":"2024-04-25T12:37:21.910887+0000","last_fullsized":"2024-04-25T12:37:21.910887+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423841+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:37:21.423841+0000","last_peered":"2024-04-25T12:37:21.423841+0000","last_clean":"2024-04-25T12:37:21.423841+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:21.423841+0000","last_undegraded":"2024-04-25T12:37:21.423841+0000","last_fullsized":"2024-04-25T12:37:21.423841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423763+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:37:21.423763+0000","last_peered":"2024-04-25T12:37:21.423763+0000","last_clean":"2024-04-25T12:37:21.423763+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:21.423763+0000","last_undegraded":"2024-04-25T12:37:21.423763+0000","last_fullsized":"2024-04-25T12:37:21.423763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908962+0000","last_change":"2024-04-25T12:37:01.242232+0000","last_active":"2024-04-25T12:37:21.908962+0000","last_peered":"2024-04-25T12:37:21.908962+0000","last_clean":"2024-04-25T12:37:21.908962+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:21.908962+0000","last_undegraded":"2024-04-25T12:37:21.908962+0000","last_fullsized":"2024-04-25T12:37:21.908962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:59.941469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404864+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:37:21.404864+0000","last_peered":"2024-04-25T12:37:21.404864+0000","last_clean":"2024-04-25T12:37:21.404864+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:21.404864+0000","last_undegraded":"2024-04-25T12:37:21.404864+0000","last_fullsized":"2024-04-25T12:37:21.404864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912041+0000","last_change":"2024-04-25T12:37:01.220247+0000","last_active":"2024-04-25T12:37:21.912041+0000","last_peered":"2024-04-25T12:37:21.912041+0000","last_clean":"2024-04-25T12:37:21.912041+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:21.912041+0000","last_undegraded":"2024-04-25T12:37:21.912041+0000","last_fullsized":"2024-04-25T12:37:21.912041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:13.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908088+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:37:21.908088+0000","last_peered":"2024-04-25T12:37:21.908088+0000","last_clean":"2024-04-25T12:37:21.908088+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:21.908088+0000","last_undegraded":"2024-04-25T12:37:21.908088+0000","last_fullsized":"2024-04-25T12:37:21.908088+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":112,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907766+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:37:21.907766+0000","last_peered":"2024-04-25T12:37:21.907766+0000","last_clean":"2024-04-25T12:37:21.907766+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:21.907766+0000","last_undegraded":"2024-04-25T12:37:21.907766+0000","last_fullsized":"2024-04-25T12:37:21.907766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911665+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:37:21.911665+0000","last_peered":"2024-04-25T12:37:21.911665+0000","last_clean":"2024-04-25T12:37:21.911665+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:21.911665+0000","last_undegraded":"2024-04-25T12:37:21.911665+0000","last_fullsized":"2024-04-25T12:37:21.911665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569231+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:37:21.569231+0000","last_peered":"2024-04-25T12:37:21.569231+0000","last_clean":"2024-04-25T12:37:21.569231+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:21.569231+0000","last_undegraded":"2024-04-25T12:37:21.569231+0000","last_fullsized":"2024-04-25T12:37:21.569231+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422865+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:37:21.422865+0000","last_peered":"2024-04-25T12:37:21.422865+0000","last_clean":"2024-04-25T12:37:21.422865+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:21.422865+0000","last_undegraded":"2024-04-25T12:37:21.422865+0000","last_fullsized":"2024-04-25T12:37:21.422865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406286+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:37:21.406286+0000","last_peered":"2024-04-25T12:37:21.406286+0000","last_clean":"2024-04-25T12:37:21.406286+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:21.406286+0000","last_undegraded":"2024-04-25T12:37:21.406286+0000","last_fullsized":"2024-04-25T12:37:21.406286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911823+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:37:21.911823+0000","last_peered":"2024-04-25T12:37:21.911823+0000","last_clean":"2024-04-25T12:37:21.911823+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:21.911823+0000","last_undegraded":"2024-04-25T12:37:21.911823+0000","last_fullsized":"2024-04-25T12:37:21.911823+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910437+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:37:21.910437+0000","last_peered":"2024-04-25T12:37:21.910437+0000","last_clean":"2024-04-25T12:37:21.910437+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:21.910437+0000","last_undegraded":"2024-04-25T12:37:21.910437+0000","last_fullsized":"2024-04-25T12:37:21.910437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422738+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:37:21.422738+0000","last_peered":"2024-04-25T12:37:21.422738+0000","last_clean":"2024-04-25T12:37:21.422738+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:21.422738+0000","last_undegraded":"2024-04-25T12:37:21.422738+0000","last_fullsized":"2024-04-25T12:37:21.422738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"75'16","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568873+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:37:21.568873+0000","last_peered":"2024-04-25T12:37:21.568873+0000","last_clean":"2024-04-25T12:37:21.568873+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:21.568873+0000","last_undegraded":"2024-04-25T12:37:21.568873+0000","last_fullsized":"2024-04-25T12:37:21.568873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908280+0000","last_change":"2024-04-25T12:37:01.226590+0000","last_active":"2024-04-25T12:37:21.908280+0000","last_peered":"2024-04-25T12:37:21.908280+0000","last_clean":"2024-04-25T12:37:21.908280+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:21.908280+0000","last_undegraded":"2024-04-25T12:37:21.908280+0000","last_fullsized":"2024-04-25T12:37:21.908280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.581831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910537+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:37:21.910537+0000","last_peered":"2024-04-25T12:37:21.910537+0000","last_clean":"2024-04-25T12:37:21.910537+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:21.910537+0000","last_undegraded":"2024-04-25T12:37:21.910537+0000","last_fullsized":"2024-04-25T12:37:21.910537+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910045+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:37:21.910045+0000","last_peered":"2024-04-25T12:37:21.910045+0000","last_clean":"2024-04-25T12:37:21.910045+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:21.910045+0000","last_undegraded":"2024-04-25T12:37:21.910045+0000","last_fullsized":"2024-04-25T12:37:21.910045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908364+0000","last_change":"2024-04-25T12:37:01.220767+0000","last_active":"2024-04-25T12:37:21.908364+0000","last_peered":"2024-04-25T12:37:21.908364+0000","last_clean":"2024-04-25T12:37:21.908364+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:21.908364+0000","last_undegraded":"2024-04-25T12:37:21.908364+0000","last_fullsized":"2024-04-25T12:37:21.908364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:15.874258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422741+0000","last_change":"2024-04-25T12:37:01.233439+0000","last_active":"2024-04-25T12:37:21.422741+0000","last_peered":"2024-04-25T12:37:21.422741+0000","last_clean":"2024-04-25T12:37:21.422741+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:37:21.422741+0000","last_undegraded":"2024-04-25T12:37:21.422741+0000","last_fullsized":"2024-04-25T12:37:21.422741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:11.957744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424042+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:37:21.424042+0000","last_peered":"2024-04-25T12:37:21.424042+0000","last_clean":"2024-04-25T12:37:21.424042+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:21.424042+0000","last_undegraded":"2024-04-25T12:37:21.424042+0000","last_fullsized":"2024-04-25T12:37:21.424042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908903+0000","last_change":"2024-04-25T12:37:01.225914+0000","last_active":"2024-04-25T12:37:21.908903+0000","last_peered":"2024-04-25T12:37:21.908903+0000","last_clean":"2024-04-25T12:37:21.908903+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:21.908903+0000","last_undegraded":"2024-04-25T12:37:21.908903+0000","last_fullsized":"2024-04-25T12:37:21.908903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:34.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908657+0000","last_change":"2024-04-25T12:37:01.229502+0000","last_active":"2024-04-25T12:37:21.908657+0000","last_peered":"2024-04-25T12:37:21.908657+0000","last_clean":"2024-04-25T12:37:21.908657+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:21.908657+0000","last_undegraded":"2024-04-25T12:37:21.908657+0000","last_fullsized":"2024-04-25T12:37:21.908657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:25.697423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403098+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:37:21.403098+0000","last_peered":"2024-04-25T12:37:21.403098+0000","last_clean":"2024-04-25T12:37:21.403098+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:21.403098+0000","last_undegraded":"2024-04-25T12:37:21.403098+0000","last_fullsized":"2024-04-25T12:37:21.403098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912070+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:37:21.912070+0000","last_peered":"2024-04-25T12:37:21.912070+0000","last_clean":"2024-04-25T12:37:21.912070+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:21.912070+0000","last_undegraded":"2024-04-25T12:37:21.912070+0000","last_fullsized":"2024-04-25T12:37:21.912070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:49.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403124+0000","last_change":"2024-04-25T12:37:01.207603+0000","last_active":"2024-04-25T12:37:21.403124+0000","last_peered":"2024-04-25T12:37:21.403124+0000","last_clean":"2024-04-25T12:37:21.403124+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:21.403124+0000","last_undegraded":"2024-04-25T12:37:21.403124+0000","last_fullsized":"2024-04-25T12:37:21.403124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:01.832860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"75'26","reported_seq":181,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405220+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:37:21.405220+0000","last_peered":"2024-04-25T12:37:21.405220+0000","last_clean":"2024-04-25T12:37:21.405220+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:21.405220+0000","last_undegraded":"2024-04-25T12:37:21.405220+0000","last_fullsized":"2024-04-25T12:37:21.405220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37030,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568703+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:37:21.568703+0000","last_peered":"2024-04-25T12:37:21.568703+0000","last_clean":"2024-04-25T12:37:21.568703+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:21.568703+0000","last_undegraded":"2024-04-25T12:37:21.568703+0000","last_fullsized":"2024-04-25T12:37:21.568703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423387+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:37:21.423387+0000","last_peered":"2024-04-25T12:37:21.423387+0000","last_clean":"2024-04-25T12:37:21.423387+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:21.423387+0000","last_undegraded":"2024-04-25T12:37:21.423387+0000","last_fullsized":"2024-04-25T12:37:21.423387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"75'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403382+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:37:21.403382+0000","last_peered":"2024-04-25T12:37:21.403382+0000","last_clean":"2024-04-25T12:37:21.403382+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:21.403382+0000","last_undegraded":"2024-04-25T12:37:21.403382+0000","last_fullsized":"2024-04-25T12:37:21.403382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405271+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:37:21.405271+0000","last_peered":"2024-04-25T12:37:21.405271+0000","last_clean":"2024-04-25T12:37:21.405271+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:21.405271+0000","last_undegraded":"2024-04-25T12:37:21.405271+0000","last_fullsized":"2024-04-25T12:37:21.405271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908126+0000","last_change":"2024-04-25T12:37:01.212753+0000","last_active":"2024-04-25T12:37:21.908126+0000","last_peered":"2024-04-25T12:37:21.908126+0000","last_clean":"2024-04-25T12:37:21.908126+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:21.908126+0000","last_undegraded":"2024-04-25T12:37:21.908126+0000","last_fullsized":"2024-04-25T12:37:21.908126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:07.000299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911540+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:37:21.911540+0000","last_peered":"2024-04-25T12:37:21.911540+0000","last_clean":"2024-04-25T12:37:21.911540+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:21.911540+0000","last_undegraded":"2024-04-25T12:37:21.911540+0000","last_fullsized":"2024-04-25T12:37:21.911540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402408+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:37:21.402408+0000","last_peered":"2024-04-25T12:37:21.402408+0000","last_clean":"2024-04-25T12:37:21.402408+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:21.402408+0000","last_undegraded":"2024-04-25T12:37:21.402408+0000","last_fullsized":"2024-04-25T12:37:21.402408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406162+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:37:21.406162+0000","last_peered":"2024-04-25T12:37:21.406162+0000","last_clean":"2024-04-25T12:37:21.406162+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:21.406162+0000","last_undegraded":"2024-04-25T12:37:21.406162+0000","last_fullsized":"2024-04-25T12:37:21.406162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913386+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:37:21.913386+0000","last_peered":"2024-04-25T12:37:21.913386+0000","last_clean":"2024-04-25T12:37:21.913386+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:21.913386+0000","last_undegraded":"2024-04-25T12:37:21.913386+0000","last_fullsized":"2024-04-25T12:37:21.913386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568895+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:37:21.568895+0000","last_peered":"2024-04-25T12:37:21.568895+0000","last_clean":"2024-04-25T12:37:21.568895+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:21.568895+0000","last_undegraded":"2024-04-25T12:37:21.568895+0000","last_fullsized":"2024-04-25T12:37:21.568895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568918+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:37:21.568918+0000","last_peered":"2024-04-25T12:37:21.568918+0000","last_clean":"2024-04-25T12:37:21.568918+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:21.568918+0000","last_undegraded":"2024-04-25T12:37:21.568918+0000","last_fullsized":"2024-04-25T12:37:21.568918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569073+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:37:21.569073+0000","last_peered":"2024-04-25T12:37:21.569073+0000","last_clean":"2024-04-25T12:37:21.569073+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:21.569073+0000","last_undegraded":"2024-04-25T12:37:21.569073+0000","last_fullsized":"2024-04-25T12:37:21.569073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423322+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:37:21.423322+0000","last_peered":"2024-04-25T12:37:21.423322+0000","last_clean":"2024-04-25T12:37:21.423322+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:21.423322+0000","last_undegraded":"2024-04-25T12:37:21.423322+0000","last_fullsized":"2024-04-25T12:37:21.423322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424396+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:37:21.424396+0000","last_peered":"2024-04-25T12:37:21.424396+0000","last_clean":"2024-04-25T12:37:21.424396+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:21.424396+0000","last_undegraded":"2024-04-25T12:37:21.424396+0000","last_fullsized":"2024-04-25T12:37:21.424396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907318+0000","last_change":"2024-04-25T12:37:01.204788+0000","last_active":"2024-04-25T12:37:21.907318+0000","last_peered":"2024-04-25T12:37:21.907318+0000","last_clean":"2024-04-25T12:37:21.907318+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:21.907318+0000","last_undegraded":"2024-04-25T12:37:21.907318+0000","last_fullsized":"2024-04-25T12:37:21.907318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:14.535103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908141+0000","last_change":"2024-04-25T12:37:01.217177+0000","last_active":"2024-04-25T12:37:21.908141+0000","last_peered":"2024-04-25T12:37:21.908141+0000","last_clean":"2024-04-25T12:37:21.908141+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:21.908141+0000","last_undegraded":"2024-04-25T12:37:21.908141+0000","last_fullsized":"2024-04-25T12:37:21.908141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:00.819525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423908+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:37:21.423908+0000","last_peered":"2024-04-25T12:37:21.423908+0000","last_clean":"2024-04-25T12:37:21.423908+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:21.423908+0000","last_undegraded":"2024-04-25T12:37:21.423908+0000","last_fullsized":"2024-04-25T12:37:21.423908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568987+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:37:21.568987+0000","last_peered":"2024-04-25T12:37:21.568987+0000","last_clean":"2024-04-25T12:37:21.568987+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:21.568987+0000","last_undegraded":"2024-04-25T12:37:21.568987+0000","last_fullsized":"2024-04-25T12:37:21.568987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569215+0000","last_change":"2024-04-25T12:37:01.209143+0000","last_active":"2024-04-25T12:37:21.569215+0000","last_peered":"2024-04-25T12:37:21.569215+0000","last_clean":"2024-04-25T12:37:21.569215+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:21.569215+0000","last_undegraded":"2024-04-25T12:37:21.569215+0000","last_fullsized":"2024-04-25T12:37:21.569215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:52.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907606+0000","last_change":"2024-04-25T12:37:01.223997+0000","last_active":"2024-04-25T12:37:21.907606+0000","last_peered":"2024-04-25T12:37:21.907606+0000","last_clean":"2024-04-25T12:37:21.907606+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:21.907606+0000","last_undegraded":"2024-04-25T12:37:21.907606+0000","last_fullsized":"2024-04-25T12:37:21.907606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:48.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911293+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:37:21.911293+0000","last_peered":"2024-04-25T12:37:21.911293+0000","last_clean":"2024-04-25T12:37:21.911293+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:21.911293+0000","last_undegraded":"2024-04-25T12:37:21.911293+0000","last_fullsized":"2024-04-25T12:37:21.911293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403285+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:37:21.403285+0000","last_peered":"2024-04-25T12:37:21.403285+0000","last_clean":"2024-04-25T12:37:21.403285+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:21.403285+0000","last_undegraded":"2024-04-25T12:37:21.403285+0000","last_fullsized":"2024-04-25T12:37:21.403285+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404558+0000","last_change":"2024-04-25T12:37:01.224898+0000","last_active":"2024-04-25T12:37:21.404558+0000","last_peered":"2024-04-25T12:37:21.404558+0000","last_clean":"2024-04-25T12:37:21.404558+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:21.404558+0000","last_undegraded":"2024-04-25T12:37:21.404558+0000","last_fullsized":"2024-04-25T12:37:21.404558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:18.991070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402556+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:37:21.402556+0000","last_peered":"2024-04-25T12:37:21.402556+0000","last_clean":"2024-04-25T12:37:21.402556+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:21.402556+0000","last_undegraded":"2024-04-25T12:37:21.402556+0000","last_fullsized":"2024-04-25T12:37:21.402556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422760+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:37:21.422760+0000","last_peered":"2024-04-25T12:37:21.422760+0000","last_clean":"2024-04-25T12:37:21.422760+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:21.422760+0000","last_undegraded":"2024-04-25T12:37:21.422760+0000","last_fullsized":"2024-04-25T12:37:21.422760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907860+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:37:21.907860+0000","last_peered":"2024-04-25T12:37:21.907860+0000","last_clean":"2024-04-25T12:37:21.907860+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:21.907860+0000","last_undegraded":"2024-04-25T12:37:21.907860+0000","last_fullsized":"2024-04-25T12:37:21.907860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908709+0000","last_change":"2024-04-25T12:37:01.228534+0000","last_active":"2024-04-25T12:37:21.908709+0000","last_peered":"2024-04-25T12:37:21.908709+0000","last_clean":"2024-04-25T12:37:21.908709+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:21.908709+0000","last_undegraded":"2024-04-25T12:37:21.908709+0000","last_fullsized":"2024-04-25T12:37:21.908709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:39.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907599+0000","last_change":"2024-04-25T12:37:01.210033+0000","last_active":"2024-04-25T12:37:21.907599+0000","last_peered":"2024-04-25T12:37:21.907599+0000","last_clean":"2024-04-25T12:37:21.907599+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:21.907599+0000","last_undegraded":"2024-04-25T12:37:21.907599+0000","last_fullsized":"2024-04-25T12:37:21.907599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:50.639901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406009+0000","last_change":"2024-04-25T12:37:01.232027+0000","last_active":"2024-04-25T12:37:21.406009+0000","last_peered":"2024-04-25T12:37:21.406009+0000","last_clean":"2024-04-25T12:37:21.406009+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:21.406009+0000","last_undegraded":"2024-04-25T12:37:21.406009+0000","last_fullsized":"2024-04-25T12:37:21.406009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908213+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:37:21.908213+0000","last_peered":"2024-04-25T12:37:21.908213+0000","last_clean":"2024-04-25T12:37:21.908213+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:21.908213+0000","last_undegraded":"2024-04-25T12:37:21.908213+0000","last_fullsized":"2024-04-25T12:37:21.908213+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423227+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:37:21.423227+0000","last_peered":"2024-04-25T12:37:21.423227+0000","last_clean":"2024-04-25T12:37:21.423227+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:21.423227+0000","last_undegraded":"2024-04-25T12:37:21.423227+0000","last_fullsized":"2024-04-25T12:37:21.423227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907222+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:37:21.907222+0000","last_peered":"2024-04-25T12:37:21.907222+0000","last_clean":"2024-04-25T12:37:21.907222+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:21.907222+0000","last_undegraded":"2024-04-25T12:37:21.907222+0000","last_fullsized":"2024-04-25T12:37:21.907222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911205+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:37:21.911205+0000","last_peered":"2024-04-25T12:37:21.911205+0000","last_clean":"2024-04-25T12:37:21.911205+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:21.911205+0000","last_undegraded":"2024-04-25T12:37:21.911205+0000","last_fullsized":"2024-04-25T12:37:21.911205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568777+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:37:21.568777+0000","last_peered":"2024-04-25T12:37:21.568777+0000","last_clean":"2024-04-25T12:37:21.568777+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:21.568777+0000","last_undegraded":"2024-04-25T12:37:21.568777+0000","last_fullsized":"2024-04-25T12:37:21.568777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910725+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:37:21.910725+0000","last_peered":"2024-04-25T12:37:21.910725+0000","last_clean":"2024-04-25T12:37:21.910725+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:21.910725+0000","last_undegraded":"2024-04-25T12:37:21.910725+0000","last_fullsized":"2024-04-25T12:37:21.910725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424005+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:37:21.424005+0000","last_peered":"2024-04-25T12:37:21.424005+0000","last_clean":"2024-04-25T12:37:21.424005+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:21.424005+0000","last_undegraded":"2024-04-25T12:37:21.424005+0000","last_fullsized":"2024-04-25T12:37:21.424005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403400+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:37:21.403400+0000","last_peered":"2024-04-25T12:37:21.403400+0000","last_clean":"2024-04-25T12:37:21.403400+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:21.403400+0000","last_undegraded":"2024-04-25T12:37:21.403400+0000","last_fullsized":"2024-04-25T12:37:21.403400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424575+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:37:21.424575+0000","last_peered":"2024-04-25T12:37:21.424575+0000","last_clean":"2024-04-25T12:37:21.424575+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:21.424575+0000","last_undegraded":"2024-04-25T12:37:21.424575+0000","last_fullsized":"2024-04-25T12:37:21.424575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423084+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:37:21.423084+0000","last_peered":"2024-04-25T12:37:21.423084+0000","last_clean":"2024-04-25T12:37:21.423084+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:21.423084+0000","last_undegraded":"2024-04-25T12:37:21.423084+0000","last_fullsized":"2024-04-25T12:37:21.423084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"70'1","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908366+0000","last_change":"2024-04-25T12:37:01.212249+0000","last_active":"2024-04-25T12:37:21.908366+0000","last_peered":"2024-04-25T12:37:21.908366+0000","last_clean":"2024-04-25T12:37:21.908366+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:21.908366+0000","last_undegraded":"2024-04-25T12:37:21.908366+0000","last_fullsized":"2024-04-25T12:37:21.908366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:47.658434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908490+0000","last_change":"2024-04-25T12:37:01.216745+0000","last_active":"2024-04-25T12:37:21.908490+0000","last_peered":"2024-04-25T12:37:21.908490+0000","last_clean":"2024-04-25T12:37:21.908490+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:21.908490+0000","last_undegraded":"2024-04-25T12:37:21.908490+0000","last_fullsized":"2024-04-25T12:37:21.908490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910517+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:37:21.910517+0000","last_peered":"2024-04-25T12:37:21.910517+0000","last_clean":"2024-04-25T12:37:21.910517+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:21.910517+0000","last_undegraded":"2024-04-25T12:37:21.910517+0000","last_fullsized":"2024-04-25T12:37:21.910517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910107+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:37:21.910107+0000","last_peered":"2024-04-25T12:37:21.910107+0000","last_clean":"2024-04-25T12:37:21.910107+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:21.910107+0000","last_undegraded":"2024-04-25T12:37:21.910107+0000","last_fullsized":"2024-04-25T12:37:21.910107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911235+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:37:21.911235+0000","last_peered":"2024-04-25T12:37:21.911235+0000","last_clean":"2024-04-25T12:37:21.911235+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:21.911235+0000","last_undegraded":"2024-04-25T12:37:21.911235+0000","last_fullsized":"2024-04-25T12:37:21.911235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913288+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:37:21.913288+0000","last_peered":"2024-04-25T12:37:21.913288+0000","last_clean":"2024-04-25T12:37:21.913288+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:21.913288+0000","last_undegraded":"2024-04-25T12:37:21.913288+0000","last_fullsized":"2024-04-25T12:37:21.913288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909742+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:37:21.909742+0000","last_peered":"2024-04-25T12:37:21.909742+0000","last_clean":"2024-04-25T12:37:21.909742+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:21.909742+0000","last_undegraded":"2024-04-25T12:37:21.909742+0000","last_fullsized":"2024-04-25T12:37:21.909742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405037+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:37:21.405037+0000","last_peered":"2024-04-25T12:37:21.405037+0000","last_clean":"2024-04-25T12:37:21.405037+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:21.405037+0000","last_undegraded":"2024-04-25T12:37:21.405037+0000","last_fullsized":"2024-04-25T12:37:21.405037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911087+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:37:21.911087+0000","last_peered":"2024-04-25T12:37:21.911087+0000","last_clean":"2024-04-25T12:37:21.911087+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:21.911087+0000","last_undegraded":"2024-04-25T12:37:21.911087+0000","last_fullsized":"2024-04-25T12:37:21.911087+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405035+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:37:21.405035+0000","last_peered":"2024-04-25T12:37:21.405035+0000","last_clean":"2024-04-25T12:37:21.405035+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:21.405035+0000","last_undegraded":"2024-04-25T12:37:21.405035+0000","last_fullsized":"2024-04-25T12:37:21.405035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907971+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:37:21.907971+0000","last_peered":"2024-04-25T12:37:21.907971+0000","last_clean":"2024-04-25T12:37:21.907971+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:21.907971+0000","last_undegraded":"2024-04-25T12:37:21.907971+0000","last_fullsized":"2024-04-25T12:37:21.907971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908194+0000","last_change":"2024-04-25T12:37:01.215943+0000","last_active":"2024-04-25T12:37:21.908194+0000","last_peered":"2024-04-25T12:37:21.908194+0000","last_clean":"2024-04-25T12:37:21.908194+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:21.908194+0000","last_undegraded":"2024-04-25T12:37:21.908194+0000","last_fullsized":"2024-04-25T12:37:21.908194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.518673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911529+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:37:21.911529+0000","last_peered":"2024-04-25T12:37:21.911529+0000","last_clean":"2024-04-25T12:37:21.911529+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:21.911529+0000","last_undegraded":"2024-04-25T12:37:21.911529+0000","last_fullsized":"2024-04-25T12:37:21.911529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41052,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":166,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82104,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462931,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29336,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342504,"statfs":{"total":96636764160,"available":96606724096,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":1207327,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1142784,"data_stored":616007,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37646,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38598,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:37:26.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:37:26.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.866+0000 7f062da00640 1 Processor -- start 2024-04-25T12:37:26.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.867+0000 7f062da00640 1 -- start start 2024-04-25T12:37:26.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.868+0000 7f062da00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f06280cdc60 0x7f06280b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.868+0000 7f062da00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f062815cf60 0x7f06280d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.868+0000 7f062da00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062815fdd0 con 0x7f06280d1ad0 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.868+0000 7f062da00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f06281600b0 con 0x7f062815cf60 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.868+0000 7f062da00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0628160390 con 0x7f06280cdc60 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f062d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f062815cf60 0x7f06280d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f062d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f062815cf60 0x7f06280d1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41406/0 (socket says 172.21.15.123:41406) 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f062d000640 1 -- 172.21.15.123:0/1162340923 learned_addr learned my addr 172.21.15.123:0/1162340923 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:26.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f0627e00640 1 --2- 172.21.15.123:0/1162340923 >> v2:172.21.15.153:3300/0 conn(0x7f06280cdc60 0x7f06280b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f062d000640 1 -- 172.21.15.123:0/1162340923 >> v2:172.21.15.153:3300/0 conn(0x7f06280cdc60 msgr2=0x7f06280b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:26.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f062d000640 1 --2- 172.21.15.123:0/1162340923 >> v2:172.21.15.153:3300/0 conn(0x7f06280cdc60 0x7f06280b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f062d000640 1 -- 172.21.15.123:0/1162340923 >> v1:172.21.15.123:6789/0 conn(0x7f06280d1ad0 legacy=0x7f062815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.869+0000 7f062d000640 1 -- 172.21.15.123:0/1162340923 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0628160670 con 0x7f062815cf60 2024-04-25T12:37:26.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.870+0000 7f062d000640 1 --2- 172.21.15.123:0/1162340923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f062815cf60 0x7f06280d1ee0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f06280cc1a0 tx=0x7f062002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a2fb1353994cff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:26.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.870+0000 7f0626a00640 1 -- 172.21.15.123:0/1162340923 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620036070 con 0x7f062815cf60 2024-04-25T12:37:26.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f0626a00640 1 -- 172.21.15.123:0/1162340923 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0620031600 con 0x7f062815cf60 2024-04-25T12:37:26.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f0626a00640 1 -- 172.21.15.123:0/1162340923 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200319d0 con 0x7f062815cf60 2024-04-25T12:37:26.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 -- 172.21.15.123:0/1162340923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f062815cf60 msgr2=0x7f06280d1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 --2- 172.21.15.123:0/1162340923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f062815cf60 0x7f06280d1ee0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f06280cc1a0 tx=0x7f062002a730 comp rx=0 tx=0).stop 2024-04-25T12:37:26.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 -- 172.21.15.123:0/1162340923 shutdown_connections 2024-04-25T12:37:26.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 --2- 172.21.15.123:0/1162340923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f062815cf60 0x7f06280d1ee0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 --2- 172.21.15.123:0/1162340923 >> v2:172.21.15.153:3300/0 conn(0x7f06280cdc60 0x7f06280b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 -- 172.21.15.123:0/1162340923 >> 172.21.15.123:0/1162340923 conn(0x7f062808d840 msgr2=0x7f062808dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:26.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 -- 172.21.15.123:0/1162340923 shutdown_connections 2024-04-25T12:37:26.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.871+0000 7f062da00640 1 -- 172.21.15.123:0/1162340923 wait complete. 2024-04-25T12:37:26.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.874+0000 7f062da00640 1 Processor -- start 2024-04-25T12:37:26.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.874+0000 7f062da00640 1 -- start start 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f062da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06280cdc60 0x7f06280b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f062da00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06280d1ad0 0x7f062815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f062da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f062815cf60 0x7f0628156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f062da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f062815bd30 con 0x7f06280cdc60 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f062da00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f062815bbb0 con 0x7f062815cf60 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f062da00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f062815beb0 con 0x7f06280d1ad0 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f0627e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06280cdc60 0x7f06280b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f0627e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06280cdc60 0x7f06280b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41416/0 (socket says 172.21.15.123:41416) 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f0627e00640 1 -- 172.21.15.123:0/3473299879 learned_addr learned my addr 172.21.15.123:0/3473299879 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f062d000640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f062815cf60 0x7f0628156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.875+0000 7f0627400640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06280d1ad0 0x7f062815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f0627e00640 1 -- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06280d1ad0 msgr2=0x7f062815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f0627e00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06280d1ad0 0x7f062815f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f0627e00640 1 -- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f062815cf60 msgr2=0x7f0628156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f0627e00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f062815cf60 0x7f0628156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f0627e00640 1 -- 172.21.15.123:0/3473299879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06280b2240 con 0x7f06280cdc60 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f0627e00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06280cdc60 0x7f06280b1d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f061800e8a0 tx=0x7f061800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f061fe00640 1 -- 172.21.15.123:0/3473299879 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618017070 con 0x7f06280cdc60 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f061fe00640 1 -- 172.21.15.123:0/3473299879 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061800c940 con 0x7f06280cdc60 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06281f5b20 con 0x7f06280cdc60 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.876+0000 7f061fe00640 1 -- 172.21.15.123:0/3473299879 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061800cd30 con 0x7f06280cdc60 2024-04-25T12:37:26.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.877+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281f60e0 con 0x7f06280cdc60 2024-04-25T12:37:26.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.878+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0628076da0 con 0x7f06280cdc60 2024-04-25T12:37:26.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.879+0000 7f061fe00640 1 -- 172.21.15.123:0/3473299879 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f061801a070 con 0x7f06280cdc60 2024-04-25T12:37:26.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.880+0000 7f061fe00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f06040744b0 0x7f06040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:26.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.880+0000 7f061fe00640 1 -- 172.21.15.123:0/3473299879 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f0618095990 con 0x7f06280cdc60 2024-04-25T12:37:26.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.880+0000 7f0627400640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f06040744b0 0x7f06040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:26.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.881+0000 7f0627400640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f06040744b0 0x7f06040769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f062808dfa0 tx=0x7f0610009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:26.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:26.884+0000 7f061fe00640 1 -- 172.21.15.123:0/3473299879 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f061805f590 con 0x7f06280cdc60 2024-04-25T12:37:27.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.065+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06280cda40 con 0x7f06040744b0 2024-04-25T12:37:27.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.075+0000 7f061fe00640 1 -- 172.21.15.123:0/3473299879 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333131 (secure 0 0 0) 0x7f06280cda40 con 0x7f06040744b0 2024-04-25T12:37:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:37:27.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f06040744b0 msgr2=0x7f06040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f06040744b0 0x7f06040769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f062808dfa0 tx=0x7f0610009040 comp rx=0 tx=0).stop 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06280cdc60 msgr2=0x7f06280b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06280cdc60 0x7f06280b1d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f061800e8a0 tx=0x7f061800ed60 comp rx=0 tx=0).stop 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 shutdown_connections 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f06040744b0 0x7f06040769c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f062815cf60 0x7f0628156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06280d1ad0 0x7f062815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 --2- 172.21.15.123:0/3473299879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06280cdc60 0x7f06280b1d00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 >> 172.21.15.123:0/3473299879 conn(0x7f062808d840 msgr2=0x7f06280b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 shutdown_connections 2024-04-25T12:37:27.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.081+0000 7f062da00640 1 -- 172.21.15.123:0/3473299879 wait complete. 2024-04-25T12:37:27.105 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-04-25T12:37:26.841148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631420,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":269,"num_write_kb":1725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234616,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":225579,"kb_avail":754740104,"statfs":{"total":773094113280,"available":772853866496,"internally_reserved":0,"allocated":9093120,"data_stored":5889728,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":230993875},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.467010"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569452+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:37:21.569452+0000","last_peered":"2024-04-25T12:37:21.569452+0000","last_clean":"2024-04-25T12:37:21.569452+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:21.569452+0000","last_undegraded":"2024-04-25T12:37:21.569452+0000","last_fullsized":"2024-04-25T12:37:21.569452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908252+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:37:21.908252+0000","last_peered":"2024-04-25T12:37:21.908252+0000","last_clean":"2024-04-25T12:37:21.908252+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:21.908252+0000","last_undegraded":"2024-04-25T12:37:21.908252+0000","last_fullsized":"2024-04-25T12:37:21.908252+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402490+0000","last_change":"2024-04-25T12:37:01.222445+0000","last_active":"2024-04-25T12:37:21.402490+0000","last_peered":"2024-04-25T12:37:21.402490+0000","last_clean":"2024-04-25T12:37:21.402490+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:21.402490+0000","last_undegraded":"2024-04-25T12:37:21.402490+0000","last_fullsized":"2024-04-25T12:37:21.402490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907880+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:37:21.907880+0000","last_peered":"2024-04-25T12:37:21.907880+0000","last_clean":"2024-04-25T12:37:21.907880+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:21.907880+0000","last_undegraded":"2024-04-25T12:37:21.907880+0000","last_fullsized":"2024-04-25T12:37:21.907880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908265+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:37:21.908265+0000","last_peered":"2024-04-25T12:37:21.908265+0000","last_clean":"2024-04-25T12:37:21.908265+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:21.908265+0000","last_undegraded":"2024-04-25T12:37:21.908265+0000","last_fullsized":"2024-04-25T12:37:21.908265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404380+0000","last_change":"2024-04-25T12:37:01.216562+0000","last_active":"2024-04-25T12:37:21.404380+0000","last_peered":"2024-04-25T12:37:21.404380+0000","last_clean":"2024-04-25T12:37:21.404380+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:21.404380+0000","last_undegraded":"2024-04-25T12:37:21.404380+0000","last_fullsized":"2024-04-25T12:37:21.404380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910309+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:37:21.910309+0000","last_peered":"2024-04-25T12:37:21.910309+0000","last_clean":"2024-04-25T12:37:21.910309+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:21.910309+0000","last_undegraded":"2024-04-25T12:37:21.910309+0000","last_fullsized":"2024-04-25T12:37:21.910309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912549+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:37:21.912549+0000","last_peered":"2024-04-25T12:37:21.912549+0000","last_clean":"2024-04-25T12:37:21.912549+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:21.912549+0000","last_undegraded":"2024-04-25T12:37:21.912549+0000","last_fullsized":"2024-04-25T12:37:21.912549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405114+0000","last_change":"2024-04-25T12:37:01.226633+0000","last_active":"2024-04-25T12:37:21.405114+0000","last_peered":"2024-04-25T12:37:21.405114+0000","last_clean":"2024-04-25T12:37:21.405114+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:21.405114+0000","last_undegraded":"2024-04-25T12:37:21.405114+0000","last_fullsized":"2024-04-25T12:37:21.405114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.771833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910815+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:37:21.910815+0000","last_peered":"2024-04-25T12:37:21.910815+0000","last_clean":"2024-04-25T12:37:21.910815+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:21.910815+0000","last_undegraded":"2024-04-25T12:37:21.910815+0000","last_fullsized":"2024-04-25T12:37:21.910815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423689+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:37:21.423689+0000","last_peered":"2024-04-25T12:37:21.423689+0000","last_clean":"2024-04-25T12:37:21.423689+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:21.423689+0000","last_undegraded":"2024-04-25T12:37:21.423689+0000","last_fullsized":"2024-04-25T12:37:21.423689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404697+0000","last_change":"2024-04-25T12:37:01.221650+0000","last_active":"2024-04-25T12:37:21.404697+0000","last_peered":"2024-04-25T12:37:21.404697+0000","last_clean":"2024-04-25T12:37:21.404697+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:21.404697+0000","last_undegraded":"2024-04-25T12:37:21.404697+0000","last_fullsized":"2024-04-25T12:37:21.404697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422862+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:37:21.422862+0000","last_peered":"2024-04-25T12:37:21.422862+0000","last_clean":"2024-04-25T12:37:21.422862+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:21.422862+0000","last_undegraded":"2024-04-25T12:37:21.422862+0000","last_fullsized":"2024-04-25T12:37:21.422862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423223+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:37:21.423223+0000","last_peered":"2024-04-25T12:37:21.423223+0000","last_clean":"2024-04-25T12:37:21.423223+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:21.423223+0000","last_undegraded":"2024-04-25T12:37:21.423223+0000","last_fullsized":"2024-04-25T12:37:21.423223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909732+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:37:21.909732+0000","last_peered":"2024-04-25T12:37:21.909732+0000","last_clean":"2024-04-25T12:37:21.909732+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:21.909732+0000","last_undegraded":"2024-04-25T12:37:21.909732+0000","last_fullsized":"2024-04-25T12:37:21.909732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402959+0000","last_change":"2024-04-25T12:37:01.209490+0000","last_active":"2024-04-25T12:37:21.402959+0000","last_peered":"2024-04-25T12:37:21.402959+0000","last_clean":"2024-04-25T12:37:21.402959+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:21.402959+0000","last_undegraded":"2024-04-25T12:37:21.402959+0000","last_fullsized":"2024-04-25T12:37:21.402959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:46.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908543+0000","last_change":"2024-04-25T12:37:01.230485+0000","last_active":"2024-04-25T12:37:21.908543+0000","last_peered":"2024-04-25T12:37:21.908543+0000","last_clean":"2024-04-25T12:37:21.908543+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:21.908543+0000","last_undegraded":"2024-04-25T12:37:21.908543+0000","last_fullsized":"2024-04-25T12:37:21.908543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:47.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909989+0000","last_change":"2024-04-25T12:37:01.222195+0000","last_active":"2024-04-25T12:37:21.909989+0000","last_peered":"2024-04-25T12:37:21.909989+0000","last_clean":"2024-04-25T12:37:21.909989+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:21.909989+0000","last_undegraded":"2024-04-25T12:37:21.909989+0000","last_fullsized":"2024-04-25T12:37:21.909989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:35.550835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568700+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:37:21.568700+0000","last_peered":"2024-04-25T12:37:21.568700+0000","last_clean":"2024-04-25T12:37:21.568700+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:21.568700+0000","last_undegraded":"2024-04-25T12:37:21.568700+0000","last_fullsized":"2024-04-25T12:37:21.568700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569070+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:37:21.569070+0000","last_peered":"2024-04-25T12:37:21.569070+0000","last_clean":"2024-04-25T12:37:21.569070+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:21.569070+0000","last_undegraded":"2024-04-25T12:37:21.569070+0000","last_fullsized":"2024-04-25T12:37:21.569070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403488+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:37:21.403488+0000","last_peered":"2024-04-25T12:37:21.403488+0000","last_clean":"2024-04-25T12:37:21.403488+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:21.403488+0000","last_undegraded":"2024-04-25T12:37:21.403488+0000","last_fullsized":"2024-04-25T12:37:21.403488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422941+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:37:21.422941+0000","last_peered":"2024-04-25T12:37:21.422941+0000","last_clean":"2024-04-25T12:37:21.422941+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:21.422941+0000","last_undegraded":"2024-04-25T12:37:21.422941+0000","last_fullsized":"2024-04-25T12:37:21.422941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422292+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:37:21.422292+0000","last_peered":"2024-04-25T12:37:21.422292+0000","last_clean":"2024-04-25T12:37:21.422292+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:37:21.422292+0000","last_undegraded":"2024-04-25T12:37:21.422292+0000","last_fullsized":"2024-04-25T12:37:21.422292+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405968+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:37:21.405968+0000","last_peered":"2024-04-25T12:37:21.405968+0000","last_clean":"2024-04-25T12:37:21.405968+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:21.405968+0000","last_undegraded":"2024-04-25T12:37:21.405968+0000","last_fullsized":"2024-04-25T12:37:21.405968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569094+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:37:21.569094+0000","last_peered":"2024-04-25T12:37:21.569094+0000","last_clean":"2024-04-25T12:37:21.569094+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:21.569094+0000","last_undegraded":"2024-04-25T12:37:21.569094+0000","last_fullsized":"2024-04-25T12:37:21.569094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424634+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:37:21.424634+0000","last_peered":"2024-04-25T12:37:21.424634+0000","last_clean":"2024-04-25T12:37:21.424634+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:21.424634+0000","last_undegraded":"2024-04-25T12:37:21.424634+0000","last_fullsized":"2024-04-25T12:37:21.424634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404952+0000","last_change":"2024-04-25T12:37:01.226160+0000","last_active":"2024-04-25T12:37:21.404952+0000","last_peered":"2024-04-25T12:37:21.404952+0000","last_clean":"2024-04-25T12:37:21.404952+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:21.404952+0000","last_undegraded":"2024-04-25T12:37:21.404952+0000","last_fullsized":"2024-04-25T12:37:21.404952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:43.475095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569434+0000","last_change":"2024-04-25T12:37:01.206319+0000","last_active":"2024-04-25T12:37:21.569434+0000","last_peered":"2024-04-25T12:37:21.569434+0000","last_clean":"2024-04-25T12:37:21.569434+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:21.569434+0000","last_undegraded":"2024-04-25T12:37:21.569434+0000","last_fullsized":"2024-04-25T12:37:21.569434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"74'12","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569349+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:37:21.569349+0000","last_peered":"2024-04-25T12:37:21.569349+0000","last_clean":"2024-04-25T12:37:21.569349+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:21.569349+0000","last_undegraded":"2024-04-25T12:37:21.569349+0000","last_fullsized":"2024-04-25T12:37:21.569349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568614+0000","last_change":"2024-04-25T12:37:01.225992+0000","last_active":"2024-04-25T12:37:21.568614+0000","last_peered":"2024-04-25T12:37:21.568614+0000","last_clean":"2024-04-25T12:37:21.568614+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:21.568614+0000","last_undegraded":"2024-04-25T12:37:21.568614+0000","last_fullsized":"2024-04-25T12:37:21.568614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:05.029213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908795+0000","last_change":"2024-04-25T12:37:01.208764+0000","last_active":"2024-04-25T12:37:21.908795+0000","last_peered":"2024-04-25T12:37:21.908795+0000","last_clean":"2024-04-25T12:37:21.908795+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:21.908795+0000","last_undegraded":"2024-04-25T12:37:21.908795+0000","last_fullsized":"2024-04-25T12:37:21.908795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:38.538923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424069+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:37:21.424069+0000","last_peered":"2024-04-25T12:37:21.424069+0000","last_clean":"2024-04-25T12:37:21.424069+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:21.424069+0000","last_undegraded":"2024-04-25T12:37:21.424069+0000","last_fullsized":"2024-04-25T12:37:21.424069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908596+0000","last_change":"2024-04-25T12:37:01.225006+0000","last_active":"2024-04-25T12:37:21.908596+0000","last_peered":"2024-04-25T12:37:21.908596+0000","last_clean":"2024-04-25T12:37:21.908596+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:21.908596+0000","last_undegraded":"2024-04-25T12:37:21.908596+0000","last_fullsized":"2024-04-25T12:37:21.908596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405700+0000","last_change":"2024-04-25T12:37:01.222179+0000","last_active":"2024-04-25T12:37:21.405700+0000","last_peered":"2024-04-25T12:37:21.405700+0000","last_clean":"2024-04-25T12:37:21.405700+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:21.405700+0000","last_undegraded":"2024-04-25T12:37:21.405700+0000","last_fullsized":"2024-04-25T12:37:21.405700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907696+0000","last_change":"2024-04-25T12:37:01.211908+0000","last_active":"2024-04-25T12:37:21.907696+0000","last_peered":"2024-04-25T12:37:21.907696+0000","last_clean":"2024-04-25T12:37:21.907696+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:21.907696+0000","last_undegraded":"2024-04-25T12:37:21.907696+0000","last_fullsized":"2024-04-25T12:37:21.907696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:23.523737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908482+0000","last_change":"2024-04-25T12:37:01.229773+0000","last_active":"2024-04-25T12:37:21.908482+0000","last_peered":"2024-04-25T12:37:21.908482+0000","last_clean":"2024-04-25T12:37:21.908482+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:21.908482+0000","last_undegraded":"2024-04-25T12:37:21.908482+0000","last_fullsized":"2024-04-25T12:37:21.908482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:55.680417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402818+0000","last_change":"2024-04-25T12:37:01.223560+0000","last_active":"2024-04-25T12:37:21.402818+0000","last_peered":"2024-04-25T12:37:21.402818+0000","last_clean":"2024-04-25T12:37:21.402818+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:21.402818+0000","last_undegraded":"2024-04-25T12:37:21.402818+0000","last_fullsized":"2024-04-25T12:37:21.402818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:18.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424227+0000","last_change":"2024-04-25T12:37:01.227566+0000","last_active":"2024-04-25T12:37:21.424227+0000","last_peered":"2024-04-25T12:37:21.424227+0000","last_clean":"2024-04-25T12:37:21.424227+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:37:21.424227+0000","last_undegraded":"2024-04-25T12:37:21.424227+0000","last_fullsized":"2024-04-25T12:37:21.424227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:43.265214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908845+0000","last_change":"2024-04-25T12:37:01.226991+0000","last_active":"2024-04-25T12:37:21.908845+0000","last_peered":"2024-04-25T12:37:21.908845+0000","last_clean":"2024-04-25T12:37:21.908845+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:21.908845+0000","last_undegraded":"2024-04-25T12:37:21.908845+0000","last_fullsized":"2024-04-25T12:37:21.908845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:23.065480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907219+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:37:21.907219+0000","last_peered":"2024-04-25T12:37:21.907219+0000","last_clean":"2024-04-25T12:37:21.907219+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:21.907219+0000","last_undegraded":"2024-04-25T12:37:21.907219+0000","last_fullsized":"2024-04-25T12:37:21.907219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'7","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402648+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:37:21.402648+0000","last_peered":"2024-04-25T12:37:21.402648+0000","last_clean":"2024-04-25T12:37:21.402648+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:21.402648+0000","last_undegraded":"2024-04-25T12:37:21.402648+0000","last_fullsized":"2024-04-25T12:37:21.402648+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907983+0000","last_change":"2024-04-25T12:37:01.222254+0000","last_active":"2024-04-25T12:37:21.907983+0000","last_peered":"2024-04-25T12:37:21.907983+0000","last_clean":"2024-04-25T12:37:21.907983+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:21.907983+0000","last_undegraded":"2024-04-25T12:37:21.907983+0000","last_fullsized":"2024-04-25T12:37:21.907983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908045+0000","last_change":"2024-04-25T12:37:01.224799+0000","last_active":"2024-04-25T12:37:21.908045+0000","last_peered":"2024-04-25T12:37:21.908045+0000","last_clean":"2024-04-25T12:37:21.908045+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:21.908045+0000","last_undegraded":"2024-04-25T12:37:21.908045+0000","last_fullsized":"2024-04-25T12:37:21.908045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:24.737805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423362+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:37:21.423362+0000","last_peered":"2024-04-25T12:37:21.423362+0000","last_clean":"2024-04-25T12:37:21.423362+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:21.423362+0000","last_undegraded":"2024-04-25T12:37:21.423362+0000","last_fullsized":"2024-04-25T12:37:21.423362+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908375+0000","last_change":"2024-04-25T12:37:01.226465+0000","last_active":"2024-04-25T12:37:21.908375+0000","last_peered":"2024-04-25T12:37:21.908375+0000","last_clean":"2024-04-25T12:37:21.908375+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:21.908375+0000","last_undegraded":"2024-04-25T12:37:21.908375+0000","last_fullsized":"2024-04-25T12:37:21.908375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:26.218787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568893+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:37:21.568893+0000","last_peered":"2024-04-25T12:37:21.568893+0000","last_clean":"2024-04-25T12:37:21.568893+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:21.568893+0000","last_undegraded":"2024-04-25T12:37:21.568893+0000","last_fullsized":"2024-04-25T12:37:21.568893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423614+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:37:21.423614+0000","last_peered":"2024-04-25T12:37:21.423614+0000","last_clean":"2024-04-25T12:37:21.423614+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:21.423614+0000","last_undegraded":"2024-04-25T12:37:21.423614+0000","last_fullsized":"2024-04-25T12:37:21.423614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910887+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:37:21.910887+0000","last_peered":"2024-04-25T12:37:21.910887+0000","last_clean":"2024-04-25T12:37:21.910887+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:21.910887+0000","last_undegraded":"2024-04-25T12:37:21.910887+0000","last_fullsized":"2024-04-25T12:37:21.910887+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423841+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:37:21.423841+0000","last_peered":"2024-04-25T12:37:21.423841+0000","last_clean":"2024-04-25T12:37:21.423841+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:21.423841+0000","last_undegraded":"2024-04-25T12:37:21.423841+0000","last_fullsized":"2024-04-25T12:37:21.423841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423763+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:37:21.423763+0000","last_peered":"2024-04-25T12:37:21.423763+0000","last_clean":"2024-04-25T12:37:21.423763+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:21.423763+0000","last_undegraded":"2024-04-25T12:37:21.423763+0000","last_fullsized":"2024-04-25T12:37:21.423763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908962+0000","last_change":"2024-04-25T12:37:01.242232+0000","last_active":"2024-04-25T12:37:21.908962+0000","last_peered":"2024-04-25T12:37:21.908962+0000","last_clean":"2024-04-25T12:37:21.908962+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:21.908962+0000","last_undegraded":"2024-04-25T12:37:21.908962+0000","last_fullsized":"2024-04-25T12:37:21.908962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:59.941469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404864+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:37:21.404864+0000","last_peered":"2024-04-25T12:37:21.404864+0000","last_clean":"2024-04-25T12:37:21.404864+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:21.404864+0000","last_undegraded":"2024-04-25T12:37:21.404864+0000","last_fullsized":"2024-04-25T12:37:21.404864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912041+0000","last_change":"2024-04-25T12:37:01.220247+0000","last_active":"2024-04-25T12:37:21.912041+0000","last_peered":"2024-04-25T12:37:21.912041+0000","last_clean":"2024-04-25T12:37:21.912041+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:21.912041+0000","last_undegraded":"2024-04-25T12:37:21.912041+0000","last_fullsized":"2024-04-25T12:37:21.912041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:13.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908088+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:37:21.908088+0000","last_peered":"2024-04-25T12:37:21.908088+0000","last_clean":"2024-04-25T12:37:21.908088+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:21.908088+0000","last_undegraded":"2024-04-25T12:37:21.908088+0000","last_fullsized":"2024-04-25T12:37:21.908088+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":112,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907766+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:37:21.907766+0000","last_peered":"2024-04-25T12:37:21.907766+0000","last_clean":"2024-04-25T12:37:21.907766+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:21.907766+0000","last_undegraded":"2024-04-25T12:37:21.907766+0000","last_fullsized":"2024-04-25T12:37:21.907766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911665+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:37:21.911665+0000","last_peered":"2024-04-25T12:37:21.911665+0000","last_clean":"2024-04-25T12:37:21.911665+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:21.911665+0000","last_undegraded":"2024-04-25T12:37:21.911665+0000","last_fullsized":"2024-04-25T12:37:21.911665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569231+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:37:21.569231+0000","last_peered":"2024-04-25T12:37:21.569231+0000","last_clean":"2024-04-25T12:37:21.569231+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:21.569231+0000","last_undegraded":"2024-04-25T12:37:21.569231+0000","last_fullsized":"2024-04-25T12:37:21.569231+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422865+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:37:21.422865+0000","last_peered":"2024-04-25T12:37:21.422865+0000","last_clean":"2024-04-25T12:37:21.422865+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:21.422865+0000","last_undegraded":"2024-04-25T12:37:21.422865+0000","last_fullsized":"2024-04-25T12:37:21.422865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406286+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:37:21.406286+0000","last_peered":"2024-04-25T12:37:21.406286+0000","last_clean":"2024-04-25T12:37:21.406286+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:21.406286+0000","last_undegraded":"2024-04-25T12:37:21.406286+0000","last_fullsized":"2024-04-25T12:37:21.406286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911823+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:37:21.911823+0000","last_peered":"2024-04-25T12:37:21.911823+0000","last_clean":"2024-04-25T12:37:21.911823+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:21.911823+0000","last_undegraded":"2024-04-25T12:37:21.911823+0000","last_fullsized":"2024-04-25T12:37:21.911823+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910437+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:37:21.910437+0000","last_peered":"2024-04-25T12:37:21.910437+0000","last_clean":"2024-04-25T12:37:21.910437+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:21.910437+0000","last_undegraded":"2024-04-25T12:37:21.910437+0000","last_fullsized":"2024-04-25T12:37:21.910437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422738+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:37:21.422738+0000","last_peered":"2024-04-25T12:37:21.422738+0000","last_clean":"2024-04-25T12:37:21.422738+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:21.422738+0000","last_undegraded":"2024-04-25T12:37:21.422738+0000","last_fullsized":"2024-04-25T12:37:21.422738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"75'16","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568873+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:37:21.568873+0000","last_peered":"2024-04-25T12:37:21.568873+0000","last_clean":"2024-04-25T12:37:21.568873+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:21.568873+0000","last_undegraded":"2024-04-25T12:37:21.568873+0000","last_fullsized":"2024-04-25T12:37:21.568873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908280+0000","last_change":"2024-04-25T12:37:01.226590+0000","last_active":"2024-04-25T12:37:21.908280+0000","last_peered":"2024-04-25T12:37:21.908280+0000","last_clean":"2024-04-25T12:37:21.908280+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:21.908280+0000","last_undegraded":"2024-04-25T12:37:21.908280+0000","last_fullsized":"2024-04-25T12:37:21.908280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.581831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910537+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:37:21.910537+0000","last_peered":"2024-04-25T12:37:21.910537+0000","last_clean":"2024-04-25T12:37:21.910537+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:21.910537+0000","last_undegraded":"2024-04-25T12:37:21.910537+0000","last_fullsized":"2024-04-25T12:37:21.910537+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910045+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:37:21.910045+0000","last_peered":"2024-04-25T12:37:21.910045+0000","last_clean":"2024-04-25T12:37:21.910045+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:21.910045+0000","last_undegraded":"2024-04-25T12:37:21.910045+0000","last_fullsized":"2024-04-25T12:37:21.910045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908364+0000","last_change":"2024-04-25T12:37:01.220767+0000","last_active":"2024-04-25T12:37:21.908364+0000","last_peered":"2024-04-25T12:37:21.908364+0000","last_clean":"2024-04-25T12:37:21.908364+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:21.908364+0000","last_undegraded":"2024-04-25T12:37:21.908364+0000","last_fullsized":"2024-04-25T12:37:21.908364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:15.874258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422741+0000","last_change":"2024-04-25T12:37:01.233439+0000","last_active":"2024-04-25T12:37:21.422741+0000","last_peered":"2024-04-25T12:37:21.422741+0000","last_clean":"2024-04-25T12:37:21.422741+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:37:21.422741+0000","last_undegraded":"2024-04-25T12:37:21.422741+0000","last_fullsized":"2024-04-25T12:37:21.422741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:11.957744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424042+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:37:21.424042+0000","last_peered":"2024-04-25T12:37:21.424042+0000","last_clean":"2024-04-25T12:37:21.424042+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:21.424042+0000","last_undegraded":"2024-04-25T12:37:21.424042+0000","last_fullsized":"2024-04-25T12:37:21.424042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908903+0000","last_change":"2024-04-25T12:37:01.225914+0000","last_active":"2024-04-25T12:37:21.908903+0000","last_peered":"2024-04-25T12:37:21.908903+0000","last_clean":"2024-04-25T12:37:21.908903+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:21.908903+0000","last_undegraded":"2024-04-25T12:37:21.908903+0000","last_fullsized":"2024-04-25T12:37:21.908903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:34.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908657+0000","last_change":"2024-04-25T12:37:01.229502+0000","last_active":"2024-04-25T12:37:21.908657+0000","last_peered":"2024-04-25T12:37:21.908657+0000","last_clean":"2024-04-25T12:37:21.908657+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:21.908657+0000","last_undegraded":"2024-04-25T12:37:21.908657+0000","last_fullsized":"2024-04-25T12:37:21.908657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:25.697423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403098+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:37:21.403098+0000","last_peered":"2024-04-25T12:37:21.403098+0000","last_clean":"2024-04-25T12:37:21.403098+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:21.403098+0000","last_undegraded":"2024-04-25T12:37:21.403098+0000","last_fullsized":"2024-04-25T12:37:21.403098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912070+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:37:21.912070+0000","last_peered":"2024-04-25T12:37:21.912070+0000","last_clean":"2024-04-25T12:37:21.912070+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:21.912070+0000","last_undegraded":"2024-04-25T12:37:21.912070+0000","last_fullsized":"2024-04-25T12:37:21.912070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:49.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403124+0000","last_change":"2024-04-25T12:37:01.207603+0000","last_active":"2024-04-25T12:37:21.403124+0000","last_peered":"2024-04-25T12:37:21.403124+0000","last_clean":"2024-04-25T12:37:21.403124+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:21.403124+0000","last_undegraded":"2024-04-25T12:37:21.403124+0000","last_fullsized":"2024-04-25T12:37:21.403124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:01.832860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"75'26","reported_seq":181,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405220+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:37:21.405220+0000","last_peered":"2024-04-25T12:37:21.405220+0000","last_clean":"2024-04-25T12:37:21.405220+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:21.405220+0000","last_undegraded":"2024-04-25T12:37:21.405220+0000","last_fullsized":"2024-04-25T12:37:21.405220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37030,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568703+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:37:21.568703+0000","last_peered":"2024-04-25T12:37:21.568703+0000","last_clean":"2024-04-25T12:37:21.568703+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:21.568703+0000","last_undegraded":"2024-04-25T12:37:21.568703+0000","last_fullsized":"2024-04-25T12:37:21.568703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423387+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:37:21.423387+0000","last_peered":"2024-04-25T12:37:21.423387+0000","last_clean":"2024-04-25T12:37:21.423387+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:21.423387+0000","last_undegraded":"2024-04-25T12:37:21.423387+0000","last_fullsized":"2024-04-25T12:37:21.423387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"75'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403382+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:37:21.403382+0000","last_peered":"2024-04-25T12:37:21.403382+0000","last_clean":"2024-04-25T12:37:21.403382+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:21.403382+0000","last_undegraded":"2024-04-25T12:37:21.403382+0000","last_fullsized":"2024-04-25T12:37:21.403382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405271+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:37:21.405271+0000","last_peered":"2024-04-25T12:37:21.405271+0000","last_clean":"2024-04-25T12:37:21.405271+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:21.405271+0000","last_undegraded":"2024-04-25T12:37:21.405271+0000","last_fullsized":"2024-04-25T12:37:21.405271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908126+0000","last_change":"2024-04-25T12:37:01.212753+0000","last_active":"2024-04-25T12:37:21.908126+0000","last_peered":"2024-04-25T12:37:21.908126+0000","last_clean":"2024-04-25T12:37:21.908126+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:21.908126+0000","last_undegraded":"2024-04-25T12:37:21.908126+0000","last_fullsized":"2024-04-25T12:37:21.908126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:07.000299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911540+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:37:21.911540+0000","last_peered":"2024-04-25T12:37:21.911540+0000","last_clean":"2024-04-25T12:37:21.911540+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:21.911540+0000","last_undegraded":"2024-04-25T12:37:21.911540+0000","last_fullsized":"2024-04-25T12:37:21.911540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402408+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:37:21.402408+0000","last_peered":"2024-04-25T12:37:21.402408+0000","last_clean":"2024-04-25T12:37:21.402408+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:21.402408+0000","last_undegraded":"2024-04-25T12:37:21.402408+0000","last_fullsized":"2024-04-25T12:37:21.402408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406162+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:37:21.406162+0000","last_peered":"2024-04-25T12:37:21.406162+0000","last_clean":"2024-04-25T12:37:21.406162+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:21.406162+0000","last_undegraded":"2024-04-25T12:37:21.406162+0000","last_fullsized":"2024-04-25T12:37:21.406162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913386+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:37:21.913386+0000","last_peered":"2024-04-25T12:37:21.913386+0000","last_clean":"2024-04-25T12:37:21.913386+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:21.913386+0000","last_undegraded":"2024-04-25T12:37:21.913386+0000","last_fullsized":"2024-04-25T12:37:21.913386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568895+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:37:21.568895+0000","last_peered":"2024-04-25T12:37:21.568895+0000","last_clean":"2024-04-25T12:37:21.568895+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:21.568895+0000","last_undegraded":"2024-04-25T12:37:21.568895+0000","last_fullsized":"2024-04-25T12:37:21.568895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568918+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:37:21.568918+0000","last_peered":"2024-04-25T12:37:21.568918+0000","last_clean":"2024-04-25T12:37:21.568918+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:21.568918+0000","last_undegraded":"2024-04-25T12:37:21.568918+0000","last_fullsized":"2024-04-25T12:37:21.568918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569073+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:37:21.569073+0000","last_peered":"2024-04-25T12:37:21.569073+0000","last_clean":"2024-04-25T12:37:21.569073+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:21.569073+0000","last_undegraded":"2024-04-25T12:37:21.569073+0000","last_fullsized":"2024-04-25T12:37:21.569073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423322+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:37:21.423322+0000","last_peered":"2024-04-25T12:37:21.423322+0000","last_clean":"2024-04-25T12:37:21.423322+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:21.423322+0000","last_undegraded":"2024-04-25T12:37:21.423322+0000","last_fullsized":"2024-04-25T12:37:21.423322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424396+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:37:21.424396+0000","last_peered":"2024-04-25T12:37:21.424396+0000","last_clean":"2024-04-25T12:37:21.424396+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:21.424396+0000","last_undegraded":"2024-04-25T12:37:21.424396+0000","last_fullsized":"2024-04-25T12:37:21.424396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907318+0000","last_change":"2024-04-25T12:37:01.204788+0000","last_active":"2024-04-25T12:37:21.907318+0000","last_peered":"2024-04-25T12:37:21.907318+0000","last_clean":"2024-04-25T12:37:21.907318+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:21.907318+0000","last_undegraded":"2024-04-25T12:37:21.907318+0000","last_fullsized":"2024-04-25T12:37:21.907318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:14.535103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908141+0000","last_change":"2024-04-25T12:37:01.217177+0000","last_active":"2024-04-25T12:37:21.908141+0000","last_peered":"2024-04-25T12:37:21.908141+0000","last_clean":"2024-04-25T12:37:21.908141+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:21.908141+0000","last_undegraded":"2024-04-25T12:37:21.908141+0000","last_fullsized":"2024-04-25T12:37:21.908141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:00.819525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423908+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:37:21.423908+0000","last_peered":"2024-04-25T12:37:21.423908+0000","last_clean":"2024-04-25T12:37:21.423908+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:21.423908+0000","last_undegraded":"2024-04-25T12:37:21.423908+0000","last_fullsized":"2024-04-25T12:37:21.423908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568987+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:37:21.568987+0000","last_peered":"2024-04-25T12:37:21.568987+0000","last_clean":"2024-04-25T12:37:21.568987+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:21.568987+0000","last_undegraded":"2024-04-25T12:37:21.568987+0000","last_fullsized":"2024-04-25T12:37:21.568987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569215+0000","last_change":"2024-04-25T12:37:01.209143+0000","last_active":"2024-04-25T12:37:21.569215+0000","last_peered":"2024-04-25T12:37:21.569215+0000","last_clean":"2024-04-25T12:37:21.569215+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:21.569215+0000","last_undegraded":"2024-04-25T12:37:21.569215+0000","last_fullsized":"2024-04-25T12:37:21.569215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:52.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907606+0000","last_change":"2024-04-25T12:37:01.223997+0000","last_active":"2024-04-25T12:37:21.907606+0000","last_peered":"2024-04-25T12:37:21.907606+0000","last_clean":"2024-04-25T12:37:21.907606+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:21.907606+0000","last_undegraded":"2024-04-25T12:37:21.907606+0000","last_fullsized":"2024-04-25T12:37:21.907606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:48.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911293+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:37:21.911293+0000","last_peered":"2024-04-25T12:37:21.911293+0000","last_clean":"2024-04-25T12:37:21.911293+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:21.911293+0000","last_undegraded":"2024-04-25T12:37:21.911293+0000","last_fullsized":"2024-04-25T12:37:21.911293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403285+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:37:21.403285+0000","last_peered":"2024-04-25T12:37:21.403285+0000","last_clean":"2024-04-25T12:37:21.403285+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:21.403285+0000","last_undegraded":"2024-04-25T12:37:21.403285+0000","last_fullsized":"2024-04-25T12:37:21.403285+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404558+0000","last_change":"2024-04-25T12:37:01.224898+0000","last_active":"2024-04-25T12:37:21.404558+0000","last_peered":"2024-04-25T12:37:21.404558+0000","last_clean":"2024-04-25T12:37:21.404558+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:21.404558+0000","last_undegraded":"2024-04-25T12:37:21.404558+0000","last_fullsized":"2024-04-25T12:37:21.404558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:18.991070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402556+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:37:21.402556+0000","last_peered":"2024-04-25T12:37:21.402556+0000","last_clean":"2024-04-25T12:37:21.402556+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:21.402556+0000","last_undegraded":"2024-04-25T12:37:21.402556+0000","last_fullsized":"2024-04-25T12:37:21.402556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422760+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:37:21.422760+0000","last_peered":"2024-04-25T12:37:21.422760+0000","last_clean":"2024-04-25T12:37:21.422760+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:21.422760+0000","last_undegraded":"2024-04-25T12:37:21.422760+0000","last_fullsized":"2024-04-25T12:37:21.422760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907860+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:37:21.907860+0000","last_peered":"2024-04-25T12:37:21.907860+0000","last_clean":"2024-04-25T12:37:21.907860+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:21.907860+0000","last_undegraded":"2024-04-25T12:37:21.907860+0000","last_fullsized":"2024-04-25T12:37:21.907860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908709+0000","last_change":"2024-04-25T12:37:01.228534+0000","last_active":"2024-04-25T12:37:21.908709+0000","last_peered":"2024-04-25T12:37:21.908709+0000","last_clean":"2024-04-25T12:37:21.908709+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:21.908709+0000","last_undegraded":"2024-04-25T12:37:21.908709+0000","last_fullsized":"2024-04-25T12:37:21.908709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:39.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907599+0000","last_change":"2024-04-25T12:37:01.210033+0000","last_active":"2024-04-25T12:37:21.907599+0000","last_peered":"2024-04-25T12:37:21.907599+0000","last_clean":"2024-04-25T12:37:21.907599+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:21.907599+0000","last_undegraded":"2024-04-25T12:37:21.907599+0000","last_fullsized":"2024-04-25T12:37:21.907599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:50.639901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406009+0000","last_change":"2024-04-25T12:37:01.232027+0000","last_active":"2024-04-25T12:37:21.406009+0000","last_peered":"2024-04-25T12:37:21.406009+0000","last_clean":"2024-04-25T12:37:21.406009+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:21.406009+0000","last_undegraded":"2024-04-25T12:37:21.406009+0000","last_fullsized":"2024-04-25T12:37:21.406009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908213+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:37:21.908213+0000","last_peered":"2024-04-25T12:37:21.908213+0000","last_clean":"2024-04-25T12:37:21.908213+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:21.908213+0000","last_undegraded":"2024-04-25T12:37:21.908213+0000","last_fullsized":"2024-04-25T12:37:21.908213+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423227+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:37:21.423227+0000","last_peered":"2024-04-25T12:37:21.423227+0000","last_clean":"2024-04-25T12:37:21.423227+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:21.423227+0000","last_undegraded":"2024-04-25T12:37:21.423227+0000","last_fullsized":"2024-04-25T12:37:21.423227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907222+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:37:21.907222+0000","last_peered":"2024-04-25T12:37:21.907222+0000","last_clean":"2024-04-25T12:37:21.907222+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:21.907222+0000","last_undegraded":"2024-04-25T12:37:21.907222+0000","last_fullsized":"2024-04-25T12:37:21.907222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911205+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:37:21.911205+0000","last_peered":"2024-04-25T12:37:21.911205+0000","last_clean":"2024-04-25T12:37:21.911205+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:21.911205+0000","last_undegraded":"2024-04-25T12:37:21.911205+0000","last_fullsized":"2024-04-25T12:37:21.911205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568777+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:37:21.568777+0000","last_peered":"2024-04-25T12:37:21.568777+0000","last_clean":"2024-04-25T12:37:21.568777+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:21.568777+0000","last_undegraded":"2024-04-25T12:37:21.568777+0000","last_fullsized":"2024-04-25T12:37:21.568777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910725+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:37:21.910725+0000","last_peered":"2024-04-25T12:37:21.910725+0000","last_clean":"2024-04-25T12:37:21.910725+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:21.910725+0000","last_undegraded":"2024-04-25T12:37:21.910725+0000","last_fullsized":"2024-04-25T12:37:21.910725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424005+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:37:21.424005+0000","last_peered":"2024-04-25T12:37:21.424005+0000","last_clean":"2024-04-25T12:37:21.424005+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:21.424005+0000","last_undegraded":"2024-04-25T12:37:21.424005+0000","last_fullsized":"2024-04-25T12:37:21.424005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403400+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:37:21.403400+0000","last_peered":"2024-04-25T12:37:21.403400+0000","last_clean":"2024-04-25T12:37:21.403400+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:21.403400+0000","last_undegraded":"2024-04-25T12:37:21.403400+0000","last_fullsized":"2024-04-25T12:37:21.403400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424575+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:37:21.424575+0000","last_peered":"2024-04-25T12:37:21.424575+0000","last_clean":"2024-04-25T12:37:21.424575+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:21.424575+0000","last_undegraded":"2024-04-25T12:37:21.424575+0000","last_fullsized":"2024-04-25T12:37:21.424575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423084+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:37:21.423084+0000","last_peered":"2024-04-25T12:37:21.423084+0000","last_clean":"2024-04-25T12:37:21.423084+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:21.423084+0000","last_undegraded":"2024-04-25T12:37:21.423084+0000","last_fullsized":"2024-04-25T12:37:21.423084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"70'1","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908366+0000","last_change":"2024-04-25T12:37:01.212249+0000","last_active":"2024-04-25T12:37:21.908366+0000","last_peered":"2024-04-25T12:37:21.908366+0000","last_clean":"2024-04-25T12:37:21.908366+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:21.908366+0000","last_undegraded":"2024-04-25T12:37:21.908366+0000","last_fullsized":"2024-04-25T12:37:21.908366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:47.658434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908490+0000","last_change":"2024-04-25T12:37:01.216745+0000","last_active":"2024-04-25T12:37:21.908490+0000","last_peered":"2024-04-25T12:37:21.908490+0000","last_clean":"2024-04-25T12:37:21.908490+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:21.908490+0000","last_undegraded":"2024-04-25T12:37:21.908490+0000","last_fullsized":"2024-04-25T12:37:21.908490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910517+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:37:21.910517+0000","last_peered":"2024-04-25T12:37:21.910517+0000","last_clean":"2024-04-25T12:37:21.910517+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:21.910517+0000","last_undegraded":"2024-04-25T12:37:21.910517+0000","last_fullsized":"2024-04-25T12:37:21.910517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910107+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:37:21.910107+0000","last_peered":"2024-04-25T12:37:21.910107+0000","last_clean":"2024-04-25T12:37:21.910107+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:21.910107+0000","last_undegraded":"2024-04-25T12:37:21.910107+0000","last_fullsized":"2024-04-25T12:37:21.910107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911235+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:37:21.911235+0000","last_peered":"2024-04-25T12:37:21.911235+0000","last_clean":"2024-04-25T12:37:21.911235+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:21.911235+0000","last_undegraded":"2024-04-25T12:37:21.911235+0000","last_fullsized":"2024-04-25T12:37:21.911235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913288+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:37:21.913288+0000","last_peered":"2024-04-25T12:37:21.913288+0000","last_clean":"2024-04-25T12:37:21.913288+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:21.913288+0000","last_undegraded":"2024-04-25T12:37:21.913288+0000","last_fullsized":"2024-04-25T12:37:21.913288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909742+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:37:21.909742+0000","last_peered":"2024-04-25T12:37:21.909742+0000","last_clean":"2024-04-25T12:37:21.909742+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:21.909742+0000","last_undegraded":"2024-04-25T12:37:21.909742+0000","last_fullsized":"2024-04-25T12:37:21.909742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405037+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:37:21.405037+0000","last_peered":"2024-04-25T12:37:21.405037+0000","last_clean":"2024-04-25T12:37:21.405037+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:21.405037+0000","last_undegraded":"2024-04-25T12:37:21.405037+0000","last_fullsized":"2024-04-25T12:37:21.405037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911087+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:37:21.911087+0000","last_peered":"2024-04-25T12:37:21.911087+0000","last_clean":"2024-04-25T12:37:21.911087+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:21.911087+0000","last_undegraded":"2024-04-25T12:37:21.911087+0000","last_fullsized":"2024-04-25T12:37:21.911087+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405035+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:37:21.405035+0000","last_peered":"2024-04-25T12:37:21.405035+0000","last_clean":"2024-04-25T12:37:21.405035+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:21.405035+0000","last_undegraded":"2024-04-25T12:37:21.405035+0000","last_fullsized":"2024-04-25T12:37:21.405035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907971+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:37:21.907971+0000","last_peered":"2024-04-25T12:37:21.907971+0000","last_clean":"2024-04-25T12:37:21.907971+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:21.907971+0000","last_undegraded":"2024-04-25T12:37:21.907971+0000","last_fullsized":"2024-04-25T12:37:21.907971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908194+0000","last_change":"2024-04-25T12:37:01.215943+0000","last_active":"2024-04-25T12:37:21.908194+0000","last_peered":"2024-04-25T12:37:21.908194+0000","last_clean":"2024-04-25T12:37:21.908194+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:21.908194+0000","last_undegraded":"2024-04-25T12:37:21.908194+0000","last_fullsized":"2024-04-25T12:37:21.908194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.518673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911529+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:37:21.911529+0000","last_peered":"2024-04-25T12:37:21.911529+0000","last_clean":"2024-04-25T12:37:21.911529+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:21.911529+0000","last_undegraded":"2024-04-25T12:37:21.911529+0000","last_fullsized":"2024-04-25T12:37:21.911529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41052,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":166,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82104,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462931,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29336,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342504,"statfs":{"total":96636764160,"available":96606724096,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":1207327,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1142784,"data_stored":616007,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37646,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38598,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:37:27.110 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-25T12:37:27.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-25T12:37:27.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.411+0000 7fcd73200640 1 Processor -- start 2024-04-25T12:37:27.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.411+0000 7fcd73200640 1 -- start start 2024-04-25T12:37:27.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd73200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcd740c7e30 0x7fcd74154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:27.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd73200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd74074290 con 0x7fcd74072cc0 2024-04-25T12:37:27.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd73200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd74074570 con 0x7fcd740730d0 2024-04-25T12:37:27.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd73200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcd74074850 con 0x7fcd740c7e30 2024-04-25T12:37:27.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd6be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcd74072cc0 0x7fcd7406c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58688/0 (socket says 172.21.15.123:58688) 2024-04-25T12:37:27.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd6be00640 1 -- 172.21.15.123:0/1712264708 learned_addr learned my addr 172.21.15.123:0/1712264708 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:27.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd71e00640 1 --2- 172.21.15.123:0/1712264708 >> v2:172.21.15.153:3300/0 conn(0x7fcd740c7e30 0x7fcd74154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd71400640 1 -- 172.21.15.123:0/1712264708 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493513956 0 0) 0x7fcd74074290 con 0x7fcd74072cc0 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd71400640 1 -- 172.21.15.123:0/1712264708 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd58003520 con 0x7fcd74072cc0 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.414+0000 7fcd71400640 1 -- 172.21.15.123:0/1712264708 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3723101997 0 0) 0x7fcd74074570 con 0x7fcd740730d0 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.415+0000 7fcd71400640 1 -- 172.21.15.123:0/1712264708 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd74074290 con 0x7fcd740730d0 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.415+0000 7fcd71e00640 1 -- 172.21.15.123:0/1712264708 >> v1:172.21.15.153:6789/0 conn(0x7fcd740730d0 legacy=0x7fcd7415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.415+0000 7fcd71e00640 1 -- 172.21.15.123:0/1712264708 >> v1:172.21.15.123:6789/0 conn(0x7fcd74072cc0 legacy=0x7fcd7406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.415+0000 7fcd71e00640 1 -- 172.21.15.123:0/1712264708 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd74074b30 con 0x7fcd740c7e30 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.416+0000 7fcd71e00640 1 --2- 172.21.15.123:0/1712264708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd740c7e30 0x7fcd74154280 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcd640097a0 tx=0x7fcd6402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c13b1a7a99a5ad9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.416+0000 7fcd71400640 1 -- 172.21.15.123:0/1712264708 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64037070 con 0x7fcd740c7e30 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.416+0000 7fcd71400640 1 -- 172.21.15.123:0/1712264708 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd64032440 con 0x7fcd740c7e30 2024-04-25T12:37:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.416+0000 7fcd71400640 1 -- 172.21.15.123:0/1712264708 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd640327f0 con 0x7fcd740c7e30 2024-04-25T12:37:27.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.417+0000 7fcd73200640 1 -- 172.21.15.123:0/1712264708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd740c7e30 msgr2=0x7fcd74154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.417+0000 7fcd73200640 1 --2- 172.21.15.123:0/1712264708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd740c7e30 0x7fcd74154280 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcd640097a0 tx=0x7fcd6402a230 comp rx=0 tx=0).stop 2024-04-25T12:37:27.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.417+0000 7fcd73200640 1 -- 172.21.15.123:0/1712264708 shutdown_connections 2024-04-25T12:37:27.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.417+0000 7fcd73200640 1 --2- 172.21.15.123:0/1712264708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd740c7e30 0x7fcd74154280 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.417+0000 7fcd73200640 1 -- 172.21.15.123:0/1712264708 >> 172.21.15.123:0/1712264708 conn(0x7fcd74070b70 msgr2=0x7fcd74070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:27.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.417+0000 7fcd73200640 1 -- 172.21.15.123:0/1712264708 shutdown_connections 2024-04-25T12:37:27.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.417+0000 7fcd73200640 1 -- 172.21.15.123:0/1712264708 wait complete. 2024-04-25T12:37:27.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.419+0000 7fcd73200640 1 Processor -- start 2024-04-25T12:37:27.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.419+0000 7fcd73200640 1 -- start start 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd73200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd74072cc0 0x7fcd740c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd73200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd740730d0 0x7fcd740c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd73200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd740c7e30 0x7fcd74152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd73200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd740c8240 con 0x7fcd740c7e30 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd73200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741547c0 con 0x7fcd74072cc0 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd73200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcd740c83c0 con 0x7fcd740730d0 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd740c7e30 0x7fcd74152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd740c7e30 0x7fcd74152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41426/0 (socket says 172.21.15.123:41426) 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 -- 172.21.15.123:0/1322676350 learned_addr learned my addr 172.21.15.123:0/1322676350 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd71e00640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd74072cc0 0x7fcd740c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd6be00640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd740730d0 0x7fcd740c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 -- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd740730d0 msgr2=0x7fcd740c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd740730d0 0x7fcd740c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 -- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd74072cc0 msgr2=0x7fcd740c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd74072cc0 0x7fcd740c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd72800640 1 -- 172.21.15.123:0/1322676350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd740c4e00 con 0x7fcd740c7e30 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.421+0000 7fcd71e00640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd74072cc0 0x7fcd740c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.422+0000 7fcd72800640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd740c7e30 0x7fcd74152b70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c6e0 tx=0x7fcd6c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.422+0000 7fcd6a000640 1 -- 172.21.15.123:0/1322676350 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c003b70 con 0x7fcd740c7e30 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.422+0000 7fcd6a000640 1 -- 172.21.15.123:0/1322676350 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6c003d10 con 0x7fcd740c7e30 2024-04-25T12:37:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.422+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd741f2520 con 0x7fcd740c7e30 2024-04-25T12:37:27.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.422+0000 7fcd6a000640 1 -- 172.21.15.123:0/1322676350 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0052d0 con 0x7fcd740c7e30 2024-04-25T12:37:27.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.422+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd741f27e0 con 0x7fcd740c7e30 2024-04-25T12:37:27.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.423+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd74076da0 con 0x7fcd740c7e30 2024-04-25T12:37:27.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.424+0000 7fcd6a000640 1 -- 172.21.15.123:0/1322676350 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd6c005470 con 0x7fcd740c7e30 2024-04-25T12:37:27.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.425+0000 7fcd6a000640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcd500744b0 0x7fcd500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:27.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.425+0000 7fcd6a000640 1 -- 172.21.15.123:0/1322676350 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fcd6c095380 con 0x7fcd740c7e30 2024-04-25T12:37:27.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.426+0000 7fcd71e00640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcd500744b0 0x7fcd500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:27.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.427+0000 7fcd71e00640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcd500744b0 0x7fcd500769c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd64034000 tx=0x7fcd6402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:27.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.430+0000 7fcd6a000640 1 -- 172.21.15.123:0/1322676350 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd6c05f140 con 0x7fcd740c7e30 2024-04-25T12:37:27.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.616+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) -- 0x7fcd740c7650 con 0x7fcd740c7e30 2024-04-25T12:37:27.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.624+0000 7fcd6a000640 1 -- 172.21.15.123:0/1322676350 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v11) ==== 118+0+0 (secure 0 0 0) 0x7fcd6c063440 con 0x7fcd740c7e30 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.626+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcd500744b0 msgr2=0x7fcd500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcd500744b0 0x7fcd500769c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd64034000 tx=0x7fcd6402e000 comp rx=0 tx=0).stop 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd740c7e30 msgr2=0x7fcd74152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd740c7e30 0x7fcd74152b70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c6e0 tx=0x7fcd6c00cba0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 shutdown_connections 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcd500744b0 0x7fcd500769c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd740c7e30 0x7fcd74152b70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd740730d0 0x7fcd740c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 --2- 172.21.15.123:0/1322676350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd74072cc0 0x7fcd740c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:27.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 >> 172.21.15.123:0/1322676350 conn(0x7fcd74070b70 msgr2=0x7fcd740c38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:27.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 shutdown_connections 2024-04-25T12:37:27.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:27.627+0000 7fcd73200640 1 -- 172.21.15.123:0/1322676350 wait complete. 2024-04-25T12:37:27.650 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T12:37:27.660 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T12:37:27.660 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:37:27.660 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:37:27.705 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T12:37:27.705 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:37:27.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:37:28.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.062+0000 7fca73e00640 1 Processor -- start 2024-04-25T12:37:28.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.062+0000 7fca73e00640 1 -- start start 2024-04-25T12:37:28.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.064+0000 7fca73e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fca741567b0 0x7fca7406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.064+0000 7fca73e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca7415d180 0x7fca740c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.064+0000 7fca73e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca7406ecd0 con 0x7fca7406cab0 2024-04-25T12:37:28.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.064+0000 7fca73e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fca7406efb0 con 0x7fca7415d180 2024-04-25T12:37:28.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.064+0000 7fca73e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fca7406f290 con 0x7fca741567b0 2024-04-25T12:37:28.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca73400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca7415d180 0x7fca740c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca73400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca7415d180 0x7fca740c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41440/0 (socket says 172.21.15.123:41440) 2024-04-25T12:37:28.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca73400640 1 -- 172.21.15.123:0/146634590 learned_addr learned my addr 172.21.15.123:0/146634590 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:28.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca72a00640 1 --2- 172.21.15.123:0/146634590 >> v2:172.21.15.153:3300/0 conn(0x7fca741567b0 0x7fca7406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca73400640 1 -- 172.21.15.123:0/146634590 >> v2:172.21.15.153:3300/0 conn(0x7fca741567b0 msgr2=0x7fca7406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:28.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca73400640 1 --2- 172.21.15.123:0/146634590 >> v2:172.21.15.153:3300/0 conn(0x7fca741567b0 0x7fca7406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca73400640 1 -- 172.21.15.123:0/146634590 >> v1:172.21.15.123:6789/0 conn(0x7fca7406cab0 legacy=0x7fca7415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca71600640 1 -- 172.21.15.123:0/146634590 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498582350 0 0) 0x7fca7406ecd0 con 0x7fca7406cab0 2024-04-25T12:37:28.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.065+0000 7fca73400640 1 -- 172.21.15.123:0/146634590 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7406f570 con 0x7fca7415d180 2024-04-25T12:37:28.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.066+0000 7fca73400640 1 --2- 172.21.15.123:0/146634590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca7415d180 0x7fca740c82b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca740c7cc0 tx=0x7fca6c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46a7d2da4d93de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:28.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.066+0000 7fca71600640 1 -- 172.21.15.123:0/146634590 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c036070 con 0x7fca7415d180 2024-04-25T12:37:28.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.066+0000 7fca71600640 1 -- 172.21.15.123:0/146634590 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca6c031600 con 0x7fca7415d180 2024-04-25T12:37:28.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.066+0000 7fca71600640 1 -- 172.21.15.123:0/146634590 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0319f0 con 0x7fca7415d180 2024-04-25T12:37:28.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 -- 172.21.15.123:0/146634590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca7415d180 msgr2=0x7fca740c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 --2- 172.21.15.123:0/146634590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca7415d180 0x7fca740c82b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca740c7cc0 tx=0x7fca6c02a7c0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 -- 172.21.15.123:0/146634590 shutdown_connections 2024-04-25T12:37:28.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 --2- 172.21.15.123:0/146634590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca7415d180 0x7fca740c82b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 --2- 172.21.15.123:0/146634590 >> v2:172.21.15.153:3300/0 conn(0x7fca741567b0 0x7fca7406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 -- 172.21.15.123:0/146634590 >> 172.21.15.123:0/146634590 conn(0x7fca740cd600 msgr2=0x7fca740cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:28.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 -- 172.21.15.123:0/146634590 shutdown_connections 2024-04-25T12:37:28.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.067+0000 7fca73e00640 1 -- 172.21.15.123:0/146634590 wait complete. 2024-04-25T12:37:28.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.069+0000 7fca73e00640 1 Processor -- start 2024-04-25T12:37:28.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.070+0000 7fca73e00640 1 -- start start 2024-04-25T12:37:28.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca73e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7406cab0 0x7fca7414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca741567b0 0x7fca740cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca73e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7415d180 0x7fca7414c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca73e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca74156bc0 con 0x7fca741567b0 2024-04-25T12:37:28.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca73e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca740c9d20 con 0x7fca7406cab0 2024-04-25T12:37:28.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca73e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fca74156d40 con 0x7fca7415d180 2024-04-25T12:37:28.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca741567b0 0x7fca740cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca741567b0 0x7fca740cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41454/0 (socket says 172.21.15.123:41454) 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca72000640 1 -- 172.21.15.123:0/189525449 learned_addr learned my addr 172.21.15.123:0/189525449 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca72a00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7406cab0 0x7fca7414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.071+0000 7fca73400640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7415d180 0x7fca7414c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca72000640 1 -- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7415d180 msgr2=0x7fca7414c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca72000640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7415d180 0x7fca7414c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca72000640 1 -- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7406cab0 msgr2=0x7fca7414e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca72000640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7406cab0 0x7fca7414e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca72000640 1 -- 172.21.15.123:0/189525449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7414ee50 con 0x7fca741567b0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca72000640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca741567b0 0x7fca740cc760 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c440 tx=0x7fca6000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca6aa00640 1 -- 172.21.15.123:0/189525449 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60017070 con 0x7fca741567b0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca6aa00640 1 -- 172.21.15.123:0/189525449 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca60005480 con 0x7fca741567b0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca741f27b0 con 0x7fca741567b0 2024-04-25T12:37:28.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca6aa00640 1 -- 172.21.15.123:0/189525449 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60003830 con 0x7fca741567b0 2024-04-25T12:37:28.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.072+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca741f2d70 con 0x7fca741567b0 2024-04-25T12:37:28.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.073+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca740b3aa0 con 0x7fca741567b0 2024-04-25T12:37:28.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.075+0000 7fca6aa00640 1 -- 172.21.15.123:0/189525449 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fca6001a070 con 0x7fca741567b0 2024-04-25T12:37:28.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.076+0000 7fca6aa00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fca540744b0 0x7fca540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.076+0000 7fca6aa00640 1 -- 172.21.15.123:0/189525449 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fca600949e0 con 0x7fca741567b0 2024-04-25T12:37:28.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.076+0000 7fca72a00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fca540744b0 0x7fca540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.077+0000 7fca72a00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fca540744b0 0x7fca540769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fca64009aa0 tx=0x7fca64009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:28.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.081+0000 7fca6aa00640 1 -- 172.21.15.123:0/189525449 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca740b3aa0 con 0x7fca741567b0 2024-04-25T12:37:28.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.281+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca74076da0 con 0x7fca741567b0 2024-04-25T12:37:28.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.284+0000 7fca6aa00640 1 -- 172.21.15.123:0/189525449 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+19058 (secure 0 0 0) 0x7fca6005e5e0 con 0x7fca741567b0 2024-04-25T12:37:28.284 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:28.284 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":93,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:37:21.396436+0000","last_up_change":"2024-04-25T12:35:59.575545+0000","last_in_change":"2024-04-25T12:37:00.163243+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":39,"up_from":47,"up_thru":76,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6827","nonce":4270711740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6829","nonce":4270711740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6833","nonce":4270711740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6831","nonce":4270711740}]},"public_addr":"172.21.15.153:6827/4270711740","cluster_addr":"172.21.15.153:6829/4270711740","heartbeat_back_addr":"172.21.15.153:6833/4270711740","heartbeat_front_addr":"172.21.15.153:6831/4270711740","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":40},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[3,1]},{"pgid":"4.15","osds":[1,0,7,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":5,"to":4}]},{"pgid":"2.c","mappings":[{"from":0,"to":2}]},{"pgid":"3.3","mappings":[{"from":0,"to":7}]},{"pgid":"3.c","mappings":[{"from":0,"to":1}]},{"pgid":"3.13","mappings":[{"from":0,"to":1}]},{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"3.17","mappings":[{"from":3,"to":7}]},{"pgid":"3.26","mappings":[{"from":3,"to":1}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.287+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fca540744b0 msgr2=0x7fca540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.287+0000 7fca73e00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fca540744b0 0x7fca540769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fca64009aa0 tx=0x7fca64009190 comp rx=0 tx=0).stop 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.287+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca741567b0 msgr2=0x7fca740cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.287+0000 7fca73e00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca741567b0 0x7fca740cc760 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c440 tx=0x7fca6000c900 comp rx=0 tx=0).stop 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 shutdown_connections 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fca540744b0 0x7fca540769c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7415d180 0x7fca7414c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca741567b0 0x7fca740cc760 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 --2- 172.21.15.123:0/189525449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7406cab0 0x7fca7414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 >> 172.21.15.123:0/189525449 conn(0x7fca740cd600 msgr2=0x7fca7415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 shutdown_connections 2024-04-25T12:37:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.288+0000 7fca73e00640 1 -- 172.21.15.123:0/189525449 wait complete. 2024-04-25T12:37:28.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:34:06.827674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:34:11.030660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:34:12.395253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T12:34:15.975923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T12:37:28.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:37:28.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.587+0000 7ff9b2c00640 1 Processor -- start 2024-04-25T12:37:28.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.587+0000 7ff9b2c00640 1 -- start start 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.589+0000 7ff9b2c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0ca4b0 0x7ff9ac15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.589+0000 7ff9b2c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9ac15d7c0 con 0x7ff9ac0cc5e0 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.589+0000 7ff9b2c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9ac0cbd50 con 0x7ff9ac0c58e0 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.589+0000 7ff9b2c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ac070970 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9b1800640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff9ac0c58e0 0x7ff9ac070e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:56014/0 (socket says 172.21.15.123:56014) 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9b0e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0ca4b0 0x7ff9ac15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9b1800640 1 -- 172.21.15.123:0/1110545452 learned_addr learned my addr 172.21.15.123:0/1110545452 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9b2200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff9ac0cc5e0 0x7ff9ac164b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58708/0 (socket says 172.21.15.123:58708) 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9abe00640 1 -- 172.21.15.123:0/1110545452 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330625025 0 0) 0x7ff9ac0cbd50 con 0x7ff9ac0c58e0 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9b0e00640 1 -- 172.21.15.123:0/1110545452 >> v1:172.21.15.153:6789/0 conn(0x7ff9ac0c58e0 legacy=0x7ff9ac070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9b0e00640 1 -- 172.21.15.123:0/1110545452 >> v1:172.21.15.123:6789/0 conn(0x7ff9ac0cc5e0 legacy=0x7ff9ac164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.590+0000 7ff9b0e00640 1 -- 172.21.15.123:0/1110545452 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac0c4510 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.591+0000 7ff9b0e00640 1 --2- 172.21.15.123:0/1110545452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0ca4b0 0x7ff9ac15e2a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff9980097d0 tx=0x7ff99802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2325befa79cd54b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.591+0000 7ff9abe00640 1 -- 172.21.15.123:0/1110545452 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998037070 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.591+0000 7ff9abe00640 1 -- 172.21.15.123:0/1110545452 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99802ae30 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.591+0000 7ff9abe00640 1 -- 172.21.15.123:0/1110545452 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998032640 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.592+0000 7ff9b2c00640 1 -- 172.21.15.123:0/1110545452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0ca4b0 msgr2=0x7ff9ac15e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.592+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/1110545452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0ca4b0 0x7ff9ac15e2a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff9980097d0 tx=0x7ff99802a1a0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.592+0000 7ff9b2c00640 1 -- 172.21.15.123:0/1110545452 shutdown_connections 2024-04-25T12:37:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.592+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/1110545452 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0ca4b0 0x7ff9ac15e2a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.592+0000 7ff9b2c00640 1 -- 172.21.15.123:0/1110545452 >> 172.21.15.123:0/1110545452 conn(0x7ff9ac072ff0 msgr2=0x7ff9ac073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.592+0000 7ff9b2c00640 1 -- 172.21.15.123:0/1110545452 shutdown_connections 2024-04-25T12:37:28.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.592+0000 7ff9b2c00640 1 -- 172.21.15.123:0/1110545452 wait complete. 2024-04-25T12:37:28.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.594+0000 7ff9b2c00640 1 Processor -- start 2024-04-25T12:37:28.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.595+0000 7ff9b2c00640 1 -- start start 2024-04-25T12:37:28.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b2c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ac0c58e0 0x7ff9ac159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b2c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9ac0ca4b0 0x7ff9ac1f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b2c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0cc5e0 0x7ff9ac150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b2c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac071510 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b2c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac0c4ad0 con 0x7ff9ac0c58e0 2024-04-25T12:37:28.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b2c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ac0ca0b0 con 0x7ff9ac0cc5e0 2024-04-25T12:37:28.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9ac0ca4b0 0x7ff9ac1f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9ac0ca4b0 0x7ff9ac1f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41466/0 (socket says 172.21.15.123:41466) 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b0e00640 1 -- 172.21.15.123:0/676636646 learned_addr learned my addr 172.21.15.123:0/676636646 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b1800640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ac0c58e0 0x7ff9ac159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.596+0000 7ff9b2200640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0cc5e0 0x7ff9ac150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9b0e00640 1 -- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0cc5e0 msgr2=0x7ff9ac150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9b0e00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0cc5e0 0x7ff9ac150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9b0e00640 1 -- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ac0c58e0 msgr2=0x7ff9ac159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9b0e00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ac0c58e0 0x7ff9ac159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9b0e00640 1 -- 172.21.15.123:0/676636646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac1f0620 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9b1800640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ac0c58e0 0x7ff9ac159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9b0e00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9ac0ca4b0 0x7ff9ac1f00e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff998000c00 tx=0x7ff9980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9a9600640 1 -- 172.21.15.123:0/676636646 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998037040 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.597+0000 7ff9a9600640 1 -- 172.21.15.123:0/676636646 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff998004030 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.598+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac150f00 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.598+0000 7ff9a9600640 1 -- 172.21.15.123:0/676636646 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998006e10 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.598+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1514c0 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.599+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ac076da0 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.600+0000 7ff9a9600640 1 -- 172.21.15.123:0/676636646 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff99803a040 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.601+0000 7ff9a9600640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff98c0744b0 0x7ff98c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:28.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.601+0000 7ff9a9600640 1 -- 172.21.15.123:0/676636646 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7ff9980b58d0 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.602+0000 7ff9b1800640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff98c0744b0 0x7ff98c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:28.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.602+0000 7ff9b1800640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff98c0744b0 0x7ff98c0769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0007f30 tx=0x7ff9a0007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:28.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.606+0000 7ff9a9600640 1 -- 172.21.15.123:0/676636646 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff998042020 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.808+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9ac0cabb0 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.809+0000 7ff9a9600640 1 -- 172.21.15.123:0/676636646 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7ff99807f560 con 0x7ff9ac0ca4b0 2024-04-25T12:37:28.808 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff98c0744b0 msgr2=0x7ff98c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff98c0744b0 0x7ff98c0769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0007f30 tx=0x7ff9a0007e80 comp rx=0 tx=0).stop 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9ac0ca4b0 msgr2=0x7ff9ac1f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9ac0ca4b0 0x7ff9ac1f00e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff998000c00 tx=0x7ff9980046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 shutdown_connections 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff98c0744b0 0x7ff98c0769c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9ac0cc5e0 0x7ff9ac150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9ac0ca4b0 0x7ff9ac1f00e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 --2- 172.21.15.123:0/676636646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ac0c58e0 0x7ff9ac159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.812+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 >> 172.21.15.123:0/676636646 conn(0x7ff9ac072ff0 msgr2=0x7ff9ac1535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.813+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 shutdown_connections 2024-04-25T12:37:28.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:28.813+0000 7ff9b2c00640 1 -- 172.21.15.123:0/676636646 wait complete. 2024-04-25T12:37:28.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:37:29.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.116+0000 7f7eb0a00640 1 Processor -- start 2024-04-25T12:37:29.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.116+0000 7f7eb0a00640 1 -- start start 2024-04-25T12:37:29.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.118+0000 7f7eb0a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7eac1528b0 0x7f7eac0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.118+0000 7f7eb0a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac152cc0 con 0x7f7eac0b34f0 2024-04-25T12:37:29.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.118+0000 7f7eb0a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac1589d0 con 0x7f7eac0c2c60 2024-04-25T12:37:29.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.118+0000 7f7eb0a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7eac0c4060 con 0x7f7eac1528b0 2024-04-25T12:37:29.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.118+0000 7f7eab400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7eac0b34f0 0x7f7eac0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58716/0 (socket says 172.21.15.123:58716) 2024-04-25T12:37:29.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.118+0000 7f7eab400640 1 -- 172.21.15.123:0/2280150874 learned_addr learned my addr 172.21.15.123:0/2280150874 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:29.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.118+0000 7f7ea3e00640 1 --2- 172.21.15.123:0/2280150874 >> v2:172.21.15.153:3300/0 conn(0x7f7eac1528b0 0x7f7eac0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.119+0000 7f7eaaa00640 1 -- 172.21.15.123:0/2280150874 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965692475 0 0) 0x7f7e9c001110 con 0x7f7eac0b34f0 2024-04-25T12:37:29.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.119+0000 7f7eaaa00640 1 -- 172.21.15.123:0/2280150874 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e8c003440 con 0x7f7eac0b34f0 2024-04-25T12:37:29.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.119+0000 7f7eaaa00640 1 -- 172.21.15.123:0/2280150874 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010433005 0 0) 0x7f7eac1589d0 con 0x7f7eac0c2c60 2024-04-25T12:37:29.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.119+0000 7f7eaaa00640 1 -- 172.21.15.123:0/2280150874 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e9c001110 con 0x7f7eac0c2c60 2024-04-25T12:37:29.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.119+0000 7f7ea3e00640 1 -- 172.21.15.123:0/2280150874 >> v1:172.21.15.153:6789/0 conn(0x7f7eac0c2c60 legacy=0x7f7eac0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.119+0000 7f7ea3e00640 1 -- 172.21.15.123:0/2280150874 >> v1:172.21.15.123:6789/0 conn(0x7f7eac0b34f0 legacy=0x7f7eac0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.119+0000 7f7ea3e00640 1 -- 172.21.15.123:0/2280150874 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac158c70 con 0x7f7eac1528b0 2024-04-25T12:37:29.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.120+0000 7f7ea3e00640 1 --2- 172.21.15.123:0/2280150874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 0x7f7eac0c2720 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7eac158ba0 tx=0x7f7e9402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1471f37438b81a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:29.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.120+0000 7f7eaaa00640 1 -- 172.21.15.123:0/2280150874 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94037070 con 0x7f7eac1528b0 2024-04-25T12:37:29.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.120+0000 7f7eaaa00640 1 -- 172.21.15.123:0/2280150874 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e94032640 con 0x7f7eac1528b0 2024-04-25T12:37:29.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.120+0000 7f7eaaa00640 1 -- 172.21.15.123:0/2280150874 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94032a10 con 0x7f7eac1528b0 2024-04-25T12:37:29.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.121+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2280150874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 msgr2=0x7f7eac0c2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.121+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2280150874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 0x7f7eac0c2720 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7eac158ba0 tx=0x7f7e9402a2c0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.121+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2280150874 shutdown_connections 2024-04-25T12:37:29.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.121+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2280150874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 0x7f7eac0c2720 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.121+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2280150874 >> 172.21.15.123:0/2280150874 conn(0x7f7eac070b70 msgr2=0x7f7eac070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:29.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.121+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2280150874 shutdown_connections 2024-04-25T12:37:29.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.121+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2280150874 wait complete. 2024-04-25T12:37:29.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.123+0000 7f7eb0a00640 1 Processor -- start 2024-04-25T12:37:29.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.123+0000 7f7eb0a00640 1 -- start start 2024-04-25T12:37:29.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eac0b34f0 0x7f7eac1e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7eac0c2c60 0x7f7eac1490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 0x7f7eac0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eb0a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac159230 con 0x7f7eac0b34f0 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eb0a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac0c88a0 con 0x7f7eac1528b0 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eb0a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac159510 con 0x7f7eac0c2c60 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eac0b34f0 0x7f7eac1e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eac0b34f0 0x7f7eac1e1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41472/0 (socket says 172.21.15.123:41472) 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eab400640 1 -- 172.21.15.123:0/2627622566 learned_addr learned my addr 172.21.15.123:0/2627622566 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7ea3e00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7eac0c2c60 0x7f7eac1490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.125+0000 7f7eabe00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 0x7f7eac0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eab400640 1 -- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7eac0c2c60 msgr2=0x7f7eac1490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eab400640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7eac0c2c60 0x7f7eac1490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eab400640 1 -- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 msgr2=0x7f7eac0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eab400640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 0x7f7eac0c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eab400640 1 -- 172.21.15.123:0/2627622566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac1512f0 con 0x7f7eac0b34f0 2024-04-25T12:37:29.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eab400640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eac0b34f0 0x7f7eac1e1c30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c760 tx=0x7f7e9c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:29.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7ea3400640 1 -- 172.21.15.123:0/2627622566 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c017070 con 0x7f7eac0b34f0 2024-04-25T12:37:29.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7ea3400640 1 -- 172.21.15.123:0/2627622566 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c005af0 con 0x7f7eac0b34f0 2024-04-25T12:37:29.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac1e2170 con 0x7f7eac0b34f0 2024-04-25T12:37:29.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7ea3400640 1 -- 172.21.15.123:0/2627622566 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c007070 con 0x7f7eac0b34f0 2024-04-25T12:37:29.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.126+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac0c4c20 con 0x7f7eac0b34f0 2024-04-25T12:37:29.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.127+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eac076da0 con 0x7f7eac0b34f0 2024-04-25T12:37:29.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.129+0000 7f7ea3400640 1 -- 172.21.15.123:0/2627622566 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e9c01a070 con 0x7f7eac0b34f0 2024-04-25T12:37:29.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.129+0000 7f7ea3400640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e880744e0 0x7f7e880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.130+0000 7f7ea3400640 1 -- 172.21.15.123:0/2627622566 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f7e9c094ab0 con 0x7f7eac0b34f0 2024-04-25T12:37:29.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.130+0000 7f7ea3e00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e880744e0 0x7f7e880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.130+0000 7f7ea3e00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e880744e0 0x7f7e880769f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7e94034000 tx=0x7f7e9402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:29.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.134+0000 7f7ea3400640 1 -- 172.21.15.123:0/2627622566 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7eac152cc0 con 0x7f7eac0b34f0 2024-04-25T12:37:29.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.331+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7eac0c9930 con 0x7f7eac0b34f0 2024-04-25T12:37:29.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.332+0000 7f7ea3400640 1 -- 172.21.15.123:0/2627622566 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f7e9c05e810 con 0x7f7eac0b34f0 2024-04-25T12:37:29.331 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:37:29.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e880744e0 msgr2=0x7f7e880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e880744e0 0x7f7e880769f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7e94034000 tx=0x7f7e9402e000 comp rx=0 tx=0).stop 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eac0b34f0 msgr2=0x7f7eac1e1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eac0b34f0 0x7f7eac1e1c30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c760 tx=0x7f7e9c00cc20 comp rx=0 tx=0).stop 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 shutdown_connections 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7e880744e0 0x7f7e880769f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eac1528b0 0x7f7eac0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7eac0c2c60 0x7f7eac1490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 --2- 172.21.15.123:0/2627622566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eac0b34f0 0x7f7eac1e1c30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.334+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 >> 172.21.15.123:0/2627622566 conn(0x7f7eac070b70 msgr2=0x7f7eac1e2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.335+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 shutdown_connections 2024-04-25T12:37:29.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.335+0000 7f7eb0a00640 1 -- 172.21.15.123:0/2627622566 wait complete. 2024-04-25T12:37:29.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:37:29.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.624+0000 7f4bfc400640 1 Processor -- start 2024-04-25T12:37:29.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.624+0000 7f4bfc400640 1 -- start start 2024-04-25T12:37:29.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.626+0000 7f4bfc400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf4074930 0x7f4bf40b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.626+0000 7f4bfc400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bf40c72c0 con 0x7f4bf406ffb0 2024-04-25T12:37:29.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.626+0000 7f4bfc400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bf415a680 con 0x7f4bf40c4c70 2024-04-25T12:37:29.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.626+0000 7f4bfc400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf40c7c90 con 0x7f4bf4074930 2024-04-25T12:37:29.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.626+0000 7f4bf9a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4bf406ffb0 0x7f4bf4154720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58730/0 (socket says 172.21.15.123:58730) 2024-04-25T12:37:29.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.626+0000 7f4bf9a00640 1 -- 172.21.15.123:0/2589667057 learned_addr learned my addr 172.21.15.123:0/2589667057 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:29.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.626+0000 7f4bf9000640 1 --2- 172.21.15.123:0/2589667057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf4074930 0x7f4bf40b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.627+0000 7f4bf3e00640 1 -- 172.21.15.123:0/2589667057 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3836673755 0 0) 0x7f4bf415a680 con 0x7f4bf40c4c70 2024-04-25T12:37:29.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.627+0000 7f4bf3e00640 1 -- 172.21.15.123:0/2589667057 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bdc003520 con 0x7f4bf40c4c70 2024-04-25T12:37:29.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.627+0000 7f4bf3e00640 1 -- 172.21.15.123:0/2589667057 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701134917 0 0) 0x7f4bf40c72c0 con 0x7f4bf406ffb0 2024-04-25T12:37:29.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.627+0000 7f4bf3e00640 1 -- 172.21.15.123:0/2589667057 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bf415a680 con 0x7f4bf406ffb0 2024-04-25T12:37:29.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.627+0000 7f4bf9000640 1 -- 172.21.15.123:0/2589667057 >> v1:172.21.15.153:6789/0 conn(0x7f4bf40c4c70 legacy=0x7f4bf415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.627+0000 7f4bf9000640 1 -- 172.21.15.123:0/2589667057 >> v1:172.21.15.123:6789/0 conn(0x7f4bf406ffb0 legacy=0x7f4bf4154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.627+0000 7f4bf9000640 1 -- 172.21.15.123:0/2589667057 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf40c6740 con 0x7f4bf4074930 2024-04-25T12:37:29.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.628+0000 7f4bf9000640 1 --2- 172.21.15.123:0/2589667057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf4074930 0x7f4bf40b2fa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40c7e10 tx=0x7f4be002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a7f5517d195fde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:29.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.628+0000 7f4bf3e00640 1 -- 172.21.15.123:0/2589667057 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0037070 con 0x7f4bf4074930 2024-04-25T12:37:29.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.628+0000 7f4bf3e00640 1 -- 172.21.15.123:0/2589667057 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be0032440 con 0x7f4bf4074930 2024-04-25T12:37:29.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.628+0000 7f4bf3e00640 1 -- 172.21.15.123:0/2589667057 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be00327f0 con 0x7f4bf4074930 2024-04-25T12:37:29.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.629+0000 7f4bfc400640 1 -- 172.21.15.123:0/2589667057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf4074930 msgr2=0x7f4bf40b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.629+0000 7f4bfc400640 1 --2- 172.21.15.123:0/2589667057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf4074930 0x7f4bf40b2fa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40c7e10 tx=0x7f4be002a290 comp rx=0 tx=0).stop 2024-04-25T12:37:29.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.629+0000 7f4bfc400640 1 -- 172.21.15.123:0/2589667057 shutdown_connections 2024-04-25T12:37:29.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.629+0000 7f4bfc400640 1 --2- 172.21.15.123:0/2589667057 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf4074930 0x7f4bf40b2fa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.629+0000 7f4bfc400640 1 -- 172.21.15.123:0/2589667057 >> 172.21.15.123:0/2589667057 conn(0x7f4bf40ca270 msgr2=0x7f4bf40ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:29.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.629+0000 7f4bfc400640 1 -- 172.21.15.123:0/2589667057 shutdown_connections 2024-04-25T12:37:29.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.629+0000 7f4bfc400640 1 -- 172.21.15.123:0/2589667057 wait complete. 2024-04-25T12:37:29.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.631+0000 7f4bfc400640 1 Processor -- start 2024-04-25T12:37:29.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.632+0000 7f4bfc400640 1 -- start start 2024-04-25T12:37:29.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfc400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf406ffb0 0x7f4bf41e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfc400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bf4074930 0x7f4bf40c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfc400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bf40c4c70 0x7f4bf4145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfc400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf40c6fe0 con 0x7f4bf40c4c70 2024-04-25T12:37:29.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfc400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf40c6d00 con 0x7f4bf4074930 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfc400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf415b270 con 0x7f4bf406ffb0 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bf40c4c70 0x7f4bf4145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bf40c4c70 0x7f4bf4145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41474/0 (socket says 172.21.15.123:41474) 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bfa400640 1 -- 172.21.15.123:0/649338907 learned_addr learned my addr 172.21.15.123:0/649338907 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bf9000640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bf4074930 0x7f4bf40c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.633+0000 7f4bf9a00640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf406ffb0 0x7f4bf41e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfa400640 1 -- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf406ffb0 msgr2=0x7f4bf41e51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfa400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf406ffb0 0x7f4bf41e51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfa400640 1 -- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bf4074930 msgr2=0x7f4bf40c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfa400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bf4074930 0x7f4bf40c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfa400640 1 -- 172.21.15.123:0/649338907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf41e5730 con 0x7f4bf40c4c70 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfa400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bf40c4c70 0x7f4bf4145270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c680 tx=0x7f4bec00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:29.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bf1600640 1 -- 172.21.15.123:0/649338907 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec003b70 con 0x7f4bf40c4c70 2024-04-25T12:37:29.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bf1600640 1 -- 172.21.15.123:0/649338907 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bec003d10 con 0x7f4bf40c4c70 2024-04-25T12:37:29.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf40c3600 con 0x7f4bf40c4c70 2024-04-25T12:37:29.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bf1600640 1 -- 172.21.15.123:0/649338907 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec005310 con 0x7f4bf40c4c70 2024-04-25T12:37:29.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.634+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41e5f60 con 0x7f4bf40c4c70 2024-04-25T12:37:29.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.636+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf40c37d0 con 0x7f4bf40c4c70 2024-04-25T12:37:29.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.637+0000 7f4bf1600640 1 -- 172.21.15.123:0/649338907 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4bec0054b0 con 0x7f4bf40c4c70 2024-04-25T12:37:29.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.638+0000 7f4bf1600640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4bd40744b0 0x7f4bd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:29.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.638+0000 7f4bf1600640 1 -- 172.21.15.123:0/649338907 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f4bec095430 con 0x7f4bf40c4c70 2024-04-25T12:37:29.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.638+0000 7f4bf9a00640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4bd40744b0 0x7f4bd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:29.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.639+0000 7f4bf9a00640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4bd40744b0 0x7f4bd40769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005880 tx=0x7f4be8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:29.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.643+0000 7f4bf1600640 1 -- 172.21.15.123:0/649338907 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bec05f0c0 con 0x7f4bf40c4c70 2024-04-25T12:37:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.842+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4bf4076700 con 0x7f4bf40c4c70 2024-04-25T12:37:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.843+0000 7f4bf1600640 1 -- 172.21.15.123:0/649338907 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f4bec0633c0 con 0x7f4bf40c4c70 2024-04-25T12:37:29.842 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T12:37:29.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4bd40744b0 msgr2=0x7f4bd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4bd40744b0 0x7f4bd40769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005880 tx=0x7f4be8009040 comp rx=0 tx=0).stop 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bf40c4c70 msgr2=0x7f4bf4145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bf40c4c70 0x7f4bf4145270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c680 tx=0x7f4bec00cb40 comp rx=0 tx=0).stop 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 shutdown_connections 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f4bd40744b0 0x7f4bd40769c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bf40c4c70 0x7f4bf4145270 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bf4074930 0x7f4bf40c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 --2- 172.21.15.123:0/649338907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4bf406ffb0 0x7f4bf41e51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 >> 172.21.15.123:0/649338907 conn(0x7f4bf40ca270 msgr2=0x7f4bf4153440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.846+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 shutdown_connections 2024-04-25T12:37:29.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:29.847+0000 7f4bfc400640 1 -- 172.21.15.123:0/649338907 wait complete. 2024-04-25T12:37:29.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T12:37:30.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.154+0000 7f445f800640 1 Processor -- start 2024-04-25T12:37:30.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.155+0000 7f445f800640 1 -- start start 2024-04-25T12:37:30.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.156+0000 7f445f800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4458152c70 0x7f4458074820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.156+0000 7f445f800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44580c2790 0x7f44580c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.156+0000 7f445f800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4458158bd0 con 0x7f44580c3fe0 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.156+0000 7f445f800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4458158eb0 con 0x7f44580c2790 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.156+0000 7f445f800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4458159190 con 0x7f4458152c70 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f445d800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f44580c3fe0 0x7f445815c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58744/0 (socket says 172.21.15.123:58744) 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f445d800640 1 -- 172.21.15.123:0/2456146064 learned_addr learned my addr 172.21.15.123:0/2456146064 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f445ce00640 1 --2- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4458152c70 0x7f4458074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f4457e00640 1 --2- 172.21.15.123:0/2456146064 >> v2:172.21.15.153:3300/0 conn(0x7f44580c2790 0x7f44580c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f4457400640 1 -- 172.21.15.123:0/2456146064 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575701548 0 0) 0x7f4458158bd0 con 0x7f44580c3fe0 2024-04-25T12:37:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f4457400640 1 -- 172.21.15.123:0/2456146064 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f443c003520 con 0x7f44580c3fe0 2024-04-25T12:37:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f4457e00640 1 -- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4458152c70 msgr2=0x7f4458074820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f4457e00640 1 --2- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4458152c70 0x7f4458074820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f4457e00640 1 -- 172.21.15.123:0/2456146064 >> v1:172.21.15.123:6789/0 conn(0x7f44580c3fe0 legacy=0x7f445815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.157+0000 7f4457e00640 1 -- 172.21.15.123:0/2456146064 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4458159470 con 0x7f44580c2790 2024-04-25T12:37:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.158+0000 7f445ce00640 1 --2- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4458152c70 0x7f4458074820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.159+0000 7f4457e00640 1 --2- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c2790 0x7f44580c8170 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f444400a940 tx=0x7f444402b4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190bcc4a46f38861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.159+0000 7f4457400640 1 -- 172.21.15.123:0/2456146064 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444400b040 con 0x7f44580c2790 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.159+0000 7f4457400640 1 -- 172.21.15.123:0/2456146064 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4444030710 con 0x7f44580c2790 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.159+0000 7f4457400640 1 -- 172.21.15.123:0/2456146064 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444030ac0 con 0x7f44580c2790 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 -- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c2790 msgr2=0x7f44580c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 --2- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c2790 0x7f44580c8170 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f444400a940 tx=0x7f444402b4c0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 -- 172.21.15.123:0/2456146064 shutdown_connections 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 --2- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c2790 0x7f44580c8170 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 --2- 172.21.15.123:0/2456146064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4458152c70 0x7f4458074820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 -- 172.21.15.123:0/2456146064 >> 172.21.15.123:0/2456146064 conn(0x7f445806f440 msgr2=0x7f445806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 -- 172.21.15.123:0/2456146064 shutdown_connections 2024-04-25T12:37:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.160+0000 7f445f800640 1 -- 172.21.15.123:0/2456146064 wait complete. 2024-04-25T12:37:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.162+0000 7f445f800640 1 Processor -- start 2024-04-25T12:37:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.162+0000 7f445f800640 1 -- start start 2024-04-25T12:37:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445f800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44580c2790 0x7f445814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445f800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c3fe0 0x7f44580cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445f800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4458152c70 0x7f44581e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445f800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f445815cc40 con 0x7f4458152c70 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445f800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f445815cac0 con 0x7f44580c3fe0 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445f800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f445814d4e0 con 0x7f44580c2790 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f4457e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c3fe0 0x7f44580cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4458152c70 0x7f44581e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445ce00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44580c2790 0x7f445814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445ce00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44580c2790 0x7f445814cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:57228/0 (socket says 172.21.15.123:57228) 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f4457e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c3fe0 0x7f44580cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:48742/0 (socket says 172.21.15.123:48742) 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.164+0000 7f445ce00640 1 -- 172.21.15.123:0/1101982893 learned_addr learned my addr 172.21.15.123:0/1101982893 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4457e00640 1 -- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44580c2790 msgr2=0x7f445814cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4457e00640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44580c2790 0x7f445814cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4457e00640 1 -- 172.21.15.123:0/1101982893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4458152c70 msgr2=0x7f44581e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4457e00640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4458152c70 0x7f44581e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4457e00640 1 -- 172.21.15.123:0/1101982893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4458157050 con 0x7f44580c3fe0 2024-04-25T12:37:30.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4457e00640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c3fe0 0x7f44580cc3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4444032000 tx=0x7f4444004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:30.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4454c00640 1 -- 172.21.15.123:0/1101982893 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444040070 con 0x7f44580c3fe0 2024-04-25T12:37:30.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f4454c00640 1 -- 172.21.15.123:0/1101982893 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4444004240 con 0x7f44580c3fe0 2024-04-25T12:37:30.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.165+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4458157390 con 0x7f44580c3fe0 2024-04-25T12:37:30.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.166+0000 7f4454c00640 1 -- 172.21.15.123:0/1101982893 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444032830 con 0x7f44580c3fe0 2024-04-25T12:37:30.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.166+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445814efe0 con 0x7f44580c3fe0 2024-04-25T12:37:30.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.167+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4458076700 con 0x7f44580c3fe0 2024-04-25T12:37:30.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.168+0000 7f4454c00640 1 -- 172.21.15.123:0/1101982893 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4444043070 con 0x7f44580c3fe0 2024-04-25T12:37:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.169+0000 7f4454c00640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f44380744b0 0x7f44380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.169+0000 7f4454c00640 1 -- 172.21.15.123:0/1101982893 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f44440b5560 con 0x7f44580c3fe0 2024-04-25T12:37:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.169+0000 7f445ce00640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f44380744b0 0x7f44380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.170+0000 7f445ce00640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f44380744b0 0x7f44380769c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f44480095e0 tx=0x7f4448009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:30.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.174+0000 7f4454c00640 1 -- 172.21.15.123:0/1101982893 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f444400a020 con 0x7f44580c3fe0 2024-04-25T12:37:30.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.375+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f44580c8d40 con 0x7f44580c3fe0 2024-04-25T12:37:30.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.376+0000 7f4454c00640 1 -- 172.21.15.123:0/1101982893 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f444407f1f0 con 0x7f44580c3fe0 2024-04-25T12:37:30.375 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T12:37:30.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f44380744b0 msgr2=0x7f44380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f44380744b0 0x7f44380769c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f44480095e0 tx=0x7f4448009190 comp rx=0 tx=0).stop 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c3fe0 msgr2=0x7f44580cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c3fe0 0x7f44580cc3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4444032000 tx=0x7f4444004730 comp rx=0 tx=0).stop 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 shutdown_connections 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f44380744b0 0x7f44380769c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4458152c70 0x7f44581e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44580c3fe0 0x7f44580cc3e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 --2- 172.21.15.123:0/1101982893 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44580c2790 0x7f445814cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.379+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 >> 172.21.15.123:0/1101982893 conn(0x7f445806f440 msgr2=0x7f44580c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.380+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 shutdown_connections 2024-04-25T12:37:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.380+0000 7f445f800640 1 -- 172.21.15.123:0/1101982893 wait complete. 2024-04-25T12:37:30.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T12:37:30.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.675+0000 7f9dd3000640 1 Processor -- start 2024-04-25T12:37:30.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.676+0000 7f9dd3000640 1 -- start start 2024-04-25T12:37:30.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.677+0000 7f9dd3000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9dcc0c9b80 0x7f9dcc0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.677+0000 7f9dd3000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.677+0000 7f9dd3000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dcc156e60 con 0x7f9dcc159820 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dd3000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9dcc0c8ba0 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dd3000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9dcc15ca70 con 0x7f9dcc0c8250 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dd1000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9dcc159820 0x7f9dcc159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58752/0 (socket says 172.21.15.123:58752) 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dd1000640 1 -- 172.21.15.123:0/952848131 learned_addr learned my addr 172.21.15.123:0/952848131 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dcbe00640 1 --2- 172.21.15.123:0/952848131 >> v2:172.21.15.153:3300/0 conn(0x7f9dcc0c9b80 0x7f9dcc0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dc3400640 1 --2- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dcbe00640 1 -- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 msgr2=0x7f9dcc0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dcbe00640 1 --2- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dcbe00640 1 -- 172.21.15.123:0/952848131 >> v1:172.21.15.123:6789/0 conn(0x7f9dcc159820 legacy=0x7f9dcc159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dcb400640 1 -- 172.21.15.123:0/952848131 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374621892 0 0) 0x7f9dcc156e60 con 0x7f9dcc159820 2024-04-25T12:37:30.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.678+0000 7f9dcbe00640 1 -- 172.21.15.123:0/952848131 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc0c7130 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.679+0000 7f9dcbe00640 1 --2- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc0742b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0097a0 tx=0x7f9dbc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e8cd39cc506847 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:30.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.680+0000 7f9dcb400640 1 -- 172.21.15.123:0/952848131 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc037070 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.680+0000 7f9dcb400640 1 -- 172.21.15.123:0/952848131 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dbc032440 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.680+0000 7f9dcb400640 1 -- 172.21.15.123:0/952848131 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc032810 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 -- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc0c9b80 msgr2=0x7f9dcc0742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 --2- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc0742b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0097a0 tx=0x7f9dbc02a180 comp rx=0 tx=0).stop 2024-04-25T12:37:30.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 -- 172.21.15.123:0/952848131 shutdown_connections 2024-04-25T12:37:30.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 --2- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 --2- 172.21.15.123:0/952848131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc0742b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 -- 172.21.15.123:0/952848131 >> 172.21.15.123:0/952848131 conn(0x7f9dcc06c750 msgr2=0x7f9dcc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:30.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 -- 172.21.15.123:0/952848131 shutdown_connections 2024-04-25T12:37:30.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.681+0000 7f9dd3000640 1 -- 172.21.15.123:0/952848131 wait complete. 2024-04-25T12:37:30.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.683+0000 7f9dd3000640 1 Processor -- start 2024-04-25T12:37:30.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.683+0000 7f9dd3000640 1 -- start start 2024-04-25T12:37:30.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dd3000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dd3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dd3000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc159820 0x7f9dcc0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dd3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc15d310 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dd3000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc15d030 con 0x7f9dcc159820 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dd3000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9dcc15d5f0 con 0x7f9dcc0c8250 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc14e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41490/0 (socket says 172.21.15.123:41490) 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 -- 172.21.15.123:0/1108057720 learned_addr learned my addr 172.21.15.123:0/1108057720 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dcbe00640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dd1000640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc159820 0x7f9dcc0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 -- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 msgr2=0x7f9dcc0ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 -- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc159820 msgr2=0x7f9dcc0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc159820 0x7f9dcc0b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.685+0000 7f9dc3400640 1 -- 172.21.15.123:0/1108057720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc0acb80 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.686+0000 7f9dc3400640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc14e2e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9db400e870 tx=0x7f9db400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.686+0000 7f9dc8c00640 1 -- 172.21.15.123:0/1108057720 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4017070 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.686+0000 7f9dc8c00640 1 -- 172.21.15.123:0/1108057720 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db400ee60 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.686+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc0ab6a0 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.686+0000 7f9dc8c00640 1 -- 172.21.15.123:0/1108057720 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db400cb60 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.686+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc1f37a0 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.687+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dcc076700 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.689+0000 7f9dc8c00640 1 -- 172.21.15.123:0/1108057720 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9db401a070 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.690+0000 7f9dc8c00640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9da80744b0 0x7f9da80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:30.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.690+0000 7f9dc8c00640 1 -- 172.21.15.123:0/1108057720 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f9db4095890 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.690+0000 7f9dcbe00640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9da80744b0 0x7f9da80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:30.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.691+0000 7f9dcbe00640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9da80744b0 0x7f9da80769c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc034000 tx=0x7f9dbc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:30.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.694+0000 7f9dc8c00640 1 -- 172.21.15.123:0/1108057720 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9db405f530 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.895+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9dcc0c9590 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.896+0000 7f9dc8c00640 1 -- 172.21.15.123:0/1108057720 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f9db4063830 con 0x7f9dcc0c9b80 2024-04-25T12:37:30.895 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T12:37:30.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9da80744b0 msgr2=0x7f9da80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9da80744b0 0x7f9da80769c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc034000 tx=0x7f9dbc02e000 comp rx=0 tx=0).stop 2024-04-25T12:37:30.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dcc0c9b80 msgr2=0x7f9dcc14e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc14e2e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9db400e870 tx=0x7f9db400ed30 comp rx=0 tx=0).stop 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 shutdown_connections 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f9da80744b0 0x7f9da80769c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dcc159820 0x7f9dcc0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dcc0c9b80 0x7f9dcc14e2e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 --2- 172.21.15.123:0/1108057720 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9dcc0c8250 0x7f9dcc0ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.898+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 >> 172.21.15.123:0/1108057720 conn(0x7f9dcc06c750 msgr2=0x7f9dcc150570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.899+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 shutdown_connections 2024-04-25T12:37:30.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:30.899+0000 7f9dd3000640 1 -- 172.21.15.123:0/1108057720 wait complete. 2024-04-25T12:37:30.922 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T12:37:30.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:37:30.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T12:37:31.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:37:31.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:37:31.043 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T12:37:31.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:37:31.107 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T12:37:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T12:37:31.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:37:31.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T12:37:31.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:37:31.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T12:37:31.353 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T12:37:31.362 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T12:37:31.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:37:31.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.603+0000 7f9853200640 1 Processor -- start 2024-04-25T12:37:31.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.604+0000 7f9853200640 1 -- start start 2024-04-25T12:37:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.606+0000 7f9853200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854152a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.606+0000 7f9853200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9854159010 con 0x7f9854073bc0 2024-04-25T12:37:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.606+0000 7f9853200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f985406eb60 con 0x7f98540c3d40 2024-04-25T12:37:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.606+0000 7f9853200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f985406fc80 con 0x7f9854072d70 2024-04-25T12:37:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.606+0000 7f9851400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9854073bc0 0x7f9854158900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58760/0 (socket says 172.21.15.123:58760) 2024-04-25T12:37:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.606+0000 7f9851e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:31.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.606+0000 7f9851400640 1 -- 172.21.15.123:0/1474601668 learned_addr learned my addr 172.21.15.123:0/1474601668 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:31.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.607+0000 7f9850a00640 1 -- 172.21.15.123:0/1474601668 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884570121 0 0) 0x7f9854159010 con 0x7f9854073bc0 2024-04-25T12:37:31.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.607+0000 7f9851e00640 1 -- 172.21.15.123:0/1474601668 >> v1:172.21.15.153:6789/0 conn(0x7f98540c3d40 legacy=0x7f9854163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:31.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.607+0000 7f9851e00640 1 -- 172.21.15.123:0/1474601668 >> v1:172.21.15.123:6789/0 conn(0x7f9854073bc0 legacy=0x7f9854158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:31.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.607+0000 7f9851e00640 1 -- 172.21.15.123:0/1474601668 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985406ff40 con 0x7f9854072d70 2024-04-25T12:37:31.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.608+0000 7f9851e00640 1 --2- 172.21.15.123:0/1474601668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854152a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9844009750 tx=0x7f984402c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3db55ae9a41c535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:31.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.608+0000 7f9850a00640 1 -- 172.21.15.123:0/1474601668 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844039070 con 0x7f9854072d70 2024-04-25T12:37:31.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.608+0000 7f9850a00640 1 -- 172.21.15.123:0/1474601668 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9844034440 con 0x7f9854072d70 2024-04-25T12:37:31.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.608+0000 7f9850a00640 1 -- 172.21.15.123:0/1474601668 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844034810 con 0x7f9854072d70 2024-04-25T12:37:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.609+0000 7f9853200640 1 -- 172.21.15.123:0/1474601668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 msgr2=0x7f9854152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.609+0000 7f9853200640 1 --2- 172.21.15.123:0/1474601668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854152a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9844009750 tx=0x7f984402c220 comp rx=0 tx=0).stop 2024-04-25T12:37:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.609+0000 7f9853200640 1 -- 172.21.15.123:0/1474601668 shutdown_connections 2024-04-25T12:37:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.609+0000 7f9853200640 1 --2- 172.21.15.123:0/1474601668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854152a90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.609+0000 7f9853200640 1 -- 172.21.15.123:0/1474601668 >> 172.21.15.123:0/1474601668 conn(0x7f98540cd600 msgr2=0x7f98540cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.609+0000 7f9853200640 1 -- 172.21.15.123:0/1474601668 shutdown_connections 2024-04-25T12:37:31.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.609+0000 7f9853200640 1 -- 172.21.15.123:0/1474601668 wait complete. 2024-04-25T12:37:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.611+0000 7f9853200640 1 Processor -- start 2024-04-25T12:37:31.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.611+0000 7f9853200640 1 -- start start 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9853200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9853200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9854073bc0 0x7f9854147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9853200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98540c3d40 0x7f9854157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9853200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f985406ee00 con 0x7f98540c3d40 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9853200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98540c9a80 con 0x7f9854073bc0 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9853200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f985406f0e0 con 0x7f9854072d70 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9851e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9851400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9854073bc0 0x7f9854147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:31.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9851e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:57282/0 (socket says 172.21.15.123:57282) 2024-04-25T12:37:31.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9851e00640 1 -- 172.21.15.123:0/1889783265 learned_addr learned my addr 172.21.15.123:0/1889783265 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:31.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9852800640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98540c3d40 0x7f9854157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9851e00640 1 -- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9854073bc0 msgr2=0x7f9854147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.613+0000 7f9851e00640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9854073bc0 0x7f9854147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f9851e00640 1 -- 172.21.15.123:0/1889783265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98540c3d40 msgr2=0x7f9854157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f9851e00640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98540c3d40 0x7f9854157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f9851e00640 1 -- 172.21.15.123:0/1889783265 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98540cd340 con 0x7f9854072d70 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f9851400640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9854073bc0 0x7f9854147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f9851e00640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854156870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9844009870 tx=0x7f98440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f984a000640 1 -- 172.21.15.123:0/1889783265 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844043070 con 0x7f9854072d70 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f984a000640 1 -- 172.21.15.123:0/1889783265 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9844032ba0 con 0x7f9854072d70 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f984a000640 1 -- 172.21.15.123:0/1889783265 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844039070 con 0x7f9854072d70 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98540cd060 con 0x7f9854072d70 2024-04-25T12:37:31.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.614+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f985414bbc0 con 0x7f9854072d70 2024-04-25T12:37:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.615+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9854076da0 con 0x7f9854072d70 2024-04-25T12:37:31.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.617+0000 7f984a000640 1 -- 172.21.15.123:0/1889783265 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9844046070 con 0x7f9854072d70 2024-04-25T12:37:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.617+0000 7f984a000640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f98300744e0 0x7f98300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.618+0000 7f984a000640 1 -- 172.21.15.123:0/1889783265 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f98440b5700 con 0x7f9854072d70 2024-04-25T12:37:31.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.618+0000 7f9851400640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f98300744e0 0x7f98300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:31.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.618+0000 7f9851400640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f98300744e0 0x7f98300769f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f983c007f30 tx=0x7f983c007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:31.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.623+0000 7f984a000640 1 -- 172.21.15.123:0/1889783265 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9844034e20 con 0x7f9854072d70 2024-04-25T12:37:31.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.823+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98540cce10 con 0x7f98300744e0 2024-04-25T12:37:31.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.833+0000 7f984a000640 1 -- 172.21.15.123:0/1889783265 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333108 (secure 0 0 0) 0x7f98540cce10 con 0x7f98300744e0 2024-04-25T12:37:31.837 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:31.839 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.840+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f98300744e0 msgr2=0x7f98300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.840+0000 7f9853200640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f98300744e0 0x7f98300769f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f983c007f30 tx=0x7f983c007e80 comp rx=0 tx=0).stop 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.840+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 msgr2=0x7f9854156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.840+0000 7f9853200640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854156870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9844009870 tx=0x7f98440046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 shutdown_connections 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f98300744e0 0x7f98300769f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98540c3d40 0x7f9854157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9854073bc0 0x7f9854147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 --2- 172.21.15.123:0/1889783265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9854072d70 0x7f9854156870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 >> 172.21.15.123:0/1889783265 conn(0x7f98540cd600 msgr2=0x7f9854073210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 shutdown_connections 2024-04-25T12:37:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:31.841+0000 7f9853200640 1 -- 172.21.15.123:0/1889783265 wait complete. 2024-04-25T12:37:31.868 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-04-25T12:37:30.842554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631420,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":269,"num_write_kb":1725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234616,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":225579,"kb_avail":754740104,"statfs":{"total":773094113280,"available":772853866496,"internally_reserved":0,"allocated":9093120,"data_stored":5889728,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":230993875},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003566"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569452+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:37:21.569452+0000","last_peered":"2024-04-25T12:37:21.569452+0000","last_clean":"2024-04-25T12:37:21.569452+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:21.569452+0000","last_undegraded":"2024-04-25T12:37:21.569452+0000","last_fullsized":"2024-04-25T12:37:21.569452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908252+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:37:21.908252+0000","last_peered":"2024-04-25T12:37:21.908252+0000","last_clean":"2024-04-25T12:37:21.908252+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:21.908252+0000","last_undegraded":"2024-04-25T12:37:21.908252+0000","last_fullsized":"2024-04-25T12:37:21.908252+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402490+0000","last_change":"2024-04-25T12:37:01.222445+0000","last_active":"2024-04-25T12:37:21.402490+0000","last_peered":"2024-04-25T12:37:21.402490+0000","last_clean":"2024-04-25T12:37:21.402490+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:21.402490+0000","last_undegraded":"2024-04-25T12:37:21.402490+0000","last_fullsized":"2024-04-25T12:37:21.402490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907880+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:37:21.907880+0000","last_peered":"2024-04-25T12:37:21.907880+0000","last_clean":"2024-04-25T12:37:21.907880+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:21.907880+0000","last_undegraded":"2024-04-25T12:37:21.907880+0000","last_fullsized":"2024-04-25T12:37:21.907880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908265+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:37:21.908265+0000","last_peered":"2024-04-25T12:37:21.908265+0000","last_clean":"2024-04-25T12:37:21.908265+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:21.908265+0000","last_undegraded":"2024-04-25T12:37:21.908265+0000","last_fullsized":"2024-04-25T12:37:21.908265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404380+0000","last_change":"2024-04-25T12:37:01.216562+0000","last_active":"2024-04-25T12:37:21.404380+0000","last_peered":"2024-04-25T12:37:21.404380+0000","last_clean":"2024-04-25T12:37:21.404380+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:21.404380+0000","last_undegraded":"2024-04-25T12:37:21.404380+0000","last_fullsized":"2024-04-25T12:37:21.404380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910309+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:37:21.910309+0000","last_peered":"2024-04-25T12:37:21.910309+0000","last_clean":"2024-04-25T12:37:21.910309+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:21.910309+0000","last_undegraded":"2024-04-25T12:37:21.910309+0000","last_fullsized":"2024-04-25T12:37:21.910309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912549+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:37:21.912549+0000","last_peered":"2024-04-25T12:37:21.912549+0000","last_clean":"2024-04-25T12:37:21.912549+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:21.912549+0000","last_undegraded":"2024-04-25T12:37:21.912549+0000","last_fullsized":"2024-04-25T12:37:21.912549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405114+0000","last_change":"2024-04-25T12:37:01.226633+0000","last_active":"2024-04-25T12:37:21.405114+0000","last_peered":"2024-04-25T12:37:21.405114+0000","last_clean":"2024-04-25T12:37:21.405114+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:21.405114+0000","last_undegraded":"2024-04-25T12:37:21.405114+0000","last_fullsized":"2024-04-25T12:37:21.405114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.771833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910815+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:37:21.910815+0000","last_peered":"2024-04-25T12:37:21.910815+0000","last_clean":"2024-04-25T12:37:21.910815+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:21.910815+0000","last_undegraded":"2024-04-25T12:37:21.910815+0000","last_fullsized":"2024-04-25T12:37:21.910815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423689+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:37:21.423689+0000","last_peered":"2024-04-25T12:37:21.423689+0000","last_clean":"2024-04-25T12:37:21.423689+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:21.423689+0000","last_undegraded":"2024-04-25T12:37:21.423689+0000","last_fullsized":"2024-04-25T12:37:21.423689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404697+0000","last_change":"2024-04-25T12:37:01.221650+0000","last_active":"2024-04-25T12:37:21.404697+0000","last_peered":"2024-04-25T12:37:21.404697+0000","last_clean":"2024-04-25T12:37:21.404697+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:21.404697+0000","last_undegraded":"2024-04-25T12:37:21.404697+0000","last_fullsized":"2024-04-25T12:37:21.404697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422862+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:37:21.422862+0000","last_peered":"2024-04-25T12:37:21.422862+0000","last_clean":"2024-04-25T12:37:21.422862+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:21.422862+0000","last_undegraded":"2024-04-25T12:37:21.422862+0000","last_fullsized":"2024-04-25T12:37:21.422862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423223+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:37:21.423223+0000","last_peered":"2024-04-25T12:37:21.423223+0000","last_clean":"2024-04-25T12:37:21.423223+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:21.423223+0000","last_undegraded":"2024-04-25T12:37:21.423223+0000","last_fullsized":"2024-04-25T12:37:21.423223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909732+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:37:21.909732+0000","last_peered":"2024-04-25T12:37:21.909732+0000","last_clean":"2024-04-25T12:37:21.909732+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:21.909732+0000","last_undegraded":"2024-04-25T12:37:21.909732+0000","last_fullsized":"2024-04-25T12:37:21.909732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402959+0000","last_change":"2024-04-25T12:37:01.209490+0000","last_active":"2024-04-25T12:37:21.402959+0000","last_peered":"2024-04-25T12:37:21.402959+0000","last_clean":"2024-04-25T12:37:21.402959+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:21.402959+0000","last_undegraded":"2024-04-25T12:37:21.402959+0000","last_fullsized":"2024-04-25T12:37:21.402959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:46.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908543+0000","last_change":"2024-04-25T12:37:01.230485+0000","last_active":"2024-04-25T12:37:21.908543+0000","last_peered":"2024-04-25T12:37:21.908543+0000","last_clean":"2024-04-25T12:37:21.908543+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:21.908543+0000","last_undegraded":"2024-04-25T12:37:21.908543+0000","last_fullsized":"2024-04-25T12:37:21.908543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:47.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909989+0000","last_change":"2024-04-25T12:37:01.222195+0000","last_active":"2024-04-25T12:37:21.909989+0000","last_peered":"2024-04-25T12:37:21.909989+0000","last_clean":"2024-04-25T12:37:21.909989+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:21.909989+0000","last_undegraded":"2024-04-25T12:37:21.909989+0000","last_fullsized":"2024-04-25T12:37:21.909989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:35.550835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568700+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:37:21.568700+0000","last_peered":"2024-04-25T12:37:21.568700+0000","last_clean":"2024-04-25T12:37:21.568700+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:21.568700+0000","last_undegraded":"2024-04-25T12:37:21.568700+0000","last_fullsized":"2024-04-25T12:37:21.568700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569070+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:37:21.569070+0000","last_peered":"2024-04-25T12:37:21.569070+0000","last_clean":"2024-04-25T12:37:21.569070+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:21.569070+0000","last_undegraded":"2024-04-25T12:37:21.569070+0000","last_fullsized":"2024-04-25T12:37:21.569070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403488+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:37:21.403488+0000","last_peered":"2024-04-25T12:37:21.403488+0000","last_clean":"2024-04-25T12:37:21.403488+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:21.403488+0000","last_undegraded":"2024-04-25T12:37:21.403488+0000","last_fullsized":"2024-04-25T12:37:21.403488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422941+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:37:21.422941+0000","last_peered":"2024-04-25T12:37:21.422941+0000","last_clean":"2024-04-25T12:37:21.422941+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:21.422941+0000","last_undegraded":"2024-04-25T12:37:21.422941+0000","last_fullsized":"2024-04-25T12:37:21.422941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422292+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:37:21.422292+0000","last_peered":"2024-04-25T12:37:21.422292+0000","last_clean":"2024-04-25T12:37:21.422292+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:37:21.422292+0000","last_undegraded":"2024-04-25T12:37:21.422292+0000","last_fullsized":"2024-04-25T12:37:21.422292+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405968+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:37:21.405968+0000","last_peered":"2024-04-25T12:37:21.405968+0000","last_clean":"2024-04-25T12:37:21.405968+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:21.405968+0000","last_undegraded":"2024-04-25T12:37:21.405968+0000","last_fullsized":"2024-04-25T12:37:21.405968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569094+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:37:21.569094+0000","last_peered":"2024-04-25T12:37:21.569094+0000","last_clean":"2024-04-25T12:37:21.569094+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:21.569094+0000","last_undegraded":"2024-04-25T12:37:21.569094+0000","last_fullsized":"2024-04-25T12:37:21.569094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424634+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:37:21.424634+0000","last_peered":"2024-04-25T12:37:21.424634+0000","last_clean":"2024-04-25T12:37:21.424634+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:21.424634+0000","last_undegraded":"2024-04-25T12:37:21.424634+0000","last_fullsized":"2024-04-25T12:37:21.424634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404952+0000","last_change":"2024-04-25T12:37:01.226160+0000","last_active":"2024-04-25T12:37:21.404952+0000","last_peered":"2024-04-25T12:37:21.404952+0000","last_clean":"2024-04-25T12:37:21.404952+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:21.404952+0000","last_undegraded":"2024-04-25T12:37:21.404952+0000","last_fullsized":"2024-04-25T12:37:21.404952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:43.475095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569434+0000","last_change":"2024-04-25T12:37:01.206319+0000","last_active":"2024-04-25T12:37:21.569434+0000","last_peered":"2024-04-25T12:37:21.569434+0000","last_clean":"2024-04-25T12:37:21.569434+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:21.569434+0000","last_undegraded":"2024-04-25T12:37:21.569434+0000","last_fullsized":"2024-04-25T12:37:21.569434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"74'12","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569349+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:37:21.569349+0000","last_peered":"2024-04-25T12:37:21.569349+0000","last_clean":"2024-04-25T12:37:21.569349+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:21.569349+0000","last_undegraded":"2024-04-25T12:37:21.569349+0000","last_fullsized":"2024-04-25T12:37:21.569349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568614+0000","last_change":"2024-04-25T12:37:01.225992+0000","last_active":"2024-04-25T12:37:21.568614+0000","last_peered":"2024-04-25T12:37:21.568614+0000","last_clean":"2024-04-25T12:37:21.568614+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:21.568614+0000","last_undegraded":"2024-04-25T12:37:21.568614+0000","last_fullsized":"2024-04-25T12:37:21.568614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:05.029213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908795+0000","last_change":"2024-04-25T12:37:01.208764+0000","last_active":"2024-04-25T12:37:21.908795+0000","last_peered":"2024-04-25T12:37:21.908795+0000","last_clean":"2024-04-25T12:37:21.908795+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:21.908795+0000","last_undegraded":"2024-04-25T12:37:21.908795+0000","last_fullsized":"2024-04-25T12:37:21.908795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:38.538923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424069+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:37:21.424069+0000","last_peered":"2024-04-25T12:37:21.424069+0000","last_clean":"2024-04-25T12:37:21.424069+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:21.424069+0000","last_undegraded":"2024-04-25T12:37:21.424069+0000","last_fullsized":"2024-04-25T12:37:21.424069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908596+0000","last_change":"2024-04-25T12:37:01.225006+0000","last_active":"2024-04-25T12:37:21.908596+0000","last_peered":"2024-04-25T12:37:21.908596+0000","last_clean":"2024-04-25T12:37:21.908596+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:21.908596+0000","last_undegraded":"2024-04-25T12:37:21.908596+0000","last_fullsized":"2024-04-25T12:37:21.908596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405700+0000","last_change":"2024-04-25T12:37:01.222179+0000","last_active":"2024-04-25T12:37:21.405700+0000","last_peered":"2024-04-25T12:37:21.405700+0000","last_clean":"2024-04-25T12:37:21.405700+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:21.405700+0000","last_undegraded":"2024-04-25T12:37:21.405700+0000","last_fullsized":"2024-04-25T12:37:21.405700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907696+0000","last_change":"2024-04-25T12:37:01.211908+0000","last_active":"2024-04-25T12:37:21.907696+0000","last_peered":"2024-04-25T12:37:21.907696+0000","last_clean":"2024-04-25T12:37:21.907696+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:21.907696+0000","last_undegraded":"2024-04-25T12:37:21.907696+0000","last_fullsized":"2024-04-25T12:37:21.907696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:23.523737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908482+0000","last_change":"2024-04-25T12:37:01.229773+0000","last_active":"2024-04-25T12:37:21.908482+0000","last_peered":"2024-04-25T12:37:21.908482+0000","last_clean":"2024-04-25T12:37:21.908482+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:21.908482+0000","last_undegraded":"2024-04-25T12:37:21.908482+0000","last_fullsized":"2024-04-25T12:37:21.908482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:55.680417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402818+0000","last_change":"2024-04-25T12:37:01.223560+0000","last_active":"2024-04-25T12:37:21.402818+0000","last_peered":"2024-04-25T12:37:21.402818+0000","last_clean":"2024-04-25T12:37:21.402818+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:21.402818+0000","last_undegraded":"2024-04-25T12:37:21.402818+0000","last_fullsized":"2024-04-25T12:37:21.402818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:18.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424227+0000","last_change":"2024-04-25T12:37:01.227566+0000","last_active":"2024-04-25T12:37:21.424227+0000","last_peered":"2024-04-25T12:37:21.424227+0000","last_clean":"2024-04-25T12:37:21.424227+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:37:21.424227+0000","last_undegraded":"2024-04-25T12:37:21.424227+0000","last_fullsized":"2024-04-25T12:37:21.424227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:43.265214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908845+0000","last_change":"2024-04-25T12:37:01.226991+0000","last_active":"2024-04-25T12:37:21.908845+0000","last_peered":"2024-04-25T12:37:21.908845+0000","last_clean":"2024-04-25T12:37:21.908845+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:21.908845+0000","last_undegraded":"2024-04-25T12:37:21.908845+0000","last_fullsized":"2024-04-25T12:37:21.908845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:23.065480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907219+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:37:21.907219+0000","last_peered":"2024-04-25T12:37:21.907219+0000","last_clean":"2024-04-25T12:37:21.907219+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:21.907219+0000","last_undegraded":"2024-04-25T12:37:21.907219+0000","last_fullsized":"2024-04-25T12:37:21.907219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'7","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402648+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:37:21.402648+0000","last_peered":"2024-04-25T12:37:21.402648+0000","last_clean":"2024-04-25T12:37:21.402648+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:21.402648+0000","last_undegraded":"2024-04-25T12:37:21.402648+0000","last_fullsized":"2024-04-25T12:37:21.402648+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907983+0000","last_change":"2024-04-25T12:37:01.222254+0000","last_active":"2024-04-25T12:37:21.907983+0000","last_peered":"2024-04-25T12:37:21.907983+0000","last_clean":"2024-04-25T12:37:21.907983+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:21.907983+0000","last_undegraded":"2024-04-25T12:37:21.907983+0000","last_fullsized":"2024-04-25T12:37:21.907983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908045+0000","last_change":"2024-04-25T12:37:01.224799+0000","last_active":"2024-04-25T12:37:21.908045+0000","last_peered":"2024-04-25T12:37:21.908045+0000","last_clean":"2024-04-25T12:37:21.908045+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:21.908045+0000","last_undegraded":"2024-04-25T12:37:21.908045+0000","last_fullsized":"2024-04-25T12:37:21.908045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:24.737805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423362+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:37:21.423362+0000","last_peered":"2024-04-25T12:37:21.423362+0000","last_clean":"2024-04-25T12:37:21.423362+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:21.423362+0000","last_undegraded":"2024-04-25T12:37:21.423362+0000","last_fullsized":"2024-04-25T12:37:21.423362+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908375+0000","last_change":"2024-04-25T12:37:01.226465+0000","last_active":"2024-04-25T12:37:21.908375+0000","last_peered":"2024-04-25T12:37:21.908375+0000","last_clean":"2024-04-25T12:37:21.908375+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:21.908375+0000","last_undegraded":"2024-04-25T12:37:21.908375+0000","last_fullsized":"2024-04-25T12:37:21.908375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:26.218787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568893+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:37:21.568893+0000","last_peered":"2024-04-25T12:37:21.568893+0000","last_clean":"2024-04-25T12:37:21.568893+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:21.568893+0000","last_undegraded":"2024-04-25T12:37:21.568893+0000","last_fullsized":"2024-04-25T12:37:21.568893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423614+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:37:21.423614+0000","last_peered":"2024-04-25T12:37:21.423614+0000","last_clean":"2024-04-25T12:37:21.423614+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:21.423614+0000","last_undegraded":"2024-04-25T12:37:21.423614+0000","last_fullsized":"2024-04-25T12:37:21.423614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910887+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:37:21.910887+0000","last_peered":"2024-04-25T12:37:21.910887+0000","last_clean":"2024-04-25T12:37:21.910887+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:21.910887+0000","last_undegraded":"2024-04-25T12:37:21.910887+0000","last_fullsized":"2024-04-25T12:37:21.910887+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423841+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:37:21.423841+0000","last_peered":"2024-04-25T12:37:21.423841+0000","last_clean":"2024-04-25T12:37:21.423841+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:21.423841+0000","last_undegraded":"2024-04-25T12:37:21.423841+0000","last_fullsized":"2024-04-25T12:37:21.423841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423763+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:37:21.423763+0000","last_peered":"2024-04-25T12:37:21.423763+0000","last_clean":"2024-04-25T12:37:21.423763+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:21.423763+0000","last_undegraded":"2024-04-25T12:37:21.423763+0000","last_fullsized":"2024-04-25T12:37:21.423763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908962+0000","last_change":"2024-04-25T12:37:01.242232+0000","last_active":"2024-04-25T12:37:21.908962+0000","last_peered":"2024-04-25T12:37:21.908962+0000","last_clean":"2024-04-25T12:37:21.908962+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:21.908962+0000","last_undegraded":"2024-04-25T12:37:21.908962+0000","last_fullsized":"2024-04-25T12:37:21.908962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:59.941469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404864+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:37:21.404864+0000","last_peered":"2024-04-25T12:37:21.404864+0000","last_clean":"2024-04-25T12:37:21.404864+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:21.404864+0000","last_undegraded":"2024-04-25T12:37:21.404864+0000","last_fullsized":"2024-04-25T12:37:21.404864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912041+0000","last_change":"2024-04-25T12:37:01.220247+0000","last_active":"2024-04-25T12:37:21.912041+0000","last_peered":"2024-04-25T12:37:21.912041+0000","last_clean":"2024-04-25T12:37:21.912041+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:21.912041+0000","last_undegraded":"2024-04-25T12:37:21.912041+0000","last_fullsized":"2024-04-25T12:37:21.912041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:13.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908088+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:37:21.908088+0000","last_peered":"2024-04-25T12:37:21.908088+0000","last_clean":"2024-04-25T12:37:21.908088+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:21.908088+0000","last_undegraded":"2024-04-25T12:37:21.908088+0000","last_fullsized":"2024-04-25T12:37:21.908088+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":112,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907766+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:37:21.907766+0000","last_peered":"2024-04-25T12:37:21.907766+0000","last_clean":"2024-04-25T12:37:21.907766+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:21.907766+0000","last_undegraded":"2024-04-25T12:37:21.907766+0000","last_fullsized":"2024-04-25T12:37:21.907766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911665+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:37:21.911665+0000","last_peered":"2024-04-25T12:37:21.911665+0000","last_clean":"2024-04-25T12:37:21.911665+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:21.911665+0000","last_undegraded":"2024-04-25T12:37:21.911665+0000","last_fullsized":"2024-04-25T12:37:21.911665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569231+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:37:21.569231+0000","last_peered":"2024-04-25T12:37:21.569231+0000","last_clean":"2024-04-25T12:37:21.569231+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:21.569231+0000","last_undegraded":"2024-04-25T12:37:21.569231+0000","last_fullsized":"2024-04-25T12:37:21.569231+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422865+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:37:21.422865+0000","last_peered":"2024-04-25T12:37:21.422865+0000","last_clean":"2024-04-25T12:37:21.422865+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:21.422865+0000","last_undegraded":"2024-04-25T12:37:21.422865+0000","last_fullsized":"2024-04-25T12:37:21.422865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406286+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:37:21.406286+0000","last_peered":"2024-04-25T12:37:21.406286+0000","last_clean":"2024-04-25T12:37:21.406286+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:21.406286+0000","last_undegraded":"2024-04-25T12:37:21.406286+0000","last_fullsized":"2024-04-25T12:37:21.406286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911823+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:37:21.911823+0000","last_peered":"2024-04-25T12:37:21.911823+0000","last_clean":"2024-04-25T12:37:21.911823+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:21.911823+0000","last_undegraded":"2024-04-25T12:37:21.911823+0000","last_fullsized":"2024-04-25T12:37:21.911823+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910437+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:37:21.910437+0000","last_peered":"2024-04-25T12:37:21.910437+0000","last_clean":"2024-04-25T12:37:21.910437+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:21.910437+0000","last_undegraded":"2024-04-25T12:37:21.910437+0000","last_fullsized":"2024-04-25T12:37:21.910437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422738+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:37:21.422738+0000","last_peered":"2024-04-25T12:37:21.422738+0000","last_clean":"2024-04-25T12:37:21.422738+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:21.422738+0000","last_undegraded":"2024-04-25T12:37:21.422738+0000","last_fullsized":"2024-04-25T12:37:21.422738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"75'16","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568873+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:37:21.568873+0000","last_peered":"2024-04-25T12:37:21.568873+0000","last_clean":"2024-04-25T12:37:21.568873+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:21.568873+0000","last_undegraded":"2024-04-25T12:37:21.568873+0000","last_fullsized":"2024-04-25T12:37:21.568873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908280+0000","last_change":"2024-04-25T12:37:01.226590+0000","last_active":"2024-04-25T12:37:21.908280+0000","last_peered":"2024-04-25T12:37:21.908280+0000","last_clean":"2024-04-25T12:37:21.908280+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:21.908280+0000","last_undegraded":"2024-04-25T12:37:21.908280+0000","last_fullsized":"2024-04-25T12:37:21.908280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.581831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910537+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:37:21.910537+0000","last_peered":"2024-04-25T12:37:21.910537+0000","last_clean":"2024-04-25T12:37:21.910537+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:21.910537+0000","last_undegraded":"2024-04-25T12:37:21.910537+0000","last_fullsized":"2024-04-25T12:37:21.910537+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910045+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:37:21.910045+0000","last_peered":"2024-04-25T12:37:21.910045+0000","last_clean":"2024-04-25T12:37:21.910045+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:21.910045+0000","last_undegraded":"2024-04-25T12:37:21.910045+0000","last_fullsized":"2024-04-25T12:37:21.910045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908364+0000","last_change":"2024-04-25T12:37:01.220767+0000","last_active":"2024-04-25T12:37:21.908364+0000","last_peered":"2024-04-25T12:37:21.908364+0000","last_clean":"2024-04-25T12:37:21.908364+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:21.908364+0000","last_undegraded":"2024-04-25T12:37:21.908364+0000","last_fullsized":"2024-04-25T12:37:21.908364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:15.874258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422741+0000","last_change":"2024-04-25T12:37:01.233439+0000","last_active":"2024-04-25T12:37:21.422741+0000","last_peered":"2024-04-25T12:37:21.422741+0000","last_clean":"2024-04-25T12:37:21.422741+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:37:21.422741+0000","last_undegraded":"2024-04-25T12:37:21.422741+0000","last_fullsized":"2024-04-25T12:37:21.422741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:11.957744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424042+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:37:21.424042+0000","last_peered":"2024-04-25T12:37:21.424042+0000","last_clean":"2024-04-25T12:37:21.424042+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:21.424042+0000","last_undegraded":"2024-04-25T12:37:21.424042+0000","last_fullsized":"2024-04-25T12:37:21.424042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908903+0000","last_change":"2024-04-25T12:37:01.225914+0000","last_active":"2024-04-25T12:37:21.908903+0000","last_peered":"2024-04-25T12:37:21.908903+0000","last_clean":"2024-04-25T12:37:21.908903+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:21.908903+0000","last_undegraded":"2024-04-25T12:37:21.908903+0000","last_fullsized":"2024-04-25T12:37:21.908903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:34.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908657+0000","last_change":"2024-04-25T12:37:01.229502+0000","last_active":"2024-04-25T12:37:21.908657+0000","last_peered":"2024-04-25T12:37:21.908657+0000","last_clean":"2024-04-25T12:37:21.908657+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:21.908657+0000","last_undegraded":"2024-04-25T12:37:21.908657+0000","last_fullsized":"2024-04-25T12:37:21.908657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:25.697423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403098+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:37:21.403098+0000","last_peered":"2024-04-25T12:37:21.403098+0000","last_clean":"2024-04-25T12:37:21.403098+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:21.403098+0000","last_undegraded":"2024-04-25T12:37:21.403098+0000","last_fullsized":"2024-04-25T12:37:21.403098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912070+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:37:21.912070+0000","last_peered":"2024-04-25T12:37:21.912070+0000","last_clean":"2024-04-25T12:37:21.912070+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:21.912070+0000","last_undegraded":"2024-04-25T12:37:21.912070+0000","last_fullsized":"2024-04-25T12:37:21.912070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:49.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403124+0000","last_change":"2024-04-25T12:37:01.207603+0000","last_active":"2024-04-25T12:37:21.403124+0000","last_peered":"2024-04-25T12:37:21.403124+0000","last_clean":"2024-04-25T12:37:21.403124+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:21.403124+0000","last_undegraded":"2024-04-25T12:37:21.403124+0000","last_fullsized":"2024-04-25T12:37:21.403124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:01.832860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"75'26","reported_seq":181,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405220+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:37:21.405220+0000","last_peered":"2024-04-25T12:37:21.405220+0000","last_clean":"2024-04-25T12:37:21.405220+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:21.405220+0000","last_undegraded":"2024-04-25T12:37:21.405220+0000","last_fullsized":"2024-04-25T12:37:21.405220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37030,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568703+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:37:21.568703+0000","last_peered":"2024-04-25T12:37:21.568703+0000","last_clean":"2024-04-25T12:37:21.568703+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:21.568703+0000","last_undegraded":"2024-04-25T12:37:21.568703+0000","last_fullsized":"2024-04-25T12:37:21.568703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423387+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:37:21.423387+0000","last_peered":"2024-04-25T12:37:21.423387+0000","last_clean":"2024-04-25T12:37:21.423387+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:21.423387+0000","last_undegraded":"2024-04-25T12:37:21.423387+0000","last_fullsized":"2024-04-25T12:37:21.423387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"75'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403382+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:37:21.403382+0000","last_peered":"2024-04-25T12:37:21.403382+0000","last_clean":"2024-04-25T12:37:21.403382+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:21.403382+0000","last_undegraded":"2024-04-25T12:37:21.403382+0000","last_fullsized":"2024-04-25T12:37:21.403382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405271+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:37:21.405271+0000","last_peered":"2024-04-25T12:37:21.405271+0000","last_clean":"2024-04-25T12:37:21.405271+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:21.405271+0000","last_undegraded":"2024-04-25T12:37:21.405271+0000","last_fullsized":"2024-04-25T12:37:21.405271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908126+0000","last_change":"2024-04-25T12:37:01.212753+0000","last_active":"2024-04-25T12:37:21.908126+0000","last_peered":"2024-04-25T12:37:21.908126+0000","last_clean":"2024-04-25T12:37:21.908126+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:21.908126+0000","last_undegraded":"2024-04-25T12:37:21.908126+0000","last_fullsized":"2024-04-25T12:37:21.908126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:07.000299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911540+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:37:21.911540+0000","last_peered":"2024-04-25T12:37:21.911540+0000","last_clean":"2024-04-25T12:37:21.911540+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:21.911540+0000","last_undegraded":"2024-04-25T12:37:21.911540+0000","last_fullsized":"2024-04-25T12:37:21.911540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402408+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:37:21.402408+0000","last_peered":"2024-04-25T12:37:21.402408+0000","last_clean":"2024-04-25T12:37:21.402408+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:21.402408+0000","last_undegraded":"2024-04-25T12:37:21.402408+0000","last_fullsized":"2024-04-25T12:37:21.402408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406162+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:37:21.406162+0000","last_peered":"2024-04-25T12:37:21.406162+0000","last_clean":"2024-04-25T12:37:21.406162+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:21.406162+0000","last_undegraded":"2024-04-25T12:37:21.406162+0000","last_fullsized":"2024-04-25T12:37:21.406162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913386+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:37:21.913386+0000","last_peered":"2024-04-25T12:37:21.913386+0000","last_clean":"2024-04-25T12:37:21.913386+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:21.913386+0000","last_undegraded":"2024-04-25T12:37:21.913386+0000","last_fullsized":"2024-04-25T12:37:21.913386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568895+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:37:21.568895+0000","last_peered":"2024-04-25T12:37:21.568895+0000","last_clean":"2024-04-25T12:37:21.568895+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:21.568895+0000","last_undegraded":"2024-04-25T12:37:21.568895+0000","last_fullsized":"2024-04-25T12:37:21.568895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568918+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:37:21.568918+0000","last_peered":"2024-04-25T12:37:21.568918+0000","last_clean":"2024-04-25T12:37:21.568918+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:21.568918+0000","last_undegraded":"2024-04-25T12:37:21.568918+0000","last_fullsized":"2024-04-25T12:37:21.568918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569073+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:37:21.569073+0000","last_peered":"2024-04-25T12:37:21.569073+0000","last_clean":"2024-04-25T12:37:21.569073+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:21.569073+0000","last_undegraded":"2024-04-25T12:37:21.569073+0000","last_fullsized":"2024-04-25T12:37:21.569073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423322+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:37:21.423322+0000","last_peered":"2024-04-25T12:37:21.423322+0000","last_clean":"2024-04-25T12:37:21.423322+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:21.423322+0000","last_undegraded":"2024-04-25T12:37:21.423322+0000","last_fullsized":"2024-04-25T12:37:21.423322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424396+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:37:21.424396+0000","last_peered":"2024-04-25T12:37:21.424396+0000","last_clean":"2024-04-25T12:37:21.424396+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:21.424396+0000","last_undegraded":"2024-04-25T12:37:21.424396+0000","last_fullsized":"2024-04-25T12:37:21.424396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907318+0000","last_change":"2024-04-25T12:37:01.204788+0000","last_active":"2024-04-25T12:37:21.907318+0000","last_peered":"2024-04-25T12:37:21.907318+0000","last_clean":"2024-04-25T12:37:21.907318+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:21.907318+0000","last_undegraded":"2024-04-25T12:37:21.907318+0000","last_fullsized":"2024-04-25T12:37:21.907318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:14.535103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908141+0000","last_change":"2024-04-25T12:37:01.217177+0000","last_active":"2024-04-25T12:37:21.908141+0000","last_peered":"2024-04-25T12:37:21.908141+0000","last_clean":"2024-04-25T12:37:21.908141+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:21.908141+0000","last_undegraded":"2024-04-25T12:37:21.908141+0000","last_fullsized":"2024-04-25T12:37:21.908141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:00.819525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423908+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:37:21.423908+0000","last_peered":"2024-04-25T12:37:21.423908+0000","last_clean":"2024-04-25T12:37:21.423908+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:21.423908+0000","last_undegraded":"2024-04-25T12:37:21.423908+0000","last_fullsized":"2024-04-25T12:37:21.423908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568987+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:37:21.568987+0000","last_peered":"2024-04-25T12:37:21.568987+0000","last_clean":"2024-04-25T12:37:21.568987+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:21.568987+0000","last_undegraded":"2024-04-25T12:37:21.568987+0000","last_fullsized":"2024-04-25T12:37:21.568987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569215+0000","last_change":"2024-04-25T12:37:01.209143+0000","last_active":"2024-04-25T12:37:21.569215+0000","last_peered":"2024-04-25T12:37:21.569215+0000","last_clean":"2024-04-25T12:37:21.569215+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:21.569215+0000","last_undegraded":"2024-04-25T12:37:21.569215+0000","last_fullsized":"2024-04-25T12:37:21.569215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:52.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907606+0000","last_change":"2024-04-25T12:37:01.223997+0000","last_active":"2024-04-25T12:37:21.907606+0000","last_peered":"2024-04-25T12:37:21.907606+0000","last_clean":"2024-04-25T12:37:21.907606+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:21.907606+0000","last_undegraded":"2024-04-25T12:37:21.907606+0000","last_fullsized":"2024-04-25T12:37:21.907606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:48.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911293+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:37:21.911293+0000","last_peered":"2024-04-25T12:37:21.911293+0000","last_clean":"2024-04-25T12:37:21.911293+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:21.911293+0000","last_undegraded":"2024-04-25T12:37:21.911293+0000","last_fullsized":"2024-04-25T12:37:21.911293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403285+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:37:21.403285+0000","last_peered":"2024-04-25T12:37:21.403285+0000","last_clean":"2024-04-25T12:37:21.403285+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:21.403285+0000","last_undegraded":"2024-04-25T12:37:21.403285+0000","last_fullsized":"2024-04-25T12:37:21.403285+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404558+0000","last_change":"2024-04-25T12:37:01.224898+0000","last_active":"2024-04-25T12:37:21.404558+0000","last_peered":"2024-04-25T12:37:21.404558+0000","last_clean":"2024-04-25T12:37:21.404558+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:21.404558+0000","last_undegraded":"2024-04-25T12:37:21.404558+0000","last_fullsized":"2024-04-25T12:37:21.404558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:18.991070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402556+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:37:21.402556+0000","last_peered":"2024-04-25T12:37:21.402556+0000","last_clean":"2024-04-25T12:37:21.402556+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:21.402556+0000","last_undegraded":"2024-04-25T12:37:21.402556+0000","last_fullsized":"2024-04-25T12:37:21.402556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422760+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:37:21.422760+0000","last_peered":"2024-04-25T12:37:21.422760+0000","last_clean":"2024-04-25T12:37:21.422760+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:21.422760+0000","last_undegraded":"2024-04-25T12:37:21.422760+0000","last_fullsized":"2024-04-25T12:37:21.422760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907860+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:37:21.907860+0000","last_peered":"2024-04-25T12:37:21.907860+0000","last_clean":"2024-04-25T12:37:21.907860+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:21.907860+0000","last_undegraded":"2024-04-25T12:37:21.907860+0000","last_fullsized":"2024-04-25T12:37:21.907860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908709+0000","last_change":"2024-04-25T12:37:01.228534+0000","last_active":"2024-04-25T12:37:21.908709+0000","last_peered":"2024-04-25T12:37:21.908709+0000","last_clean":"2024-04-25T12:37:21.908709+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:21.908709+0000","last_undegraded":"2024-04-25T12:37:21.908709+0000","last_fullsized":"2024-04-25T12:37:21.908709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:39.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907599+0000","last_change":"2024-04-25T12:37:01.210033+0000","last_active":"2024-04-25T12:37:21.907599+0000","last_peered":"2024-04-25T12:37:21.907599+0000","last_clean":"2024-04-25T12:37:21.907599+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:21.907599+0000","last_undegraded":"2024-04-25T12:37:21.907599+0000","last_fullsized":"2024-04-25T12:37:21.907599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:50.639901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406009+0000","last_change":"2024-04-25T12:37:01.232027+0000","last_active":"2024-04-25T12:37:21.406009+0000","last_peered":"2024-04-25T12:37:21.406009+0000","last_clean":"2024-04-25T12:37:21.406009+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:21.406009+0000","last_undegraded":"2024-04-25T12:37:21.406009+0000","last_fullsized":"2024-04-25T12:37:21.406009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908213+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:37:21.908213+0000","last_peered":"2024-04-25T12:37:21.908213+0000","last_clean":"2024-04-25T12:37:21.908213+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:21.908213+0000","last_undegraded":"2024-04-25T12:37:21.908213+0000","last_fullsized":"2024-04-25T12:37:21.908213+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423227+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:37:21.423227+0000","last_peered":"2024-04-25T12:37:21.423227+0000","last_clean":"2024-04-25T12:37:21.423227+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:21.423227+0000","last_undegraded":"2024-04-25T12:37:21.423227+0000","last_fullsized":"2024-04-25T12:37:21.423227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907222+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:37:21.907222+0000","last_peered":"2024-04-25T12:37:21.907222+0000","last_clean":"2024-04-25T12:37:21.907222+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:21.907222+0000","last_undegraded":"2024-04-25T12:37:21.907222+0000","last_fullsized":"2024-04-25T12:37:21.907222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911205+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:37:21.911205+0000","last_peered":"2024-04-25T12:37:21.911205+0000","last_clean":"2024-04-25T12:37:21.911205+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:21.911205+0000","last_undegraded":"2024-04-25T12:37:21.911205+0000","last_fullsized":"2024-04-25T12:37:21.911205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568777+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:37:21.568777+0000","last_peered":"2024-04-25T12:37:21.568777+0000","last_clean":"2024-04-25T12:37:21.568777+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:21.568777+0000","last_undegraded":"2024-04-25T12:37:21.568777+0000","last_fullsized":"2024-04-25T12:37:21.568777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910725+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:37:21.910725+0000","last_peered":"2024-04-25T12:37:21.910725+0000","last_clean":"2024-04-25T12:37:21.910725+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:21.910725+0000","last_undegraded":"2024-04-25T12:37:21.910725+0000","last_fullsized":"2024-04-25T12:37:21.910725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424005+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:37:21.424005+0000","last_peered":"2024-04-25T12:37:21.424005+0000","last_clean":"2024-04-25T12:37:21.424005+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:21.424005+0000","last_undegraded":"2024-04-25T12:37:21.424005+0000","last_fullsized":"2024-04-25T12:37:21.424005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403400+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:37:21.403400+0000","last_peered":"2024-04-25T12:37:21.403400+0000","last_clean":"2024-04-25T12:37:21.403400+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:21.403400+0000","last_undegraded":"2024-04-25T12:37:21.403400+0000","last_fullsized":"2024-04-25T12:37:21.403400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424575+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:37:21.424575+0000","last_peered":"2024-04-25T12:37:21.424575+0000","last_clean":"2024-04-25T12:37:21.424575+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:21.424575+0000","last_undegraded":"2024-04-25T12:37:21.424575+0000","last_fullsized":"2024-04-25T12:37:21.424575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423084+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:37:21.423084+0000","last_peered":"2024-04-25T12:37:21.423084+0000","last_clean":"2024-04-25T12:37:21.423084+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:21.423084+0000","last_undegraded":"2024-04-25T12:37:21.423084+0000","last_fullsized":"2024-04-25T12:37:21.423084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"70'1","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908366+0000","last_change":"2024-04-25T12:37:01.212249+0000","last_active":"2024-04-25T12:37:21.908366+0000","last_peered":"2024-04-25T12:37:21.908366+0000","last_clean":"2024-04-25T12:37:21.908366+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:21.908366+0000","last_undegraded":"2024-04-25T12:37:21.908366+0000","last_fullsized":"2024-04-25T12:37:21.908366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:47.658434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908490+0000","last_change":"2024-04-25T12:37:01.216745+0000","last_active":"2024-04-25T12:37:21.908490+0000","last_peered":"2024-04-25T12:37:21.908490+0000","last_clean":"2024-04-25T12:37:21.908490+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:21.908490+0000","last_undegraded":"2024-04-25T12:37:21.908490+0000","last_fullsized":"2024-04-25T12:37:21.908490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910517+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:37:21.910517+0000","last_peered":"2024-04-25T12:37:21.910517+0000","last_clean":"2024-04-25T12:37:21.910517+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:21.910517+0000","last_undegraded":"2024-04-25T12:37:21.910517+0000","last_fullsized":"2024-04-25T12:37:21.910517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910107+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:37:21.910107+0000","last_peered":"2024-04-25T12:37:21.910107+0000","last_clean":"2024-04-25T12:37:21.910107+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:21.910107+0000","last_undegraded":"2024-04-25T12:37:21.910107+0000","last_fullsized":"2024-04-25T12:37:21.910107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911235+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:37:21.911235+0000","last_peered":"2024-04-25T12:37:21.911235+0000","last_clean":"2024-04-25T12:37:21.911235+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:21.911235+0000","last_undegraded":"2024-04-25T12:37:21.911235+0000","last_fullsized":"2024-04-25T12:37:21.911235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913288+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:37:21.913288+0000","last_peered":"2024-04-25T12:37:21.913288+0000","last_clean":"2024-04-25T12:37:21.913288+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:21.913288+0000","last_undegraded":"2024-04-25T12:37:21.913288+0000","last_fullsized":"2024-04-25T12:37:21.913288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909742+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:37:21.909742+0000","last_peered":"2024-04-25T12:37:21.909742+0000","last_clean":"2024-04-25T12:37:21.909742+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:21.909742+0000","last_undegraded":"2024-04-25T12:37:21.909742+0000","last_fullsized":"2024-04-25T12:37:21.909742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405037+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:37:21.405037+0000","last_peered":"2024-04-25T12:37:21.405037+0000","last_clean":"2024-04-25T12:37:21.405037+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:21.405037+0000","last_undegraded":"2024-04-25T12:37:21.405037+0000","last_fullsized":"2024-04-25T12:37:21.405037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911087+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:37:21.911087+0000","last_peered":"2024-04-25T12:37:21.911087+0000","last_clean":"2024-04-25T12:37:21.911087+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:21.911087+0000","last_undegraded":"2024-04-25T12:37:21.911087+0000","last_fullsized":"2024-04-25T12:37:21.911087+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405035+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:37:21.405035+0000","last_peered":"2024-04-25T12:37:21.405035+0000","last_clean":"2024-04-25T12:37:21.405035+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:21.405035+0000","last_undegraded":"2024-04-25T12:37:21.405035+0000","last_fullsized":"2024-04-25T12:37:21.405035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907971+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:37:21.907971+0000","last_peered":"2024-04-25T12:37:21.907971+0000","last_clean":"2024-04-25T12:37:21.907971+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:21.907971+0000","last_undegraded":"2024-04-25T12:37:21.907971+0000","last_fullsized":"2024-04-25T12:37:21.907971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908194+0000","last_change":"2024-04-25T12:37:01.215943+0000","last_active":"2024-04-25T12:37:21.908194+0000","last_peered":"2024-04-25T12:37:21.908194+0000","last_clean":"2024-04-25T12:37:21.908194+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:21.908194+0000","last_undegraded":"2024-04-25T12:37:21.908194+0000","last_fullsized":"2024-04-25T12:37:21.908194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.518673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911529+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:37:21.911529+0000","last_peered":"2024-04-25T12:37:21.911529+0000","last_clean":"2024-04-25T12:37:21.911529+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:21.911529+0000","last_undegraded":"2024-04-25T12:37:21.911529+0000","last_fullsized":"2024-04-25T12:37:21.911529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41052,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":166,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82104,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462932,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29336,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342504,"statfs":{"total":96636764160,"available":96606724096,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":1207327,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1142784,"data_stored":616007,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37646,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38598,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:37:31.873 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:37:32.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.187+0000 7fe2c1800640 1 Processor -- start 2024-04-25T12:37:32.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.187+0000 7fe2c1800640 1 -- start start 2024-04-25T12:37:32.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.189+0000 7fe2c1800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe2bc0c9b80 0x7fe2bc0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2c1800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2bc0c8250 0x7fe2bc0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2c1800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2bc156e60 con 0x7fe2bc159820 2024-04-25T12:37:32.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2c1800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe2bc0c8ba0 con 0x7fe2bc0c8250 2024-04-25T12:37:32.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2c1800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe2bc15ca70 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2b2000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2bc0c8250 0x7fe2bc0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2bb400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe2bc159820 0x7fe2bc159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58768/0 (socket says 172.21.15.123:58768) 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2bb400640 1 -- 172.21.15.123:0/2300324018 learned_addr learned my addr 172.21.15.123:0/2300324018 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2baa00640 1 --2- 172.21.15.123:0/2300324018 >> v2:172.21.15.153:3300/0 conn(0x7fe2bc0c9b80 0x7fe2bc0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.190+0000 7fe2ba000640 1 -- 172.21.15.123:0/2300324018 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1489259485 0 0) 0x7fe2bc156e60 con 0x7fe2bc159820 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.191+0000 7fe2ba000640 1 -- 172.21.15.123:0/2300324018 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2a0003520 con 0x7fe2bc159820 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.191+0000 7fe2b2000640 1 -- 172.21.15.123:0/2300324018 >> v2:172.21.15.153:3300/0 conn(0x7fe2bc0c9b80 msgr2=0x7fe2bc0742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.191+0000 7fe2b2000640 1 --2- 172.21.15.123:0/2300324018 >> v2:172.21.15.153:3300/0 conn(0x7fe2bc0c9b80 0x7fe2bc0742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.191+0000 7fe2b2000640 1 -- 172.21.15.123:0/2300324018 >> v1:172.21.15.123:6789/0 conn(0x7fe2bc159820 legacy=0x7fe2bc159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.191+0000 7fe2b2000640 1 -- 172.21.15.123:0/2300324018 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc0c7130 con 0x7fe2bc0c8250 2024-04-25T12:37:32.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.192+0000 7fe2b2000640 1 --2- 172.21.15.123:0/2300324018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc0c8660 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc074b80 tx=0x7fe2a402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16589d4dce1d802 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:32.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.192+0000 7fe2ba000640 1 -- 172.21.15.123:0/2300324018 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4007740 con 0x7fe2bc0c8250 2024-04-25T12:37:32.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.192+0000 7fe2ba000640 1 -- 172.21.15.123:0/2300324018 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a40078e0 con 0x7fe2bc0c8250 2024-04-25T12:37:32.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.192+0000 7fe2ba000640 1 -- 172.21.15.123:0/2300324018 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4007c90 con 0x7fe2bc0c8250 2024-04-25T12:37:32.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 -- 172.21.15.123:0/2300324018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c8250 msgr2=0x7fe2bc0c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 --2- 172.21.15.123:0/2300324018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc0c8660 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc074b80 tx=0x7fe2a402a680 comp rx=0 tx=0).stop 2024-04-25T12:37:32.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 -- 172.21.15.123:0/2300324018 shutdown_connections 2024-04-25T12:37:32.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 --2- 172.21.15.123:0/2300324018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc0c8660 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 --2- 172.21.15.123:0/2300324018 >> v2:172.21.15.153:3300/0 conn(0x7fe2bc0c9b80 0x7fe2bc0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 -- 172.21.15.123:0/2300324018 >> 172.21.15.123:0/2300324018 conn(0x7fe2bc06c750 msgr2=0x7fe2bc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:32.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 -- 172.21.15.123:0/2300324018 shutdown_connections 2024-04-25T12:37:32.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.193+0000 7fe2c1800640 1 -- 172.21.15.123:0/2300324018 wait complete. 2024-04-25T12:37:32.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.195+0000 7fe2c1800640 1 Processor -- start 2024-04-25T12:37:32.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.196+0000 7fe2c1800640 1 -- start start 2024-04-25T12:37:32.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2c1800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc1f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2c1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c9b80 0x7fe2bc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2c1800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2bc159820 0x7fe2bc1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2c1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc15d310 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2c1800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc15d030 con 0x7fe2bc0c8250 2024-04-25T12:37:32.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2c1800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe2bc15d5f0 con 0x7fe2bc159820 2024-04-25T12:37:32.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2b2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c9b80 0x7fe2bc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2b2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c9b80 0x7fe2bc0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41516/0 (socket says 172.21.15.123:41516) 2024-04-25T12:37:32.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2b2000640 1 -- 172.21.15.123:0/3188210405 learned_addr learned my addr 172.21.15.123:0/3188210405 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:32.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2baa00640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc1f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.197+0000 7fe2bb400640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2bc159820 0x7fe2bc1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.198+0000 7fe2b2000640 1 -- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2bc159820 msgr2=0x7fe2bc1f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.198+0000 7fe2b2000640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2bc159820 0x7fe2bc1f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.198+0000 7fe2b2000640 1 -- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2bc0c8250 msgr2=0x7fe2bc1f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.198+0000 7fe2b2000640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc1f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.198+0000 7fe2b2000640 1 -- 172.21.15.123:0/3188210405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc1f3b00 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.198+0000 7fe2baa00640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc1f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.198+0000 7fe2b2000640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c9b80 0x7fe2bc0731a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009e80 tx=0x7fe2a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.199+0000 7fe2b3400640 1 -- 172.21.15.123:0/3188210405 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a403f070 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.199+0000 7fe2b3400640 1 -- 172.21.15.123:0/3188210405 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a4031c40 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.199+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc0ac980 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.199+0000 7fe2b3400640 1 -- 172.21.15.123:0/3188210405 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4009450 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.199+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1f30a0 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.200+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2bc0aae90 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.201+0000 7fe2b3400640 1 -- 172.21.15.123:0/3188210405 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe2a4044460 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.202+0000 7fe2b3400640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2980744e0 0x7fe2980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.202+0000 7fe2b3400640 1 -- 172.21.15.123:0/3188210405 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fe2a40b6c30 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.203+0000 7fe2baa00640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2980744e0 0x7fe2980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.203+0000 7fe2baa00640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2980744e0 0x7fe2980769f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009e90 tx=0x7fe2ac009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:32.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.207+0000 7fe2b3400640 1 -- 172.21.15.123:0/3188210405 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2a4080960 con 0x7fe2bc0c9b80 2024-04-25T12:37:32.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.403+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2bc076700 con 0x7fe2980744e0 2024-04-25T12:37:32.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.412+0000 7fe2b3400640 1 -- 172.21.15.123:0/3188210405 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333108 (secure 0 0 0) 0x7fe2bc076700 con 0x7fe2980744e0 2024-04-25T12:37:32.415 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:32.417 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.417+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2980744e0 msgr2=0x7fe2980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.417+0000 7fe2c1800640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2980744e0 0x7fe2980769f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009e90 tx=0x7fe2ac009190 comp rx=0 tx=0).stop 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c9b80 msgr2=0x7fe2bc0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c9b80 0x7fe2bc0731a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009e80 tx=0x7fe2a4004a40 comp rx=0 tx=0).stop 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 shutdown_connections 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe2980744e0 0x7fe2980769f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2bc159820 0x7fe2bc1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2bc0c9b80 0x7fe2bc0731a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 --2- 172.21.15.123:0/3188210405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2bc0c8250 0x7fe2bc1f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 >> 172.21.15.123:0/3188210405 conn(0x7fe2bc06c750 msgr2=0x7fe2bc0707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 shutdown_connections 2024-04-25T12:37:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.418+0000 7fe2c1800640 1 -- 172.21.15.123:0/3188210405 wait complete. 2024-04-25T12:37:32.442 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-04-25T12:37:30.842554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631420,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":269,"num_write_kb":1725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234616,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":225579,"kb_avail":754740104,"statfs":{"total":773094113280,"available":772853866496,"internally_reserved":0,"allocated":9093120,"data_stored":5889728,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":230993875},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003566"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569452+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:37:21.569452+0000","last_peered":"2024-04-25T12:37:21.569452+0000","last_clean":"2024-04-25T12:37:21.569452+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:21.569452+0000","last_undegraded":"2024-04-25T12:37:21.569452+0000","last_fullsized":"2024-04-25T12:37:21.569452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908252+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:37:21.908252+0000","last_peered":"2024-04-25T12:37:21.908252+0000","last_clean":"2024-04-25T12:37:21.908252+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:21.908252+0000","last_undegraded":"2024-04-25T12:37:21.908252+0000","last_fullsized":"2024-04-25T12:37:21.908252+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402490+0000","last_change":"2024-04-25T12:37:01.222445+0000","last_active":"2024-04-25T12:37:21.402490+0000","last_peered":"2024-04-25T12:37:21.402490+0000","last_clean":"2024-04-25T12:37:21.402490+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:21.402490+0000","last_undegraded":"2024-04-25T12:37:21.402490+0000","last_fullsized":"2024-04-25T12:37:21.402490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907880+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:37:21.907880+0000","last_peered":"2024-04-25T12:37:21.907880+0000","last_clean":"2024-04-25T12:37:21.907880+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:21.907880+0000","last_undegraded":"2024-04-25T12:37:21.907880+0000","last_fullsized":"2024-04-25T12:37:21.907880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908265+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:37:21.908265+0000","last_peered":"2024-04-25T12:37:21.908265+0000","last_clean":"2024-04-25T12:37:21.908265+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:21.908265+0000","last_undegraded":"2024-04-25T12:37:21.908265+0000","last_fullsized":"2024-04-25T12:37:21.908265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404380+0000","last_change":"2024-04-25T12:37:01.216562+0000","last_active":"2024-04-25T12:37:21.404380+0000","last_peered":"2024-04-25T12:37:21.404380+0000","last_clean":"2024-04-25T12:37:21.404380+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:21.404380+0000","last_undegraded":"2024-04-25T12:37:21.404380+0000","last_fullsized":"2024-04-25T12:37:21.404380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910309+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:37:21.910309+0000","last_peered":"2024-04-25T12:37:21.910309+0000","last_clean":"2024-04-25T12:37:21.910309+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:21.910309+0000","last_undegraded":"2024-04-25T12:37:21.910309+0000","last_fullsized":"2024-04-25T12:37:21.910309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912549+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:37:21.912549+0000","last_peered":"2024-04-25T12:37:21.912549+0000","last_clean":"2024-04-25T12:37:21.912549+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:21.912549+0000","last_undegraded":"2024-04-25T12:37:21.912549+0000","last_fullsized":"2024-04-25T12:37:21.912549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405114+0000","last_change":"2024-04-25T12:37:01.226633+0000","last_active":"2024-04-25T12:37:21.405114+0000","last_peered":"2024-04-25T12:37:21.405114+0000","last_clean":"2024-04-25T12:37:21.405114+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:21.405114+0000","last_undegraded":"2024-04-25T12:37:21.405114+0000","last_fullsized":"2024-04-25T12:37:21.405114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.771833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910815+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:37:21.910815+0000","last_peered":"2024-04-25T12:37:21.910815+0000","last_clean":"2024-04-25T12:37:21.910815+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:21.910815+0000","last_undegraded":"2024-04-25T12:37:21.910815+0000","last_fullsized":"2024-04-25T12:37:21.910815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423689+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:37:21.423689+0000","last_peered":"2024-04-25T12:37:21.423689+0000","last_clean":"2024-04-25T12:37:21.423689+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:21.423689+0000","last_undegraded":"2024-04-25T12:37:21.423689+0000","last_fullsized":"2024-04-25T12:37:21.423689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404697+0000","last_change":"2024-04-25T12:37:01.221650+0000","last_active":"2024-04-25T12:37:21.404697+0000","last_peered":"2024-04-25T12:37:21.404697+0000","last_clean":"2024-04-25T12:37:21.404697+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:21.404697+0000","last_undegraded":"2024-04-25T12:37:21.404697+0000","last_fullsized":"2024-04-25T12:37:21.404697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422862+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:37:21.422862+0000","last_peered":"2024-04-25T12:37:21.422862+0000","last_clean":"2024-04-25T12:37:21.422862+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:21.422862+0000","last_undegraded":"2024-04-25T12:37:21.422862+0000","last_fullsized":"2024-04-25T12:37:21.422862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423223+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:37:21.423223+0000","last_peered":"2024-04-25T12:37:21.423223+0000","last_clean":"2024-04-25T12:37:21.423223+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:21.423223+0000","last_undegraded":"2024-04-25T12:37:21.423223+0000","last_fullsized":"2024-04-25T12:37:21.423223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909732+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:37:21.909732+0000","last_peered":"2024-04-25T12:37:21.909732+0000","last_clean":"2024-04-25T12:37:21.909732+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:21.909732+0000","last_undegraded":"2024-04-25T12:37:21.909732+0000","last_fullsized":"2024-04-25T12:37:21.909732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402959+0000","last_change":"2024-04-25T12:37:01.209490+0000","last_active":"2024-04-25T12:37:21.402959+0000","last_peered":"2024-04-25T12:37:21.402959+0000","last_clean":"2024-04-25T12:37:21.402959+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:21.402959+0000","last_undegraded":"2024-04-25T12:37:21.402959+0000","last_fullsized":"2024-04-25T12:37:21.402959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:46.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908543+0000","last_change":"2024-04-25T12:37:01.230485+0000","last_active":"2024-04-25T12:37:21.908543+0000","last_peered":"2024-04-25T12:37:21.908543+0000","last_clean":"2024-04-25T12:37:21.908543+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:21.908543+0000","last_undegraded":"2024-04-25T12:37:21.908543+0000","last_fullsized":"2024-04-25T12:37:21.908543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:47.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909989+0000","last_change":"2024-04-25T12:37:01.222195+0000","last_active":"2024-04-25T12:37:21.909989+0000","last_peered":"2024-04-25T12:37:21.909989+0000","last_clean":"2024-04-25T12:37:21.909989+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:21.909989+0000","last_undegraded":"2024-04-25T12:37:21.909989+0000","last_fullsized":"2024-04-25T12:37:21.909989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:35.550835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568700+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:37:21.568700+0000","last_peered":"2024-04-25T12:37:21.568700+0000","last_clean":"2024-04-25T12:37:21.568700+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:21.568700+0000","last_undegraded":"2024-04-25T12:37:21.568700+0000","last_fullsized":"2024-04-25T12:37:21.568700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569070+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:37:21.569070+0000","last_peered":"2024-04-25T12:37:21.569070+0000","last_clean":"2024-04-25T12:37:21.569070+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:21.569070+0000","last_undegraded":"2024-04-25T12:37:21.569070+0000","last_fullsized":"2024-04-25T12:37:21.569070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403488+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:37:21.403488+0000","last_peered":"2024-04-25T12:37:21.403488+0000","last_clean":"2024-04-25T12:37:21.403488+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:21.403488+0000","last_undegraded":"2024-04-25T12:37:21.403488+0000","last_fullsized":"2024-04-25T12:37:21.403488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422941+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:37:21.422941+0000","last_peered":"2024-04-25T12:37:21.422941+0000","last_clean":"2024-04-25T12:37:21.422941+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:21.422941+0000","last_undegraded":"2024-04-25T12:37:21.422941+0000","last_fullsized":"2024-04-25T12:37:21.422941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422292+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:37:21.422292+0000","last_peered":"2024-04-25T12:37:21.422292+0000","last_clean":"2024-04-25T12:37:21.422292+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:37:21.422292+0000","last_undegraded":"2024-04-25T12:37:21.422292+0000","last_fullsized":"2024-04-25T12:37:21.422292+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405968+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:37:21.405968+0000","last_peered":"2024-04-25T12:37:21.405968+0000","last_clean":"2024-04-25T12:37:21.405968+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:21.405968+0000","last_undegraded":"2024-04-25T12:37:21.405968+0000","last_fullsized":"2024-04-25T12:37:21.405968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569094+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:37:21.569094+0000","last_peered":"2024-04-25T12:37:21.569094+0000","last_clean":"2024-04-25T12:37:21.569094+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:21.569094+0000","last_undegraded":"2024-04-25T12:37:21.569094+0000","last_fullsized":"2024-04-25T12:37:21.569094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424634+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:37:21.424634+0000","last_peered":"2024-04-25T12:37:21.424634+0000","last_clean":"2024-04-25T12:37:21.424634+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:21.424634+0000","last_undegraded":"2024-04-25T12:37:21.424634+0000","last_fullsized":"2024-04-25T12:37:21.424634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404952+0000","last_change":"2024-04-25T12:37:01.226160+0000","last_active":"2024-04-25T12:37:21.404952+0000","last_peered":"2024-04-25T12:37:21.404952+0000","last_clean":"2024-04-25T12:37:21.404952+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:21.404952+0000","last_undegraded":"2024-04-25T12:37:21.404952+0000","last_fullsized":"2024-04-25T12:37:21.404952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:43.475095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569434+0000","last_change":"2024-04-25T12:37:01.206319+0000","last_active":"2024-04-25T12:37:21.569434+0000","last_peered":"2024-04-25T12:37:21.569434+0000","last_clean":"2024-04-25T12:37:21.569434+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:21.569434+0000","last_undegraded":"2024-04-25T12:37:21.569434+0000","last_fullsized":"2024-04-25T12:37:21.569434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"74'12","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569349+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:37:21.569349+0000","last_peered":"2024-04-25T12:37:21.569349+0000","last_clean":"2024-04-25T12:37:21.569349+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:21.569349+0000","last_undegraded":"2024-04-25T12:37:21.569349+0000","last_fullsized":"2024-04-25T12:37:21.569349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568614+0000","last_change":"2024-04-25T12:37:01.225992+0000","last_active":"2024-04-25T12:37:21.568614+0000","last_peered":"2024-04-25T12:37:21.568614+0000","last_clean":"2024-04-25T12:37:21.568614+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:21.568614+0000","last_undegraded":"2024-04-25T12:37:21.568614+0000","last_fullsized":"2024-04-25T12:37:21.568614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:05.029213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908795+0000","last_change":"2024-04-25T12:37:01.208764+0000","last_active":"2024-04-25T12:37:21.908795+0000","last_peered":"2024-04-25T12:37:21.908795+0000","last_clean":"2024-04-25T12:37:21.908795+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:21.908795+0000","last_undegraded":"2024-04-25T12:37:21.908795+0000","last_fullsized":"2024-04-25T12:37:21.908795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:38.538923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424069+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:37:21.424069+0000","last_peered":"2024-04-25T12:37:21.424069+0000","last_clean":"2024-04-25T12:37:21.424069+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:21.424069+0000","last_undegraded":"2024-04-25T12:37:21.424069+0000","last_fullsized":"2024-04-25T12:37:21.424069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908596+0000","last_change":"2024-04-25T12:37:01.225006+0000","last_active":"2024-04-25T12:37:21.908596+0000","last_peered":"2024-04-25T12:37:21.908596+0000","last_clean":"2024-04-25T12:37:21.908596+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:21.908596+0000","last_undegraded":"2024-04-25T12:37:21.908596+0000","last_fullsized":"2024-04-25T12:37:21.908596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405700+0000","last_change":"2024-04-25T12:37:01.222179+0000","last_active":"2024-04-25T12:37:21.405700+0000","last_peered":"2024-04-25T12:37:21.405700+0000","last_clean":"2024-04-25T12:37:21.405700+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:21.405700+0000","last_undegraded":"2024-04-25T12:37:21.405700+0000","last_fullsized":"2024-04-25T12:37:21.405700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907696+0000","last_change":"2024-04-25T12:37:01.211908+0000","last_active":"2024-04-25T12:37:21.907696+0000","last_peered":"2024-04-25T12:37:21.907696+0000","last_clean":"2024-04-25T12:37:21.907696+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:21.907696+0000","last_undegraded":"2024-04-25T12:37:21.907696+0000","last_fullsized":"2024-04-25T12:37:21.907696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:23.523737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908482+0000","last_change":"2024-04-25T12:37:01.229773+0000","last_active":"2024-04-25T12:37:21.908482+0000","last_peered":"2024-04-25T12:37:21.908482+0000","last_clean":"2024-04-25T12:37:21.908482+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:21.908482+0000","last_undegraded":"2024-04-25T12:37:21.908482+0000","last_fullsized":"2024-04-25T12:37:21.908482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:55.680417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402818+0000","last_change":"2024-04-25T12:37:01.223560+0000","last_active":"2024-04-25T12:37:21.402818+0000","last_peered":"2024-04-25T12:37:21.402818+0000","last_clean":"2024-04-25T12:37:21.402818+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:21.402818+0000","last_undegraded":"2024-04-25T12:37:21.402818+0000","last_fullsized":"2024-04-25T12:37:21.402818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:18.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424227+0000","last_change":"2024-04-25T12:37:01.227566+0000","last_active":"2024-04-25T12:37:21.424227+0000","last_peered":"2024-04-25T12:37:21.424227+0000","last_clean":"2024-04-25T12:37:21.424227+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:37:21.424227+0000","last_undegraded":"2024-04-25T12:37:21.424227+0000","last_fullsized":"2024-04-25T12:37:21.424227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:43.265214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908845+0000","last_change":"2024-04-25T12:37:01.226991+0000","last_active":"2024-04-25T12:37:21.908845+0000","last_peered":"2024-04-25T12:37:21.908845+0000","last_clean":"2024-04-25T12:37:21.908845+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:21.908845+0000","last_undegraded":"2024-04-25T12:37:21.908845+0000","last_fullsized":"2024-04-25T12:37:21.908845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:23.065480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907219+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:37:21.907219+0000","last_peered":"2024-04-25T12:37:21.907219+0000","last_clean":"2024-04-25T12:37:21.907219+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:21.907219+0000","last_undegraded":"2024-04-25T12:37:21.907219+0000","last_fullsized":"2024-04-25T12:37:21.907219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'7","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402648+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:37:21.402648+0000","last_peered":"2024-04-25T12:37:21.402648+0000","last_clean":"2024-04-25T12:37:21.402648+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:21.402648+0000","last_undegraded":"2024-04-25T12:37:21.402648+0000","last_fullsized":"2024-04-25T12:37:21.402648+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907983+0000","last_change":"2024-04-25T12:37:01.222254+0000","last_active":"2024-04-25T12:37:21.907983+0000","last_peered":"2024-04-25T12:37:21.907983+0000","last_clean":"2024-04-25T12:37:21.907983+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:21.907983+0000","last_undegraded":"2024-04-25T12:37:21.907983+0000","last_fullsized":"2024-04-25T12:37:21.907983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908045+0000","last_change":"2024-04-25T12:37:01.224799+0000","last_active":"2024-04-25T12:37:21.908045+0000","last_peered":"2024-04-25T12:37:21.908045+0000","last_clean":"2024-04-25T12:37:21.908045+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:21.908045+0000","last_undegraded":"2024-04-25T12:37:21.908045+0000","last_fullsized":"2024-04-25T12:37:21.908045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:24.737805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423362+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:37:21.423362+0000","last_peered":"2024-04-25T12:37:21.423362+0000","last_clean":"2024-04-25T12:37:21.423362+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:21.423362+0000","last_undegraded":"2024-04-25T12:37:21.423362+0000","last_fullsized":"2024-04-25T12:37:21.423362+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908375+0000","last_change":"2024-04-25T12:37:01.226465+0000","last_active":"2024-04-25T12:37:21.908375+0000","last_peered":"2024-04-25T12:37:21.908375+0000","last_clean":"2024-04-25T12:37:21.908375+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:21.908375+0000","last_undegraded":"2024-04-25T12:37:21.908375+0000","last_fullsized":"2024-04-25T12:37:21.908375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:26.218787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568893+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:37:21.568893+0000","last_peered":"2024-04-25T12:37:21.568893+0000","last_clean":"2024-04-25T12:37:21.568893+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:21.568893+0000","last_undegraded":"2024-04-25T12:37:21.568893+0000","last_fullsized":"2024-04-25T12:37:21.568893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423614+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:37:21.423614+0000","last_peered":"2024-04-25T12:37:21.423614+0000","last_clean":"2024-04-25T12:37:21.423614+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:21.423614+0000","last_undegraded":"2024-04-25T12:37:21.423614+0000","last_fullsized":"2024-04-25T12:37:21.423614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910887+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:37:21.910887+0000","last_peered":"2024-04-25T12:37:21.910887+0000","last_clean":"2024-04-25T12:37:21.910887+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:21.910887+0000","last_undegraded":"2024-04-25T12:37:21.910887+0000","last_fullsized":"2024-04-25T12:37:21.910887+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423841+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:37:21.423841+0000","last_peered":"2024-04-25T12:37:21.423841+0000","last_clean":"2024-04-25T12:37:21.423841+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:21.423841+0000","last_undegraded":"2024-04-25T12:37:21.423841+0000","last_fullsized":"2024-04-25T12:37:21.423841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423763+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:37:21.423763+0000","last_peered":"2024-04-25T12:37:21.423763+0000","last_clean":"2024-04-25T12:37:21.423763+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:21.423763+0000","last_undegraded":"2024-04-25T12:37:21.423763+0000","last_fullsized":"2024-04-25T12:37:21.423763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908962+0000","last_change":"2024-04-25T12:37:01.242232+0000","last_active":"2024-04-25T12:37:21.908962+0000","last_peered":"2024-04-25T12:37:21.908962+0000","last_clean":"2024-04-25T12:37:21.908962+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:21.908962+0000","last_undegraded":"2024-04-25T12:37:21.908962+0000","last_fullsized":"2024-04-25T12:37:21.908962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:59.941469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404864+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:37:21.404864+0000","last_peered":"2024-04-25T12:37:21.404864+0000","last_clean":"2024-04-25T12:37:21.404864+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:21.404864+0000","last_undegraded":"2024-04-25T12:37:21.404864+0000","last_fullsized":"2024-04-25T12:37:21.404864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912041+0000","last_change":"2024-04-25T12:37:01.220247+0000","last_active":"2024-04-25T12:37:21.912041+0000","last_peered":"2024-04-25T12:37:21.912041+0000","last_clean":"2024-04-25T12:37:21.912041+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:21.912041+0000","last_undegraded":"2024-04-25T12:37:21.912041+0000","last_fullsized":"2024-04-25T12:37:21.912041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:13.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908088+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:37:21.908088+0000","last_peered":"2024-04-25T12:37:21.908088+0000","last_clean":"2024-04-25T12:37:21.908088+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:21.908088+0000","last_undegraded":"2024-04-25T12:37:21.908088+0000","last_fullsized":"2024-04-25T12:37:21.908088+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":112,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907766+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:37:21.907766+0000","last_peered":"2024-04-25T12:37:21.907766+0000","last_clean":"2024-04-25T12:37:21.907766+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:21.907766+0000","last_undegraded":"2024-04-25T12:37:21.907766+0000","last_fullsized":"2024-04-25T12:37:21.907766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911665+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:37:21.911665+0000","last_peered":"2024-04-25T12:37:21.911665+0000","last_clean":"2024-04-25T12:37:21.911665+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:21.911665+0000","last_undegraded":"2024-04-25T12:37:21.911665+0000","last_fullsized":"2024-04-25T12:37:21.911665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569231+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:37:21.569231+0000","last_peered":"2024-04-25T12:37:21.569231+0000","last_clean":"2024-04-25T12:37:21.569231+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:21.569231+0000","last_undegraded":"2024-04-25T12:37:21.569231+0000","last_fullsized":"2024-04-25T12:37:21.569231+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422865+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:37:21.422865+0000","last_peered":"2024-04-25T12:37:21.422865+0000","last_clean":"2024-04-25T12:37:21.422865+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:21.422865+0000","last_undegraded":"2024-04-25T12:37:21.422865+0000","last_fullsized":"2024-04-25T12:37:21.422865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406286+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:37:21.406286+0000","last_peered":"2024-04-25T12:37:21.406286+0000","last_clean":"2024-04-25T12:37:21.406286+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:21.406286+0000","last_undegraded":"2024-04-25T12:37:21.406286+0000","last_fullsized":"2024-04-25T12:37:21.406286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911823+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:37:21.911823+0000","last_peered":"2024-04-25T12:37:21.911823+0000","last_clean":"2024-04-25T12:37:21.911823+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:21.911823+0000","last_undegraded":"2024-04-25T12:37:21.911823+0000","last_fullsized":"2024-04-25T12:37:21.911823+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910437+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:37:21.910437+0000","last_peered":"2024-04-25T12:37:21.910437+0000","last_clean":"2024-04-25T12:37:21.910437+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:21.910437+0000","last_undegraded":"2024-04-25T12:37:21.910437+0000","last_fullsized":"2024-04-25T12:37:21.910437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422738+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:37:21.422738+0000","last_peered":"2024-04-25T12:37:21.422738+0000","last_clean":"2024-04-25T12:37:21.422738+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:21.422738+0000","last_undegraded":"2024-04-25T12:37:21.422738+0000","last_fullsized":"2024-04-25T12:37:21.422738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"75'16","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568873+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:37:21.568873+0000","last_peered":"2024-04-25T12:37:21.568873+0000","last_clean":"2024-04-25T12:37:21.568873+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:21.568873+0000","last_undegraded":"2024-04-25T12:37:21.568873+0000","last_fullsized":"2024-04-25T12:37:21.568873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908280+0000","last_change":"2024-04-25T12:37:01.226590+0000","last_active":"2024-04-25T12:37:21.908280+0000","last_peered":"2024-04-25T12:37:21.908280+0000","last_clean":"2024-04-25T12:37:21.908280+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:21.908280+0000","last_undegraded":"2024-04-25T12:37:21.908280+0000","last_fullsized":"2024-04-25T12:37:21.908280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.581831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910537+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:37:21.910537+0000","last_peered":"2024-04-25T12:37:21.910537+0000","last_clean":"2024-04-25T12:37:21.910537+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:21.910537+0000","last_undegraded":"2024-04-25T12:37:21.910537+0000","last_fullsized":"2024-04-25T12:37:21.910537+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910045+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:37:21.910045+0000","last_peered":"2024-04-25T12:37:21.910045+0000","last_clean":"2024-04-25T12:37:21.910045+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:21.910045+0000","last_undegraded":"2024-04-25T12:37:21.910045+0000","last_fullsized":"2024-04-25T12:37:21.910045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908364+0000","last_change":"2024-04-25T12:37:01.220767+0000","last_active":"2024-04-25T12:37:21.908364+0000","last_peered":"2024-04-25T12:37:21.908364+0000","last_clean":"2024-04-25T12:37:21.908364+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:21.908364+0000","last_undegraded":"2024-04-25T12:37:21.908364+0000","last_fullsized":"2024-04-25T12:37:21.908364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:15.874258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422741+0000","last_change":"2024-04-25T12:37:01.233439+0000","last_active":"2024-04-25T12:37:21.422741+0000","last_peered":"2024-04-25T12:37:21.422741+0000","last_clean":"2024-04-25T12:37:21.422741+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:37:21.422741+0000","last_undegraded":"2024-04-25T12:37:21.422741+0000","last_fullsized":"2024-04-25T12:37:21.422741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:11.957744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424042+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:37:21.424042+0000","last_peered":"2024-04-25T12:37:21.424042+0000","last_clean":"2024-04-25T12:37:21.424042+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:21.424042+0000","last_undegraded":"2024-04-25T12:37:21.424042+0000","last_fullsized":"2024-04-25T12:37:21.424042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908903+0000","last_change":"2024-04-25T12:37:01.225914+0000","last_active":"2024-04-25T12:37:21.908903+0000","last_peered":"2024-04-25T12:37:21.908903+0000","last_clean":"2024-04-25T12:37:21.908903+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:21.908903+0000","last_undegraded":"2024-04-25T12:37:21.908903+0000","last_fullsized":"2024-04-25T12:37:21.908903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:34.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908657+0000","last_change":"2024-04-25T12:37:01.229502+0000","last_active":"2024-04-25T12:37:21.908657+0000","last_peered":"2024-04-25T12:37:21.908657+0000","last_clean":"2024-04-25T12:37:21.908657+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:21.908657+0000","last_undegraded":"2024-04-25T12:37:21.908657+0000","last_fullsized":"2024-04-25T12:37:21.908657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:25.697423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403098+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:37:21.403098+0000","last_peered":"2024-04-25T12:37:21.403098+0000","last_clean":"2024-04-25T12:37:21.403098+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:21.403098+0000","last_undegraded":"2024-04-25T12:37:21.403098+0000","last_fullsized":"2024-04-25T12:37:21.403098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912070+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:37:21.912070+0000","last_peered":"2024-04-25T12:37:21.912070+0000","last_clean":"2024-04-25T12:37:21.912070+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:21.912070+0000","last_undegraded":"2024-04-25T12:37:21.912070+0000","last_fullsized":"2024-04-25T12:37:21.912070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:49.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403124+0000","last_change":"2024-04-25T12:37:01.207603+0000","last_active":"2024-04-25T12:37:21.403124+0000","last_peered":"2024-04-25T12:37:21.403124+0000","last_clean":"2024-04-25T12:37:21.403124+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:21.403124+0000","last_undegraded":"2024-04-25T12:37:21.403124+0000","last_fullsized":"2024-04-25T12:37:21.403124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:01.832860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"75'26","reported_seq":181,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405220+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:37:21.405220+0000","last_peered":"2024-04-25T12:37:21.405220+0000","last_clean":"2024-04-25T12:37:21.405220+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:21.405220+0000","last_undegraded":"2024-04-25T12:37:21.405220+0000","last_fullsized":"2024-04-25T12:37:21.405220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37030,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568703+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:37:21.568703+0000","last_peered":"2024-04-25T12:37:21.568703+0000","last_clean":"2024-04-25T12:37:21.568703+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:21.568703+0000","last_undegraded":"2024-04-25T12:37:21.568703+0000","last_fullsized":"2024-04-25T12:37:21.568703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423387+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:37:21.423387+0000","last_peered":"2024-04-25T12:37:21.423387+0000","last_clean":"2024-04-25T12:37:21.423387+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:21.423387+0000","last_undegraded":"2024-04-25T12:37:21.423387+0000","last_fullsized":"2024-04-25T12:37:21.423387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"75'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403382+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:37:21.403382+0000","last_peered":"2024-04-25T12:37:21.403382+0000","last_clean":"2024-04-25T12:37:21.403382+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:21.403382+0000","last_undegraded":"2024-04-25T12:37:21.403382+0000","last_fullsized":"2024-04-25T12:37:21.403382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405271+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:37:21.405271+0000","last_peered":"2024-04-25T12:37:21.405271+0000","last_clean":"2024-04-25T12:37:21.405271+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:21.405271+0000","last_undegraded":"2024-04-25T12:37:21.405271+0000","last_fullsized":"2024-04-25T12:37:21.405271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908126+0000","last_change":"2024-04-25T12:37:01.212753+0000","last_active":"2024-04-25T12:37:21.908126+0000","last_peered":"2024-04-25T12:37:21.908126+0000","last_clean":"2024-04-25T12:37:21.908126+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:21.908126+0000","last_undegraded":"2024-04-25T12:37:21.908126+0000","last_fullsized":"2024-04-25T12:37:21.908126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:07.000299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911540+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:37:21.911540+0000","last_peered":"2024-04-25T12:37:21.911540+0000","last_clean":"2024-04-25T12:37:21.911540+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:21.911540+0000","last_undegraded":"2024-04-25T12:37:21.911540+0000","last_fullsized":"2024-04-25T12:37:21.911540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402408+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:37:21.402408+0000","last_peered":"2024-04-25T12:37:21.402408+0000","last_clean":"2024-04-25T12:37:21.402408+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:21.402408+0000","last_undegraded":"2024-04-25T12:37:21.402408+0000","last_fullsized":"2024-04-25T12:37:21.402408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406162+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:37:21.406162+0000","last_peered":"2024-04-25T12:37:21.406162+0000","last_clean":"2024-04-25T12:37:21.406162+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:21.406162+0000","last_undegraded":"2024-04-25T12:37:21.406162+0000","last_fullsized":"2024-04-25T12:37:21.406162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913386+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:37:21.913386+0000","last_peered":"2024-04-25T12:37:21.913386+0000","last_clean":"2024-04-25T12:37:21.913386+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:21.913386+0000","last_undegraded":"2024-04-25T12:37:21.913386+0000","last_fullsized":"2024-04-25T12:37:21.913386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568895+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:37:21.568895+0000","last_peered":"2024-04-25T12:37:21.568895+0000","last_clean":"2024-04-25T12:37:21.568895+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:21.568895+0000","last_undegraded":"2024-04-25T12:37:21.568895+0000","last_fullsized":"2024-04-25T12:37:21.568895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568918+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:37:21.568918+0000","last_peered":"2024-04-25T12:37:21.568918+0000","last_clean":"2024-04-25T12:37:21.568918+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:21.568918+0000","last_undegraded":"2024-04-25T12:37:21.568918+0000","last_fullsized":"2024-04-25T12:37:21.568918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569073+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:37:21.569073+0000","last_peered":"2024-04-25T12:37:21.569073+0000","last_clean":"2024-04-25T12:37:21.569073+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:21.569073+0000","last_undegraded":"2024-04-25T12:37:21.569073+0000","last_fullsized":"2024-04-25T12:37:21.569073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423322+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:37:21.423322+0000","last_peered":"2024-04-25T12:37:21.423322+0000","last_clean":"2024-04-25T12:37:21.423322+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:21.423322+0000","last_undegraded":"2024-04-25T12:37:21.423322+0000","last_fullsized":"2024-04-25T12:37:21.423322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424396+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:37:21.424396+0000","last_peered":"2024-04-25T12:37:21.424396+0000","last_clean":"2024-04-25T12:37:21.424396+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:21.424396+0000","last_undegraded":"2024-04-25T12:37:21.424396+0000","last_fullsized":"2024-04-25T12:37:21.424396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907318+0000","last_change":"2024-04-25T12:37:01.204788+0000","last_active":"2024-04-25T12:37:21.907318+0000","last_peered":"2024-04-25T12:37:21.907318+0000","last_clean":"2024-04-25T12:37:21.907318+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:21.907318+0000","last_undegraded":"2024-04-25T12:37:21.907318+0000","last_fullsized":"2024-04-25T12:37:21.907318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:14.535103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908141+0000","last_change":"2024-04-25T12:37:01.217177+0000","last_active":"2024-04-25T12:37:21.908141+0000","last_peered":"2024-04-25T12:37:21.908141+0000","last_clean":"2024-04-25T12:37:21.908141+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:21.908141+0000","last_undegraded":"2024-04-25T12:37:21.908141+0000","last_fullsized":"2024-04-25T12:37:21.908141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:00.819525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423908+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:37:21.423908+0000","last_peered":"2024-04-25T12:37:21.423908+0000","last_clean":"2024-04-25T12:37:21.423908+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:21.423908+0000","last_undegraded":"2024-04-25T12:37:21.423908+0000","last_fullsized":"2024-04-25T12:37:21.423908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568987+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:37:21.568987+0000","last_peered":"2024-04-25T12:37:21.568987+0000","last_clean":"2024-04-25T12:37:21.568987+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:21.568987+0000","last_undegraded":"2024-04-25T12:37:21.568987+0000","last_fullsized":"2024-04-25T12:37:21.568987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569215+0000","last_change":"2024-04-25T12:37:01.209143+0000","last_active":"2024-04-25T12:37:21.569215+0000","last_peered":"2024-04-25T12:37:21.569215+0000","last_clean":"2024-04-25T12:37:21.569215+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:21.569215+0000","last_undegraded":"2024-04-25T12:37:21.569215+0000","last_fullsized":"2024-04-25T12:37:21.569215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:52.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907606+0000","last_change":"2024-04-25T12:37:01.223997+0000","last_active":"2024-04-25T12:37:21.907606+0000","last_peered":"2024-04-25T12:37:21.907606+0000","last_clean":"2024-04-25T12:37:21.907606+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:21.907606+0000","last_undegraded":"2024-04-25T12:37:21.907606+0000","last_fullsized":"2024-04-25T12:37:21.907606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:48.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911293+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:37:21.911293+0000","last_peered":"2024-04-25T12:37:21.911293+0000","last_clean":"2024-04-25T12:37:21.911293+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:21.911293+0000","last_undegraded":"2024-04-25T12:37:21.911293+0000","last_fullsized":"2024-04-25T12:37:21.911293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403285+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:37:21.403285+0000","last_peered":"2024-04-25T12:37:21.403285+0000","last_clean":"2024-04-25T12:37:21.403285+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:21.403285+0000","last_undegraded":"2024-04-25T12:37:21.403285+0000","last_fullsized":"2024-04-25T12:37:21.403285+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404558+0000","last_change":"2024-04-25T12:37:01.224898+0000","last_active":"2024-04-25T12:37:21.404558+0000","last_peered":"2024-04-25T12:37:21.404558+0000","last_clean":"2024-04-25T12:37:21.404558+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:21.404558+0000","last_undegraded":"2024-04-25T12:37:21.404558+0000","last_fullsized":"2024-04-25T12:37:21.404558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:18.991070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402556+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:37:21.402556+0000","last_peered":"2024-04-25T12:37:21.402556+0000","last_clean":"2024-04-25T12:37:21.402556+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:21.402556+0000","last_undegraded":"2024-04-25T12:37:21.402556+0000","last_fullsized":"2024-04-25T12:37:21.402556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422760+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:37:21.422760+0000","last_peered":"2024-04-25T12:37:21.422760+0000","last_clean":"2024-04-25T12:37:21.422760+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:21.422760+0000","last_undegraded":"2024-04-25T12:37:21.422760+0000","last_fullsized":"2024-04-25T12:37:21.422760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907860+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:37:21.907860+0000","last_peered":"2024-04-25T12:37:21.907860+0000","last_clean":"2024-04-25T12:37:21.907860+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:21.907860+0000","last_undegraded":"2024-04-25T12:37:21.907860+0000","last_fullsized":"2024-04-25T12:37:21.907860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908709+0000","last_change":"2024-04-25T12:37:01.228534+0000","last_active":"2024-04-25T12:37:21.908709+0000","last_peered":"2024-04-25T12:37:21.908709+0000","last_clean":"2024-04-25T12:37:21.908709+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:21.908709+0000","last_undegraded":"2024-04-25T12:37:21.908709+0000","last_fullsized":"2024-04-25T12:37:21.908709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:39.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907599+0000","last_change":"2024-04-25T12:37:01.210033+0000","last_active":"2024-04-25T12:37:21.907599+0000","last_peered":"2024-04-25T12:37:21.907599+0000","last_clean":"2024-04-25T12:37:21.907599+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:21.907599+0000","last_undegraded":"2024-04-25T12:37:21.907599+0000","last_fullsized":"2024-04-25T12:37:21.907599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:50.639901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.406009+0000","last_change":"2024-04-25T12:37:01.232027+0000","last_active":"2024-04-25T12:37:21.406009+0000","last_peered":"2024-04-25T12:37:21.406009+0000","last_clean":"2024-04-25T12:37:21.406009+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:21.406009+0000","last_undegraded":"2024-04-25T12:37:21.406009+0000","last_fullsized":"2024-04-25T12:37:21.406009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908213+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:37:21.908213+0000","last_peered":"2024-04-25T12:37:21.908213+0000","last_clean":"2024-04-25T12:37:21.908213+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:21.908213+0000","last_undegraded":"2024-04-25T12:37:21.908213+0000","last_fullsized":"2024-04-25T12:37:21.908213+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423227+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:37:21.423227+0000","last_peered":"2024-04-25T12:37:21.423227+0000","last_clean":"2024-04-25T12:37:21.423227+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:21.423227+0000","last_undegraded":"2024-04-25T12:37:21.423227+0000","last_fullsized":"2024-04-25T12:37:21.423227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907222+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:37:21.907222+0000","last_peered":"2024-04-25T12:37:21.907222+0000","last_clean":"2024-04-25T12:37:21.907222+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:21.907222+0000","last_undegraded":"2024-04-25T12:37:21.907222+0000","last_fullsized":"2024-04-25T12:37:21.907222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911205+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:37:21.911205+0000","last_peered":"2024-04-25T12:37:21.911205+0000","last_clean":"2024-04-25T12:37:21.911205+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:21.911205+0000","last_undegraded":"2024-04-25T12:37:21.911205+0000","last_fullsized":"2024-04-25T12:37:21.911205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568777+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:37:21.568777+0000","last_peered":"2024-04-25T12:37:21.568777+0000","last_clean":"2024-04-25T12:37:21.568777+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:21.568777+0000","last_undegraded":"2024-04-25T12:37:21.568777+0000","last_fullsized":"2024-04-25T12:37:21.568777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910725+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:37:21.910725+0000","last_peered":"2024-04-25T12:37:21.910725+0000","last_clean":"2024-04-25T12:37:21.910725+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:21.910725+0000","last_undegraded":"2024-04-25T12:37:21.910725+0000","last_fullsized":"2024-04-25T12:37:21.910725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424005+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:37:21.424005+0000","last_peered":"2024-04-25T12:37:21.424005+0000","last_clean":"2024-04-25T12:37:21.424005+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:21.424005+0000","last_undegraded":"2024-04-25T12:37:21.424005+0000","last_fullsized":"2024-04-25T12:37:21.424005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403400+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:37:21.403400+0000","last_peered":"2024-04-25T12:37:21.403400+0000","last_clean":"2024-04-25T12:37:21.403400+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:21.403400+0000","last_undegraded":"2024-04-25T12:37:21.403400+0000","last_fullsized":"2024-04-25T12:37:21.403400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424575+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:37:21.424575+0000","last_peered":"2024-04-25T12:37:21.424575+0000","last_clean":"2024-04-25T12:37:21.424575+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:21.424575+0000","last_undegraded":"2024-04-25T12:37:21.424575+0000","last_fullsized":"2024-04-25T12:37:21.424575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423084+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:37:21.423084+0000","last_peered":"2024-04-25T12:37:21.423084+0000","last_clean":"2024-04-25T12:37:21.423084+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:21.423084+0000","last_undegraded":"2024-04-25T12:37:21.423084+0000","last_fullsized":"2024-04-25T12:37:21.423084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"70'1","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908366+0000","last_change":"2024-04-25T12:37:01.212249+0000","last_active":"2024-04-25T12:37:21.908366+0000","last_peered":"2024-04-25T12:37:21.908366+0000","last_clean":"2024-04-25T12:37:21.908366+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:21.908366+0000","last_undegraded":"2024-04-25T12:37:21.908366+0000","last_fullsized":"2024-04-25T12:37:21.908366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:47.658434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908490+0000","last_change":"2024-04-25T12:37:01.216745+0000","last_active":"2024-04-25T12:37:21.908490+0000","last_peered":"2024-04-25T12:37:21.908490+0000","last_clean":"2024-04-25T12:37:21.908490+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:21.908490+0000","last_undegraded":"2024-04-25T12:37:21.908490+0000","last_fullsized":"2024-04-25T12:37:21.908490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910517+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:37:21.910517+0000","last_peered":"2024-04-25T12:37:21.910517+0000","last_clean":"2024-04-25T12:37:21.910517+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:21.910517+0000","last_undegraded":"2024-04-25T12:37:21.910517+0000","last_fullsized":"2024-04-25T12:37:21.910517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910107+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:37:21.910107+0000","last_peered":"2024-04-25T12:37:21.910107+0000","last_clean":"2024-04-25T12:37:21.910107+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:21.910107+0000","last_undegraded":"2024-04-25T12:37:21.910107+0000","last_fullsized":"2024-04-25T12:37:21.910107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911235+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:37:21.911235+0000","last_peered":"2024-04-25T12:37:21.911235+0000","last_clean":"2024-04-25T12:37:21.911235+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:21.911235+0000","last_undegraded":"2024-04-25T12:37:21.911235+0000","last_fullsized":"2024-04-25T12:37:21.911235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913288+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:37:21.913288+0000","last_peered":"2024-04-25T12:37:21.913288+0000","last_clean":"2024-04-25T12:37:21.913288+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:21.913288+0000","last_undegraded":"2024-04-25T12:37:21.913288+0000","last_fullsized":"2024-04-25T12:37:21.913288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909742+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:37:21.909742+0000","last_peered":"2024-04-25T12:37:21.909742+0000","last_clean":"2024-04-25T12:37:21.909742+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:21.909742+0000","last_undegraded":"2024-04-25T12:37:21.909742+0000","last_fullsized":"2024-04-25T12:37:21.909742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405037+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:37:21.405037+0000","last_peered":"2024-04-25T12:37:21.405037+0000","last_clean":"2024-04-25T12:37:21.405037+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:21.405037+0000","last_undegraded":"2024-04-25T12:37:21.405037+0000","last_fullsized":"2024-04-25T12:37:21.405037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911087+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:37:21.911087+0000","last_peered":"2024-04-25T12:37:21.911087+0000","last_clean":"2024-04-25T12:37:21.911087+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:21.911087+0000","last_undegraded":"2024-04-25T12:37:21.911087+0000","last_fullsized":"2024-04-25T12:37:21.911087+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405035+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:37:21.405035+0000","last_peered":"2024-04-25T12:37:21.405035+0000","last_clean":"2024-04-25T12:37:21.405035+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:21.405035+0000","last_undegraded":"2024-04-25T12:37:21.405035+0000","last_fullsized":"2024-04-25T12:37:21.405035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907971+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:37:21.907971+0000","last_peered":"2024-04-25T12:37:21.907971+0000","last_clean":"2024-04-25T12:37:21.907971+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:21.907971+0000","last_undegraded":"2024-04-25T12:37:21.907971+0000","last_fullsized":"2024-04-25T12:37:21.907971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908194+0000","last_change":"2024-04-25T12:37:01.215943+0000","last_active":"2024-04-25T12:37:21.908194+0000","last_peered":"2024-04-25T12:37:21.908194+0000","last_clean":"2024-04-25T12:37:21.908194+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:21.908194+0000","last_undegraded":"2024-04-25T12:37:21.908194+0000","last_fullsized":"2024-04-25T12:37:21.908194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.518673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911529+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:37:21.911529+0000","last_peered":"2024-04-25T12:37:21.911529+0000","last_clean":"2024-04-25T12:37:21.911529+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:21.911529+0000","last_undegraded":"2024-04-25T12:37:21.911529+0000","last_fullsized":"2024-04-25T12:37:21.911529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41052,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":166,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82104,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462932,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29336,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342504,"statfs":{"total":96636764160,"available":96606724096,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":1207327,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1142784,"data_stored":616007,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37646,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38598,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:37:32.447 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T12:37:32.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:37:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.756+0000 7f8615000640 1 Processor -- start 2024-04-25T12:37:32.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.756+0000 7f8615000640 1 -- start start 2024-04-25T12:37:32.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f8615000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86100c9080 0x7f86101546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f8615000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c82b0 0x7f861015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f8615000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861015a330 con 0x7f861015e590 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f8615000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8610073b10 con 0x7f86100c9080 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f8615000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8610073df0 con 0x7f86100c82b0 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f860f400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86100c9080 0x7f86101546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f860fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f861015e590 0x7f861006c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58778/0 (socket says 172.21.15.123:58778) 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f860fe00640 1 -- 172.21.15.123:0/3487348563 learned_addr learned my addr 172.21.15.123:0/3487348563 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.758+0000 7f8607e00640 1 --2- 172.21.15.123:0/3487348563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c82b0 0x7f861015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.759+0000 7f860f400640 1 -- 172.21.15.123:0/3487348563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c82b0 msgr2=0x7f861015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.759+0000 7f860f400640 1 --2- 172.21.15.123:0/3487348563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c82b0 0x7f861015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.759+0000 7f860f400640 1 -- 172.21.15.123:0/3487348563 >> v1:172.21.15.123:6789/0 conn(0x7f861015e590 legacy=0x7f861006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.759+0000 7f860f400640 1 -- 172.21.15.123:0/3487348563 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86100740d0 con 0x7f86100c9080 2024-04-25T12:37:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.759+0000 7f860ea00640 1 -- 172.21.15.123:0/3487348563 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501465837 0 0) 0x7f861015a330 con 0x7f861015e590 2024-04-25T12:37:32.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.760+0000 7f860f400640 1 --2- 172.21.15.123:0/3487348563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c9080 0x7f86101546d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f861015a500 tx=0x7f860002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38554f4bf070538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:32.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.760+0000 7f860ea00640 1 -- 172.21.15.123:0/3487348563 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600037070 con 0x7f86100c9080 2024-04-25T12:37:32.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.760+0000 7f860ea00640 1 -- 172.21.15.123:0/3487348563 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8600032440 con 0x7f86100c9080 2024-04-25T12:37:32.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.760+0000 7f860ea00640 1 -- 172.21.15.123:0/3487348563 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600032810 con 0x7f86100c9080 2024-04-25T12:37:32.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 -- 172.21.15.123:0/3487348563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c9080 msgr2=0x7f86101546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 --2- 172.21.15.123:0/3487348563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c9080 0x7f86101546d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f861015a500 tx=0x7f860002a180 comp rx=0 tx=0).stop 2024-04-25T12:37:32.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 -- 172.21.15.123:0/3487348563 shutdown_connections 2024-04-25T12:37:32.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 --2- 172.21.15.123:0/3487348563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c82b0 0x7f861015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 --2- 172.21.15.123:0/3487348563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c9080 0x7f86101546d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 -- 172.21.15.123:0/3487348563 >> 172.21.15.123:0/3487348563 conn(0x7f86100cd600 msgr2=0x7f86100cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:32.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 -- 172.21.15.123:0/3487348563 shutdown_connections 2024-04-25T12:37:32.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.761+0000 7f8615000640 1 -- 172.21.15.123:0/3487348563 wait complete. 2024-04-25T12:37:32.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.763+0000 7f8615000640 1 Processor -- start 2024-04-25T12:37:32.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.764+0000 7f8615000640 1 -- start start 2024-04-25T12:37:32.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f8615000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c82b0 0x7f86101591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f8615000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c9080 0x7f8610159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f8615000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861015e590 0x7f8610151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f8615000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8610073760 con 0x7f86100c82b0 2024-04-25T12:37:32.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f8615000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f861015db30 con 0x7f861015e590 2024-04-25T12:37:32.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f8615000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8610152df0 con 0x7f86100c9080 2024-04-25T12:37:32.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f860f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c82b0 0x7f86101591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f860f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c82b0 0x7f86101591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41534/0 (socket says 172.21.15.123:41534) 2024-04-25T12:37:32.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f860f400640 1 -- 172.21.15.123:0/2235080563 learned_addr learned my addr 172.21.15.123:0/2235080563 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:32.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f8607e00640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c9080 0x7f8610159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.765+0000 7f860fe00640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861015e590 0x7f8610151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f860f400640 1 -- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c9080 msgr2=0x7f8610159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f860f400640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c9080 0x7f8610159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f860f400640 1 -- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861015e590 msgr2=0x7f8610151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f860f400640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861015e590 0x7f8610151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f860f400640 1 -- 172.21.15.123:0/2235080563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86101f1f80 con 0x7f86100c82b0 2024-04-25T12:37:32.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f860f400640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c82b0 0x7f86101591f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f86000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:32.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f8607400640 1 -- 172.21.15.123:0/2235080563 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600037040 con 0x7f86100c82b0 2024-04-25T12:37:32.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f8607400640 1 -- 172.21.15.123:0/2235080563 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8600042070 con 0x7f86100c82b0 2024-04-25T12:37:32.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f8607400640 1 -- 172.21.15.123:0/2235080563 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600030920 con 0x7f86100c82b0 2024-04-25T12:37:32.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.766+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8610154110 con 0x7f86100c82b0 2024-04-25T12:37:32.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.767+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101f24e0 con 0x7f86100c82b0 2024-04-25T12:37:32.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.768+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8610076da0 con 0x7f86100c82b0 2024-04-25T12:37:32.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.769+0000 7f8607400640 1 -- 172.21.15.123:0/2235080563 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8600030ac0 con 0x7f86100c82b0 2024-04-25T12:37:32.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.770+0000 7f8607400640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85ec0744b0 0x7f85ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:32.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.770+0000 7f8607400640 1 -- 172.21.15.123:0/2235080563 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f86000b5e30 con 0x7f86100c82b0 2024-04-25T12:37:32.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.770+0000 7f8607e00640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85ec0744b0 0x7f85ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:32.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.771+0000 7f8607e00640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85ec0744b0 0x7f85ec0769c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86100c4900 tx=0x7f85f8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:32.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.775+0000 7f8607400640 1 -- 172.21.15.123:0/2235080563 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f860004c020 con 0x7f86100c82b0 2024-04-25T12:37:32.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.975+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86100c8e60 con 0x7f85ec0744b0 2024-04-25T12:37:32.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.985+0000 7f8607400640 1 -- 172.21.15.123:0/2235080563 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333248 (secure 0 0 0) 0x7f86100c8e60 con 0x7f85ec0744b0 2024-04-25T12:37:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:32.991 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:37:32.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85ec0744b0 msgr2=0x7f85ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85ec0744b0 0x7f85ec0769c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86100c4900 tx=0x7f85f8009190 comp rx=0 tx=0).stop 2024-04-25T12:37:32.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c82b0 msgr2=0x7f86101591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:32.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c82b0 0x7f86101591f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f86000046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 shutdown_connections 2024-04-25T12:37:32.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f85ec0744b0 0x7f85ec0769c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861015e590 0x7f8610151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86100c9080 0x7f8610159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 --2- 172.21.15.123:0/2235080563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86100c82b0 0x7f86101591f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:32.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 >> 172.21.15.123:0/2235080563 conn(0x7f86100cd600 msgr2=0x7f86100c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:32.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 shutdown_connections 2024-04-25T12:37:32.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:32.993+0000 7f8615000640 1 -- 172.21.15.123:0/2235080563 wait complete. 2024-04-25T12:37:33.017 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-04-25T12:37:32.842991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631669,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":274,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":486,"ondisk_log_size":486,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238768,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":229739,"kb_avail":754735952,"statfs":{"total":773094113280,"available":772849614848,"internally_reserved":0,"allocated":9093120,"data_stored":5889977,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":235253715},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.442107"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569452+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:37:21.569452+0000","last_peered":"2024-04-25T12:37:21.569452+0000","last_clean":"2024-04-25T12:37:21.569452+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:21.569452+0000","last_undegraded":"2024-04-25T12:37:21.569452+0000","last_fullsized":"2024-04-25T12:37:21.569452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908252+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:37:21.908252+0000","last_peered":"2024-04-25T12:37:21.908252+0000","last_clean":"2024-04-25T12:37:21.908252+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:21.908252+0000","last_undegraded":"2024-04-25T12:37:21.908252+0000","last_fullsized":"2024-04-25T12:37:21.908252+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402490+0000","last_change":"2024-04-25T12:37:01.222445+0000","last_active":"2024-04-25T12:37:21.402490+0000","last_peered":"2024-04-25T12:37:21.402490+0000","last_clean":"2024-04-25T12:37:21.402490+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:21.402490+0000","last_undegraded":"2024-04-25T12:37:21.402490+0000","last_fullsized":"2024-04-25T12:37:21.402490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907880+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:37:21.907880+0000","last_peered":"2024-04-25T12:37:21.907880+0000","last_clean":"2024-04-25T12:37:21.907880+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:21.907880+0000","last_undegraded":"2024-04-25T12:37:21.907880+0000","last_fullsized":"2024-04-25T12:37:21.907880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908265+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:37:21.908265+0000","last_peered":"2024-04-25T12:37:21.908265+0000","last_clean":"2024-04-25T12:37:21.908265+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:21.908265+0000","last_undegraded":"2024-04-25T12:37:21.908265+0000","last_fullsized":"2024-04-25T12:37:21.908265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404380+0000","last_change":"2024-04-25T12:37:01.216562+0000","last_active":"2024-04-25T12:37:21.404380+0000","last_peered":"2024-04-25T12:37:21.404380+0000","last_clean":"2024-04-25T12:37:21.404380+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:21.404380+0000","last_undegraded":"2024-04-25T12:37:21.404380+0000","last_fullsized":"2024-04-25T12:37:21.404380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910309+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:37:21.910309+0000","last_peered":"2024-04-25T12:37:21.910309+0000","last_clean":"2024-04-25T12:37:21.910309+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:21.910309+0000","last_undegraded":"2024-04-25T12:37:21.910309+0000","last_fullsized":"2024-04-25T12:37:21.910309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:15.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.077468+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:37:31.077468+0000","last_peered":"2024-04-25T12:37:31.077468+0000","last_clean":"2024-04-25T12:37:31.077468+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:31.077468+0000","last_undegraded":"2024-04-25T12:37:31.077468+0000","last_fullsized":"2024-04-25T12:37:31.077468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405114+0000","last_change":"2024-04-25T12:37:01.226633+0000","last_active":"2024-04-25T12:37:21.405114+0000","last_peered":"2024-04-25T12:37:21.405114+0000","last_clean":"2024-04-25T12:37:21.405114+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:21.405114+0000","last_undegraded":"2024-04-25T12:37:21.405114+0000","last_fullsized":"2024-04-25T12:37:21.405114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.771833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910815+0000","last_change":"2024-04-25T12:34:12.927410+0000","last_active":"2024-04-25T12:37:21.910815+0000","last_peered":"2024-04-25T12:37:21.910815+0000","last_clean":"2024-04-25T12:37:21.910815+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:21.910815+0000","last_undegraded":"2024-04-25T12:37:21.910815+0000","last_fullsized":"2024-04-25T12:37:21.910815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:01.329789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423689+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:37:21.423689+0000","last_peered":"2024-04-25T12:37:21.423689+0000","last_clean":"2024-04-25T12:37:21.423689+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:21.423689+0000","last_undegraded":"2024-04-25T12:37:21.423689+0000","last_fullsized":"2024-04-25T12:37:21.423689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404697+0000","last_change":"2024-04-25T12:37:01.221650+0000","last_active":"2024-04-25T12:37:21.404697+0000","last_peered":"2024-04-25T12:37:21.404697+0000","last_clean":"2024-04-25T12:37:21.404697+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:21.404697+0000","last_undegraded":"2024-04-25T12:37:21.404697+0000","last_fullsized":"2024-04-25T12:37:21.404697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422862+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:37:21.422862+0000","last_peered":"2024-04-25T12:37:21.422862+0000","last_clean":"2024-04-25T12:37:21.422862+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:21.422862+0000","last_undegraded":"2024-04-25T12:37:21.422862+0000","last_fullsized":"2024-04-25T12:37:21.422862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423223+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:37:21.423223+0000","last_peered":"2024-04-25T12:37:21.423223+0000","last_clean":"2024-04-25T12:37:21.423223+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:21.423223+0000","last_undegraded":"2024-04-25T12:37:21.423223+0000","last_fullsized":"2024-04-25T12:37:21.423223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909732+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:37:21.909732+0000","last_peered":"2024-04-25T12:37:21.909732+0000","last_clean":"2024-04-25T12:37:21.909732+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:21.909732+0000","last_undegraded":"2024-04-25T12:37:21.909732+0000","last_fullsized":"2024-04-25T12:37:21.909732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:41.006345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.044483+0000","last_change":"2024-04-25T12:37:01.209490+0000","last_active":"2024-04-25T12:37:31.044483+0000","last_peered":"2024-04-25T12:37:31.044483+0000","last_clean":"2024-04-25T12:37:31.044483+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:31.044483+0000","last_undegraded":"2024-04-25T12:37:31.044483+0000","last_fullsized":"2024-04-25T12:37:31.044483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:46.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908543+0000","last_change":"2024-04-25T12:37:01.230485+0000","last_active":"2024-04-25T12:37:21.908543+0000","last_peered":"2024-04-25T12:37:21.908543+0000","last_clean":"2024-04-25T12:37:21.908543+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:21.908543+0000","last_undegraded":"2024-04-25T12:37:21.908543+0000","last_fullsized":"2024-04-25T12:37:21.908543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:47.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909989+0000","last_change":"2024-04-25T12:37:01.222195+0000","last_active":"2024-04-25T12:37:21.909989+0000","last_peered":"2024-04-25T12:37:21.909989+0000","last_clean":"2024-04-25T12:37:21.909989+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:21.909989+0000","last_undegraded":"2024-04-25T12:37:21.909989+0000","last_fullsized":"2024-04-25T12:37:21.909989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:35.550835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.038851+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:37:31.038851+0000","last_peered":"2024-04-25T12:37:31.038851+0000","last_clean":"2024-04-25T12:37:31.038851+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:31.038851+0000","last_undegraded":"2024-04-25T12:37:31.038851+0000","last_fullsized":"2024-04-25T12:37:31.038851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569070+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:37:21.569070+0000","last_peered":"2024-04-25T12:37:21.569070+0000","last_clean":"2024-04-25T12:37:21.569070+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:21.569070+0000","last_undegraded":"2024-04-25T12:37:21.569070+0000","last_fullsized":"2024-04-25T12:37:21.569070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403488+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:37:21.403488+0000","last_peered":"2024-04-25T12:37:21.403488+0000","last_clean":"2024-04-25T12:37:21.403488+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:21.403488+0000","last_undegraded":"2024-04-25T12:37:21.403488+0000","last_fullsized":"2024-04-25T12:37:21.403488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422941+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:37:21.422941+0000","last_peered":"2024-04-25T12:37:21.422941+0000","last_clean":"2024-04-25T12:37:21.422941+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:21.422941+0000","last_undegraded":"2024-04-25T12:37:21.422941+0000","last_fullsized":"2024-04-25T12:37:21.422941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422292+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:37:21.422292+0000","last_peered":"2024-04-25T12:37:21.422292+0000","last_clean":"2024-04-25T12:37:21.422292+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:37:21.422292+0000","last_undegraded":"2024-04-25T12:37:21.422292+0000","last_fullsized":"2024-04-25T12:37:21.422292+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405968+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:37:21.405968+0000","last_peered":"2024-04-25T12:37:21.405968+0000","last_clean":"2024-04-25T12:37:21.405968+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:21.405968+0000","last_undegraded":"2024-04-25T12:37:21.405968+0000","last_fullsized":"2024-04-25T12:37:21.405968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.038670+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:37:31.038670+0000","last_peered":"2024-04-25T12:37:31.038670+0000","last_clean":"2024-04-25T12:37:31.038670+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:31.038670+0000","last_undegraded":"2024-04-25T12:37:31.038670+0000","last_fullsized":"2024-04-25T12:37:31.038670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424634+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:37:21.424634+0000","last_peered":"2024-04-25T12:37:21.424634+0000","last_clean":"2024-04-25T12:37:21.424634+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:21.424634+0000","last_undegraded":"2024-04-25T12:37:21.424634+0000","last_fullsized":"2024-04-25T12:37:21.424634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404952+0000","last_change":"2024-04-25T12:37:01.226160+0000","last_active":"2024-04-25T12:37:21.404952+0000","last_peered":"2024-04-25T12:37:21.404952+0000","last_clean":"2024-04-25T12:37:21.404952+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:21.404952+0000","last_undegraded":"2024-04-25T12:37:21.404952+0000","last_fullsized":"2024-04-25T12:37:21.404952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:43.475095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.038094+0000","last_change":"2024-04-25T12:37:01.206319+0000","last_active":"2024-04-25T12:37:31.038094+0000","last_peered":"2024-04-25T12:37:31.038094+0000","last_clean":"2024-04-25T12:37:31.038094+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:31.038094+0000","last_undegraded":"2024-04-25T12:37:31.038094+0000","last_fullsized":"2024-04-25T12:37:31.038094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"93'13","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569402+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:37:21.569402+0000","last_peered":"2024-04-25T12:37:21.569402+0000","last_clean":"2024-04-25T12:37:21.569402+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:21.569402+0000","last_undegraded":"2024-04-25T12:37:21.569402+0000","last_fullsized":"2024-04-25T12:37:21.569402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568614+0000","last_change":"2024-04-25T12:37:01.225992+0000","last_active":"2024-04-25T12:37:21.568614+0000","last_peered":"2024-04-25T12:37:21.568614+0000","last_clean":"2024-04-25T12:37:21.568614+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:21.568614+0000","last_undegraded":"2024-04-25T12:37:21.568614+0000","last_fullsized":"2024-04-25T12:37:21.568614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:05.029213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.047473+0000","last_change":"2024-04-25T12:37:01.208764+0000","last_active":"2024-04-25T12:37:31.047473+0000","last_peered":"2024-04-25T12:37:31.047473+0000","last_clean":"2024-04-25T12:37:31.047473+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:31.047473+0000","last_undegraded":"2024-04-25T12:37:31.047473+0000","last_fullsized":"2024-04-25T12:37:31.047473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:38.538923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424069+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:37:21.424069+0000","last_peered":"2024-04-25T12:37:21.424069+0000","last_clean":"2024-04-25T12:37:21.424069+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:21.424069+0000","last_undegraded":"2024-04-25T12:37:21.424069+0000","last_fullsized":"2024-04-25T12:37:21.424069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908596+0000","last_change":"2024-04-25T12:37:01.225006+0000","last_active":"2024-04-25T12:37:21.908596+0000","last_peered":"2024-04-25T12:37:21.908596+0000","last_clean":"2024-04-25T12:37:21.908596+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:21.908596+0000","last_undegraded":"2024-04-25T12:37:21.908596+0000","last_fullsized":"2024-04-25T12:37:21.908596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.054034+0000","last_change":"2024-04-25T12:37:01.222179+0000","last_active":"2024-04-25T12:37:31.054034+0000","last_peered":"2024-04-25T12:37:31.054034+0000","last_clean":"2024-04-25T12:37:31.054034+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:31.054034+0000","last_undegraded":"2024-04-25T12:37:31.054034+0000","last_fullsized":"2024-04-25T12:37:31.054034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907696+0000","last_change":"2024-04-25T12:37:01.211908+0000","last_active":"2024-04-25T12:37:21.907696+0000","last_peered":"2024-04-25T12:37:21.907696+0000","last_clean":"2024-04-25T12:37:21.907696+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:21.907696+0000","last_undegraded":"2024-04-25T12:37:21.907696+0000","last_fullsized":"2024-04-25T12:37:21.907696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:23.523737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908482+0000","last_change":"2024-04-25T12:37:01.229773+0000","last_active":"2024-04-25T12:37:21.908482+0000","last_peered":"2024-04-25T12:37:21.908482+0000","last_clean":"2024-04-25T12:37:21.908482+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:21.908482+0000","last_undegraded":"2024-04-25T12:37:21.908482+0000","last_fullsized":"2024-04-25T12:37:21.908482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:55.680417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052653+0000","last_change":"2024-04-25T12:37:01.223560+0000","last_active":"2024-04-25T12:37:31.052653+0000","last_peered":"2024-04-25T12:37:31.052653+0000","last_clean":"2024-04-25T12:37:31.052653+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:31.052653+0000","last_undegraded":"2024-04-25T12:37:31.052653+0000","last_fullsized":"2024-04-25T12:37:31.052653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:18.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424227+0000","last_change":"2024-04-25T12:37:01.227566+0000","last_active":"2024-04-25T12:37:21.424227+0000","last_peered":"2024-04-25T12:37:21.424227+0000","last_clean":"2024-04-25T12:37:21.424227+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:37:21.424227+0000","last_undegraded":"2024-04-25T12:37:21.424227+0000","last_fullsized":"2024-04-25T12:37:21.424227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:43.265214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908845+0000","last_change":"2024-04-25T12:37:01.226991+0000","last_active":"2024-04-25T12:37:21.908845+0000","last_peered":"2024-04-25T12:37:21.908845+0000","last_clean":"2024-04-25T12:37:21.908845+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:21.908845+0000","last_undegraded":"2024-04-25T12:37:21.908845+0000","last_fullsized":"2024-04-25T12:37:21.908845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:23.065480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907219+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:37:21.907219+0000","last_peered":"2024-04-25T12:37:21.907219+0000","last_clean":"2024-04-25T12:37:21.907219+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:21.907219+0000","last_undegraded":"2024-04-25T12:37:21.907219+0000","last_fullsized":"2024-04-25T12:37:21.907219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'7","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402648+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:37:21.402648+0000","last_peered":"2024-04-25T12:37:21.402648+0000","last_clean":"2024-04-25T12:37:21.402648+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:21.402648+0000","last_undegraded":"2024-04-25T12:37:21.402648+0000","last_fullsized":"2024-04-25T12:37:21.402648+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907983+0000","last_change":"2024-04-25T12:37:01.222254+0000","last_active":"2024-04-25T12:37:21.907983+0000","last_peered":"2024-04-25T12:37:21.907983+0000","last_clean":"2024-04-25T12:37:21.907983+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:21.907983+0000","last_undegraded":"2024-04-25T12:37:21.907983+0000","last_fullsized":"2024-04-25T12:37:21.907983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052986+0000","last_change":"2024-04-25T12:37:01.224799+0000","last_active":"2024-04-25T12:37:31.052986+0000","last_peered":"2024-04-25T12:37:31.052986+0000","last_clean":"2024-04-25T12:37:31.052986+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:31.052986+0000","last_undegraded":"2024-04-25T12:37:31.052986+0000","last_fullsized":"2024-04-25T12:37:31.052986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:24.737805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423362+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:37:21.423362+0000","last_peered":"2024-04-25T12:37:21.423362+0000","last_clean":"2024-04-25T12:37:21.423362+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:21.423362+0000","last_undegraded":"2024-04-25T12:37:21.423362+0000","last_fullsized":"2024-04-25T12:37:21.423362+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908375+0000","last_change":"2024-04-25T12:37:01.226465+0000","last_active":"2024-04-25T12:37:21.908375+0000","last_peered":"2024-04-25T12:37:21.908375+0000","last_clean":"2024-04-25T12:37:21.908375+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:21.908375+0000","last_undegraded":"2024-04-25T12:37:21.908375+0000","last_fullsized":"2024-04-25T12:37:21.908375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:26.218787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.054718+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:37:31.054718+0000","last_peered":"2024-04-25T12:37:31.054718+0000","last_clean":"2024-04-25T12:37:31.054718+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:31.054718+0000","last_undegraded":"2024-04-25T12:37:31.054718+0000","last_fullsized":"2024-04-25T12:37:31.054718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423614+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:37:21.423614+0000","last_peered":"2024-04-25T12:37:21.423614+0000","last_clean":"2024-04-25T12:37:21.423614+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:21.423614+0000","last_undegraded":"2024-04-25T12:37:21.423614+0000","last_fullsized":"2024-04-25T12:37:21.423614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910887+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:37:21.910887+0000","last_peered":"2024-04-25T12:37:21.910887+0000","last_clean":"2024-04-25T12:37:21.910887+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:21.910887+0000","last_undegraded":"2024-04-25T12:37:21.910887+0000","last_fullsized":"2024-04-25T12:37:21.910887+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423841+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:37:21.423841+0000","last_peered":"2024-04-25T12:37:21.423841+0000","last_clean":"2024-04-25T12:37:21.423841+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:21.423841+0000","last_undegraded":"2024-04-25T12:37:21.423841+0000","last_fullsized":"2024-04-25T12:37:21.423841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423763+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:37:21.423763+0000","last_peered":"2024-04-25T12:37:21.423763+0000","last_clean":"2024-04-25T12:37:21.423763+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:21.423763+0000","last_undegraded":"2024-04-25T12:37:21.423763+0000","last_fullsized":"2024-04-25T12:37:21.423763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908962+0000","last_change":"2024-04-25T12:37:01.242232+0000","last_active":"2024-04-25T12:37:21.908962+0000","last_peered":"2024-04-25T12:37:21.908962+0000","last_clean":"2024-04-25T12:37:21.908962+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:21.908962+0000","last_undegraded":"2024-04-25T12:37:21.908962+0000","last_fullsized":"2024-04-25T12:37:21.908962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:59.941469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404864+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:37:21.404864+0000","last_peered":"2024-04-25T12:37:21.404864+0000","last_clean":"2024-04-25T12:37:21.404864+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:21.404864+0000","last_undegraded":"2024-04-25T12:37:21.404864+0000","last_fullsized":"2024-04-25T12:37:21.404864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.062818+0000","last_change":"2024-04-25T12:37:01.220247+0000","last_active":"2024-04-25T12:37:31.062818+0000","last_peered":"2024-04-25T12:37:31.062818+0000","last_clean":"2024-04-25T12:37:31.062818+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:31.062818+0000","last_undegraded":"2024-04-25T12:37:31.062818+0000","last_fullsized":"2024-04-25T12:37:31.062818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:13.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908088+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:37:21.908088+0000","last_peered":"2024-04-25T12:37:21.908088+0000","last_clean":"2024-04-25T12:37:21.908088+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:21.908088+0000","last_undegraded":"2024-04-25T12:37:21.908088+0000","last_fullsized":"2024-04-25T12:37:21.908088+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":112,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907766+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:37:21.907766+0000","last_peered":"2024-04-25T12:37:21.907766+0000","last_clean":"2024-04-25T12:37:21.907766+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:21.907766+0000","last_undegraded":"2024-04-25T12:37:21.907766+0000","last_fullsized":"2024-04-25T12:37:21.907766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911665+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:37:21.911665+0000","last_peered":"2024-04-25T12:37:21.911665+0000","last_clean":"2024-04-25T12:37:21.911665+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:21.911665+0000","last_undegraded":"2024-04-25T12:37:21.911665+0000","last_fullsized":"2024-04-25T12:37:21.911665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:23.447606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569231+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:37:21.569231+0000","last_peered":"2024-04-25T12:37:21.569231+0000","last_clean":"2024-04-25T12:37:21.569231+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:21.569231+0000","last_undegraded":"2024-04-25T12:37:21.569231+0000","last_fullsized":"2024-04-25T12:37:21.569231+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422865+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:37:21.422865+0000","last_peered":"2024-04-25T12:37:21.422865+0000","last_clean":"2024-04-25T12:37:21.422865+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:21.422865+0000","last_undegraded":"2024-04-25T12:37:21.422865+0000","last_fullsized":"2024-04-25T12:37:21.422865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.076775+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:37:31.076775+0000","last_peered":"2024-04-25T12:37:31.076775+0000","last_clean":"2024-04-25T12:37:31.076775+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:31.076775+0000","last_undegraded":"2024-04-25T12:37:31.076775+0000","last_fullsized":"2024-04-25T12:37:31.076775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911823+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:37:21.911823+0000","last_peered":"2024-04-25T12:37:21.911823+0000","last_clean":"2024-04-25T12:37:21.911823+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:21.911823+0000","last_undegraded":"2024-04-25T12:37:21.911823+0000","last_fullsized":"2024-04-25T12:37:21.911823+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:41.349167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910437+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:37:21.910437+0000","last_peered":"2024-04-25T12:37:21.910437+0000","last_clean":"2024-04-25T12:37:21.910437+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:21.910437+0000","last_undegraded":"2024-04-25T12:37:21.910437+0000","last_fullsized":"2024-04-25T12:37:21.910437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:19.646935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422738+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:37:21.422738+0000","last_peered":"2024-04-25T12:37:21.422738+0000","last_clean":"2024-04-25T12:37:21.422738+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:21.422738+0000","last_undegraded":"2024-04-25T12:37:21.422738+0000","last_fullsized":"2024-04-25T12:37:21.422738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"75'16","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568873+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:37:21.568873+0000","last_peered":"2024-04-25T12:37:21.568873+0000","last_clean":"2024-04-25T12:37:21.568873+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:21.568873+0000","last_undegraded":"2024-04-25T12:37:21.568873+0000","last_fullsized":"2024-04-25T12:37:21.568873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908280+0000","last_change":"2024-04-25T12:37:01.226590+0000","last_active":"2024-04-25T12:37:21.908280+0000","last_peered":"2024-04-25T12:37:21.908280+0000","last_clean":"2024-04-25T12:37:21.908280+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:21.908280+0000","last_undegraded":"2024-04-25T12:37:21.908280+0000","last_fullsized":"2024-04-25T12:37:21.908280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.581831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.053564+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:37:31.053564+0000","last_peered":"2024-04-25T12:37:31.053564+0000","last_clean":"2024-04-25T12:37:31.053564+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:31.053564+0000","last_undegraded":"2024-04-25T12:37:31.053564+0000","last_fullsized":"2024-04-25T12:37:31.053564+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:22.792696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910045+0000","last_change":"2024-04-25T12:34:12.917193+0000","last_active":"2024-04-25T12:37:21.910045+0000","last_peered":"2024-04-25T12:37:21.910045+0000","last_clean":"2024-04-25T12:37:21.910045+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:21.910045+0000","last_undegraded":"2024-04-25T12:37:21.910045+0000","last_fullsized":"2024-04-25T12:37:21.910045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:13.751222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908364+0000","last_change":"2024-04-25T12:37:01.220767+0000","last_active":"2024-04-25T12:37:21.908364+0000","last_peered":"2024-04-25T12:37:21.908364+0000","last_clean":"2024-04-25T12:37:21.908364+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:21.908364+0000","last_undegraded":"2024-04-25T12:37:21.908364+0000","last_fullsized":"2024-04-25T12:37:21.908364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:15.874258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422741+0000","last_change":"2024-04-25T12:37:01.233439+0000","last_active":"2024-04-25T12:37:21.422741+0000","last_peered":"2024-04-25T12:37:21.422741+0000","last_clean":"2024-04-25T12:37:21.422741+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:37:21.422741+0000","last_undegraded":"2024-04-25T12:37:21.422741+0000","last_fullsized":"2024-04-25T12:37:21.422741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:11.957744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424042+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:37:21.424042+0000","last_peered":"2024-04-25T12:37:21.424042+0000","last_clean":"2024-04-25T12:37:21.424042+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:21.424042+0000","last_undegraded":"2024-04-25T12:37:21.424042+0000","last_fullsized":"2024-04-25T12:37:21.424042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908903+0000","last_change":"2024-04-25T12:37:01.225914+0000","last_active":"2024-04-25T12:37:21.908903+0000","last_peered":"2024-04-25T12:37:21.908903+0000","last_clean":"2024-04-25T12:37:21.908903+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:21.908903+0000","last_undegraded":"2024-04-25T12:37:21.908903+0000","last_fullsized":"2024-04-25T12:37:21.908903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:34.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.044913+0000","last_change":"2024-04-25T12:37:01.229502+0000","last_active":"2024-04-25T12:37:31.044913+0000","last_peered":"2024-04-25T12:37:31.044913+0000","last_clean":"2024-04-25T12:37:31.044913+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:31.044913+0000","last_undegraded":"2024-04-25T12:37:31.044913+0000","last_fullsized":"2024-04-25T12:37:31.044913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:25.697423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403098+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:37:21.403098+0000","last_peered":"2024-04-25T12:37:21.403098+0000","last_clean":"2024-04-25T12:37:21.403098+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:21.403098+0000","last_undegraded":"2024-04-25T12:37:21.403098+0000","last_fullsized":"2024-04-25T12:37:21.403098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912070+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:37:21.912070+0000","last_peered":"2024-04-25T12:37:21.912070+0000","last_clean":"2024-04-25T12:37:21.912070+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:21.912070+0000","last_undegraded":"2024-04-25T12:37:21.912070+0000","last_fullsized":"2024-04-25T12:37:21.912070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:49.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.042486+0000","last_change":"2024-04-25T12:37:01.207603+0000","last_active":"2024-04-25T12:37:31.042486+0000","last_peered":"2024-04-25T12:37:31.042486+0000","last_clean":"2024-04-25T12:37:31.042486+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:31.042486+0000","last_undegraded":"2024-04-25T12:37:31.042486+0000","last_fullsized":"2024-04-25T12:37:31.042486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:01.832860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"93'29","reported_seq":184,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.015655+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:37:31.015655+0000","last_peered":"2024-04-25T12:37:31.015655+0000","last_clean":"2024-04-25T12:37:31.015655+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:31.015655+0000","last_undegraded":"2024-04-25T12:37:31.015655+0000","last_fullsized":"2024-04-25T12:37:31.015655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37279,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568703+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:37:21.568703+0000","last_peered":"2024-04-25T12:37:21.568703+0000","last_clean":"2024-04-25T12:37:21.568703+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:21.568703+0000","last_undegraded":"2024-04-25T12:37:21.568703+0000","last_fullsized":"2024-04-25T12:37:21.568703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423387+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:37:21.423387+0000","last_peered":"2024-04-25T12:37:21.423387+0000","last_clean":"2024-04-25T12:37:21.423387+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:21.423387+0000","last_undegraded":"2024-04-25T12:37:21.423387+0000","last_fullsized":"2024-04-25T12:37:21.423387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"75'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403382+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:37:21.403382+0000","last_peered":"2024-04-25T12:37:21.403382+0000","last_clean":"2024-04-25T12:37:21.403382+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:21.403382+0000","last_undegraded":"2024-04-25T12:37:21.403382+0000","last_fullsized":"2024-04-25T12:37:21.403382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405271+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:37:21.405271+0000","last_peered":"2024-04-25T12:37:21.405271+0000","last_clean":"2024-04-25T12:37:21.405271+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:21.405271+0000","last_undegraded":"2024-04-25T12:37:21.405271+0000","last_fullsized":"2024-04-25T12:37:21.405271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908126+0000","last_change":"2024-04-25T12:37:01.212753+0000","last_active":"2024-04-25T12:37:21.908126+0000","last_peered":"2024-04-25T12:37:21.908126+0000","last_clean":"2024-04-25T12:37:21.908126+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:21.908126+0000","last_undegraded":"2024-04-25T12:37:21.908126+0000","last_fullsized":"2024-04-25T12:37:21.908126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:07.000299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911540+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:37:21.911540+0000","last_peered":"2024-04-25T12:37:21.911540+0000","last_clean":"2024-04-25T12:37:21.911540+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:21.911540+0000","last_undegraded":"2024-04-25T12:37:21.911540+0000","last_fullsized":"2024-04-25T12:37:21.911540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402408+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:37:21.402408+0000","last_peered":"2024-04-25T12:37:21.402408+0000","last_clean":"2024-04-25T12:37:21.402408+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:21.402408+0000","last_undegraded":"2024-04-25T12:37:21.402408+0000","last_fullsized":"2024-04-25T12:37:21.402408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.087322+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:37:31.087322+0000","last_peered":"2024-04-25T12:37:31.087322+0000","last_clean":"2024-04-25T12:37:31.087322+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:31.087322+0000","last_undegraded":"2024-04-25T12:37:31.087322+0000","last_fullsized":"2024-04-25T12:37:31.087322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913386+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:37:21.913386+0000","last_peered":"2024-04-25T12:37:21.913386+0000","last_clean":"2024-04-25T12:37:21.913386+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:21.913386+0000","last_undegraded":"2024-04-25T12:37:21.913386+0000","last_fullsized":"2024-04-25T12:37:21.913386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568895+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:37:21.568895+0000","last_peered":"2024-04-25T12:37:21.568895+0000","last_clean":"2024-04-25T12:37:21.568895+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:21.568895+0000","last_undegraded":"2024-04-25T12:37:21.568895+0000","last_fullsized":"2024-04-25T12:37:21.568895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568918+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:37:21.568918+0000","last_peered":"2024-04-25T12:37:21.568918+0000","last_clean":"2024-04-25T12:37:21.568918+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:21.568918+0000","last_undegraded":"2024-04-25T12:37:21.568918+0000","last_fullsized":"2024-04-25T12:37:21.568918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"93'10","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.044218+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:37:31.044218+0000","last_peered":"2024-04-25T12:37:31.044218+0000","last_clean":"2024-04-25T12:37:31.044218+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:31.044218+0000","last_undegraded":"2024-04-25T12:37:31.044218+0000","last_fullsized":"2024-04-25T12:37:31.044218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423322+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:37:21.423322+0000","last_peered":"2024-04-25T12:37:21.423322+0000","last_clean":"2024-04-25T12:37:21.423322+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:21.423322+0000","last_undegraded":"2024-04-25T12:37:21.423322+0000","last_fullsized":"2024-04-25T12:37:21.423322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424396+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:37:21.424396+0000","last_peered":"2024-04-25T12:37:21.424396+0000","last_clean":"2024-04-25T12:37:21.424396+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:21.424396+0000","last_undegraded":"2024-04-25T12:37:21.424396+0000","last_fullsized":"2024-04-25T12:37:21.424396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907318+0000","last_change":"2024-04-25T12:37:01.204788+0000","last_active":"2024-04-25T12:37:21.907318+0000","last_peered":"2024-04-25T12:37:21.907318+0000","last_clean":"2024-04-25T12:37:21.907318+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:21.907318+0000","last_undegraded":"2024-04-25T12:37:21.907318+0000","last_fullsized":"2024-04-25T12:37:21.907318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:14.535103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908141+0000","last_change":"2024-04-25T12:37:01.217177+0000","last_active":"2024-04-25T12:37:21.908141+0000","last_peered":"2024-04-25T12:37:21.908141+0000","last_clean":"2024-04-25T12:37:21.908141+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:21.908141+0000","last_undegraded":"2024-04-25T12:37:21.908141+0000","last_fullsized":"2024-04-25T12:37:21.908141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:00.819525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423908+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:37:21.423908+0000","last_peered":"2024-04-25T12:37:21.423908+0000","last_clean":"2024-04-25T12:37:21.423908+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:21.423908+0000","last_undegraded":"2024-04-25T12:37:21.423908+0000","last_fullsized":"2024-04-25T12:37:21.423908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.045592+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:37:31.045592+0000","last_peered":"2024-04-25T12:37:31.045592+0000","last_clean":"2024-04-25T12:37:31.045592+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:31.045592+0000","last_undegraded":"2024-04-25T12:37:31.045592+0000","last_fullsized":"2024-04-25T12:37:31.045592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569215+0000","last_change":"2024-04-25T12:37:01.209143+0000","last_active":"2024-04-25T12:37:21.569215+0000","last_peered":"2024-04-25T12:37:21.569215+0000","last_clean":"2024-04-25T12:37:21.569215+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:21.569215+0000","last_undegraded":"2024-04-25T12:37:21.569215+0000","last_fullsized":"2024-04-25T12:37:21.569215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:52.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907606+0000","last_change":"2024-04-25T12:37:01.223997+0000","last_active":"2024-04-25T12:37:21.907606+0000","last_peered":"2024-04-25T12:37:21.907606+0000","last_clean":"2024-04-25T12:37:21.907606+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:21.907606+0000","last_undegraded":"2024-04-25T12:37:21.907606+0000","last_fullsized":"2024-04-25T12:37:21.907606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:48.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.064092+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:37:31.064092+0000","last_peered":"2024-04-25T12:37:31.064092+0000","last_clean":"2024-04-25T12:37:31.064092+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:31.064092+0000","last_undegraded":"2024-04-25T12:37:31.064092+0000","last_fullsized":"2024-04-25T12:37:31.064092+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403285+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:37:21.403285+0000","last_peered":"2024-04-25T12:37:21.403285+0000","last_clean":"2024-04-25T12:37:21.403285+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:21.403285+0000","last_undegraded":"2024-04-25T12:37:21.403285+0000","last_fullsized":"2024-04-25T12:37:21.403285+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.057004+0000","last_change":"2024-04-25T12:37:01.224898+0000","last_active":"2024-04-25T12:37:31.057004+0000","last_peered":"2024-04-25T12:37:31.057004+0000","last_clean":"2024-04-25T12:37:31.057004+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:31.057004+0000","last_undegraded":"2024-04-25T12:37:31.057004+0000","last_fullsized":"2024-04-25T12:37:31.057004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:18.991070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.063908+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:37:31.063908+0000","last_peered":"2024-04-25T12:37:31.063908+0000","last_clean":"2024-04-25T12:37:31.063908+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:31.063908+0000","last_undegraded":"2024-04-25T12:37:31.063908+0000","last_fullsized":"2024-04-25T12:37:31.063908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422760+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:37:21.422760+0000","last_peered":"2024-04-25T12:37:21.422760+0000","last_clean":"2024-04-25T12:37:21.422760+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:21.422760+0000","last_undegraded":"2024-04-25T12:37:21.422760+0000","last_fullsized":"2024-04-25T12:37:21.422760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907860+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:37:21.907860+0000","last_peered":"2024-04-25T12:37:21.907860+0000","last_clean":"2024-04-25T12:37:21.907860+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:21.907860+0000","last_undegraded":"2024-04-25T12:37:21.907860+0000","last_fullsized":"2024-04-25T12:37:21.907860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.049799+0000","last_change":"2024-04-25T12:37:01.228534+0000","last_active":"2024-04-25T12:37:31.049799+0000","last_peered":"2024-04-25T12:37:31.049799+0000","last_clean":"2024-04-25T12:37:31.049799+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:31.049799+0000","last_undegraded":"2024-04-25T12:37:31.049799+0000","last_fullsized":"2024-04-25T12:37:31.049799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:39.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.041140+0000","last_change":"2024-04-25T12:37:01.210033+0000","last_active":"2024-04-25T12:37:31.041140+0000","last_peered":"2024-04-25T12:37:31.041140+0000","last_clean":"2024-04-25T12:37:31.041140+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:31.041140+0000","last_undegraded":"2024-04-25T12:37:31.041140+0000","last_fullsized":"2024-04-25T12:37:31.041140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:50.639901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.081340+0000","last_change":"2024-04-25T12:37:01.232027+0000","last_active":"2024-04-25T12:37:31.081340+0000","last_peered":"2024-04-25T12:37:31.081340+0000","last_clean":"2024-04-25T12:37:31.081340+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:31.081340+0000","last_undegraded":"2024-04-25T12:37:31.081340+0000","last_fullsized":"2024-04-25T12:37:31.081340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908213+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:37:21.908213+0000","last_peered":"2024-04-25T12:37:21.908213+0000","last_clean":"2024-04-25T12:37:21.908213+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:21.908213+0000","last_undegraded":"2024-04-25T12:37:21.908213+0000","last_fullsized":"2024-04-25T12:37:21.908213+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423227+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:37:21.423227+0000","last_peered":"2024-04-25T12:37:21.423227+0000","last_clean":"2024-04-25T12:37:21.423227+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:21.423227+0000","last_undegraded":"2024-04-25T12:37:21.423227+0000","last_fullsized":"2024-04-25T12:37:21.423227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907222+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:37:21.907222+0000","last_peered":"2024-04-25T12:37:21.907222+0000","last_clean":"2024-04-25T12:37:21.907222+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:21.907222+0000","last_undegraded":"2024-04-25T12:37:21.907222+0000","last_fullsized":"2024-04-25T12:37:21.907222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.077029+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:37:31.077029+0000","last_peered":"2024-04-25T12:37:31.077029+0000","last_clean":"2024-04-25T12:37:31.077029+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:31.077029+0000","last_undegraded":"2024-04-25T12:37:31.077029+0000","last_fullsized":"2024-04-25T12:37:31.077029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.055206+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:37:31.055206+0000","last_peered":"2024-04-25T12:37:31.055206+0000","last_clean":"2024-04-25T12:37:31.055206+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:31.055206+0000","last_undegraded":"2024-04-25T12:37:31.055206+0000","last_fullsized":"2024-04-25T12:37:31.055206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910725+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:37:21.910725+0000","last_peered":"2024-04-25T12:37:21.910725+0000","last_clean":"2024-04-25T12:37:21.910725+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:21.910725+0000","last_undegraded":"2024-04-25T12:37:21.910725+0000","last_fullsized":"2024-04-25T12:37:21.910725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424005+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:37:21.424005+0000","last_peered":"2024-04-25T12:37:21.424005+0000","last_clean":"2024-04-25T12:37:21.424005+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:21.424005+0000","last_undegraded":"2024-04-25T12:37:21.424005+0000","last_fullsized":"2024-04-25T12:37:21.424005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.060716+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:37:31.060716+0000","last_peered":"2024-04-25T12:37:31.060716+0000","last_clean":"2024-04-25T12:37:31.060716+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:31.060716+0000","last_undegraded":"2024-04-25T12:37:31.060716+0000","last_fullsized":"2024-04-25T12:37:31.060716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424575+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:37:21.424575+0000","last_peered":"2024-04-25T12:37:21.424575+0000","last_clean":"2024-04-25T12:37:21.424575+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:21.424575+0000","last_undegraded":"2024-04-25T12:37:21.424575+0000","last_fullsized":"2024-04-25T12:37:21.424575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423084+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:37:21.423084+0000","last_peered":"2024-04-25T12:37:21.423084+0000","last_clean":"2024-04-25T12:37:21.423084+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:21.423084+0000","last_undegraded":"2024-04-25T12:37:21.423084+0000","last_fullsized":"2024-04-25T12:37:21.423084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"93'8","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.041792+0000","last_change":"2024-04-25T12:37:01.212249+0000","last_active":"2024-04-25T12:37:31.041792+0000","last_peered":"2024-04-25T12:37:31.041792+0000","last_clean":"2024-04-25T12:37:31.041792+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:31.041792+0000","last_undegraded":"2024-04-25T12:37:31.041792+0000","last_fullsized":"2024-04-25T12:37:31.041792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:47.658434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.046222+0000","last_change":"2024-04-25T12:37:01.216745+0000","last_active":"2024-04-25T12:37:31.046222+0000","last_peered":"2024-04-25T12:37:31.046222+0000","last_clean":"2024-04-25T12:37:31.046222+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:31.046222+0000","last_undegraded":"2024-04-25T12:37:31.046222+0000","last_fullsized":"2024-04-25T12:37:31.046222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910517+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:37:21.910517+0000","last_peered":"2024-04-25T12:37:21.910517+0000","last_clean":"2024-04-25T12:37:21.910517+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:21.910517+0000","last_undegraded":"2024-04-25T12:37:21.910517+0000","last_fullsized":"2024-04-25T12:37:21.910517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.657939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910107+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:37:21.910107+0000","last_peered":"2024-04-25T12:37:21.910107+0000","last_clean":"2024-04-25T12:37:21.910107+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:21.910107+0000","last_undegraded":"2024-04-25T12:37:21.910107+0000","last_fullsized":"2024-04-25T12:37:21.910107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:51.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911235+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:37:21.911235+0000","last_peered":"2024-04-25T12:37:21.911235+0000","last_clean":"2024-04-25T12:37:21.911235+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:21.911235+0000","last_undegraded":"2024-04-25T12:37:21.911235+0000","last_fullsized":"2024-04-25T12:37:21.911235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:41.175417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.068114+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:37:31.068114+0000","last_peered":"2024-04-25T12:37:31.068114+0000","last_clean":"2024-04-25T12:37:31.068114+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:31.068114+0000","last_undegraded":"2024-04-25T12:37:31.068114+0000","last_fullsized":"2024-04-25T12:37:31.068114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907340+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:37:21.907340+0000","last_peered":"2024-04-25T12:37:21.907340+0000","last_clean":"2024-04-25T12:37:21.907340+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:21.907340+0000","last_undegraded":"2024-04-25T12:37:21.907340+0000","last_fullsized":"2024-04-25T12:37:21.907340+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909742+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:37:21.909742+0000","last_peered":"2024-04-25T12:37:21.909742+0000","last_clean":"2024-04-25T12:37:21.909742+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:21.909742+0000","last_undegraded":"2024-04-25T12:37:21.909742+0000","last_fullsized":"2024-04-25T12:37:21.909742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:20.461543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052307+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:37:31.052307+0000","last_peered":"2024-04-25T12:37:31.052307+0000","last_clean":"2024-04-25T12:37:31.052307+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:31.052307+0000","last_undegraded":"2024-04-25T12:37:31.052307+0000","last_fullsized":"2024-04-25T12:37:31.052307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911087+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:37:21.911087+0000","last_peered":"2024-04-25T12:37:21.911087+0000","last_clean":"2024-04-25T12:37:21.911087+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:21.911087+0000","last_undegraded":"2024-04-25T12:37:21.911087+0000","last_fullsized":"2024-04-25T12:37:21.911087+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:37.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.054794+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:37:31.054794+0000","last_peered":"2024-04-25T12:37:31.054794+0000","last_clean":"2024-04-25T12:37:31.054794+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:31.054794+0000","last_undegraded":"2024-04-25T12:37:31.054794+0000","last_fullsized":"2024-04-25T12:37:31.054794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907971+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:37:21.907971+0000","last_peered":"2024-04-25T12:37:21.907971+0000","last_clean":"2024-04-25T12:37:21.907971+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:21.907971+0000","last_undegraded":"2024-04-25T12:37:21.907971+0000","last_fullsized":"2024-04-25T12:37:21.907971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908194+0000","last_change":"2024-04-25T12:37:01.215943+0000","last_active":"2024-04-25T12:37:21.908194+0000","last_peered":"2024-04-25T12:37:21.908194+0000","last_clean":"2024-04-25T12:37:21.908194+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:21.908194+0000","last_undegraded":"2024-04-25T12:37:21.908194+0000","last_fullsized":"2024-04-25T12:37:21.908194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.518673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911529+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:37:21.911529+0000","last_peered":"2024-04-25T12:37:21.911529+0000","last_clean":"2024-04-25T12:37:21.911529+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:21.911529+0000","last_undegraded":"2024-04-25T12:37:21.911529+0000","last_fullsized":"2024-04-25T12:37:21.911529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:29.018280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41301,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":171,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82602,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462932,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29336,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342504,"statfs":{"total":96636764160,"available":96606724096,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1216512,"data_stored":1207327,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1142784,"data_stored":616256,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37895,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38847,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:37:33.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:37:33.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.340+0000 7f0f50a00640 1 Processor -- start 2024-04-25T12:37:33.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.341+0000 7f0f50a00640 1 -- start start 2024-04-25T12:37:33.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.342+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c15ac10 0x7f0f4c161000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:33.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.342+0000 7f0f50a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f4c164390 con 0x7f0f4c163ec0 2024-04-25T12:37:33.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.342+0000 7f0f50a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f4c164630 con 0x7f0f4c161540 2024-04-25T12:37:33.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.342+0000 7f0f50a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c164e70 con 0x7f0f4c15ac10 2024-04-25T12:37:33.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f4a800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0f4c163ec0 0x7f0f4c0c8520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60874/0 (socket says 172.21.15.123:60874) 2024-04-25T12:37:33.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f4a800640 1 -- 172.21.15.123:0/3925888225 learned_addr learned my addr 172.21.15.123:0/3925888225 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:33.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49e00640 1 --2- 172.21.15.123:0/3925888225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c15ac10 0x7f0f4c161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:33.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49400640 1 -- 172.21.15.123:0/3925888225 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2229731067 0 0) 0x7f0f4c164390 con 0x7f0f4c163ec0 2024-04-25T12:37:33.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49400640 1 -- 172.21.15.123:0/3925888225 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f30003520 con 0x7f0f4c163ec0 2024-04-25T12:37:33.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49400640 1 -- 172.21.15.123:0/3925888225 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 473132606 0 0) 0x7f0f4c164630 con 0x7f0f4c161540 2024-04-25T12:37:33.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49400640 1 -- 172.21.15.123:0/3925888225 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f4c164390 con 0x7f0f4c161540 2024-04-25T12:37:33.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49e00640 1 -- 172.21.15.123:0/3925888225 >> v1:172.21.15.153:6789/0 conn(0x7f0f4c161540 legacy=0x7f0f4c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:33.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49e00640 1 -- 172.21.15.123:0/3925888225 >> v1:172.21.15.123:6789/0 conn(0x7f0f4c163ec0 legacy=0x7f0f4c0c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:33.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.343+0000 7f0f49e00640 1 -- 172.21.15.123:0/3925888225 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c0c8c30 con 0x7f0f4c15ac10 2024-04-25T12:37:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.344+0000 7f0f49e00640 1 --2- 172.21.15.123:0/3925888225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c15ac10 0x7f0f4c161000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f380097d0 tx=0x7f0f3802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ea501392b8dd15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.345+0000 7f0f49400640 1 -- 172.21.15.123:0/3925888225 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38037070 con 0x7f0f4c15ac10 2024-04-25T12:37:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.345+0000 7f0f49400640 1 -- 172.21.15.123:0/3925888225 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f38032440 con 0x7f0f4c15ac10 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.345+0000 7f0f49400640 1 -- 172.21.15.123:0/3925888225 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38032810 con 0x7f0f4c15ac10 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.345+0000 7f0f50a00640 1 -- 172.21.15.123:0/3925888225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c15ac10 msgr2=0x7f0f4c161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.345+0000 7f0f50a00640 1 --2- 172.21.15.123:0/3925888225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c15ac10 0x7f0f4c161000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f380097d0 tx=0x7f0f3802a210 comp rx=0 tx=0).stop 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.346+0000 7f0f50a00640 1 -- 172.21.15.123:0/3925888225 shutdown_connections 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.346+0000 7f0f50a00640 1 --2- 172.21.15.123:0/3925888225 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c15ac10 0x7f0f4c161000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.346+0000 7f0f50a00640 1 -- 172.21.15.123:0/3925888225 >> 172.21.15.123:0/3925888225 conn(0x7f0f4c0cd600 msgr2=0x7f0f4c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.346+0000 7f0f50a00640 1 -- 172.21.15.123:0/3925888225 shutdown_connections 2024-04-25T12:37:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.346+0000 7f0f50a00640 1 -- 172.21.15.123:0/3925888225 wait complete. 2024-04-25T12:37:33.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.348+0000 7f0f50a00640 1 Processor -- start 2024-04-25T12:37:33.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.348+0000 7f0f50a00640 1 -- start start 2024-04-25T12:37:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.349+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f4c15ac10 0x7f0f4c1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c161540 0x7f0f4c1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f4c163ec0 0x7f0f4c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f50a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c160b70 con 0x7f0f4c15ac10 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f50a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c15b020 con 0x7f0f4c163ec0 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f50a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c164b90 con 0x7f0f4c161540 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f4c15ac10 0x7f0f4c1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f4c15ac10 0x7f0f4c1567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59944/0 (socket says 172.21.15.123:59944) 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 -- 172.21.15.123:0/1806363521 learned_addr learned my addr 172.21.15.123:0/1806363521 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f49e00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c161540 0x7f0f4c1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4b200640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f4c163ec0 0x7f0f4c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 -- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c161540 msgr2=0x7f0f4c1f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c161540 0x7f0f4c1f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 -- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f4c163ec0 msgr2=0x7f0f4c15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f4c163ec0 0x7f0f4c15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f4a800640 1 -- 172.21.15.123:0/1806363521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1f7800 con 0x7f0f4c15ac10 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.350+0000 7f0f49e00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c161540 0x7f0f4c1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.351+0000 7f0f4a800640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f4c15ac10 0x7f0f4c1567c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c440 tx=0x7f0f3c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.351+0000 7f0f42a00640 1 -- 172.21.15.123:0/1806363521 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c017070 con 0x7f0f4c15ac10 2024-04-25T12:37:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.351+0000 7f0f42a00640 1 -- 172.21.15.123:0/1806363521 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c005ce0 con 0x7f0f4c15ac10 2024-04-25T12:37:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.351+0000 7f0f42a00640 1 -- 172.21.15.123:0/1806363521 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c007270 con 0x7f0f4c15ac10 2024-04-25T12:37:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.351+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c156550 con 0x7f0f4c15ac10 2024-04-25T12:37:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.351+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c155c90 con 0x7f0f4c15ac10 2024-04-25T12:37:33.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.352+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4c076da0 con 0x7f0f4c15ac10 2024-04-25T12:37:33.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.354+0000 7f0f42a00640 1 -- 172.21.15.123:0/1806363521 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f3c01a070 con 0x7f0f4c15ac10 2024-04-25T12:37:33.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.355+0000 7f0f42a00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f2c0744e0 0x7f0f2c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:33.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.355+0000 7f0f42a00640 1 -- 172.21.15.123:0/1806363521 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f0f3c094880 con 0x7f0f4c15ac10 2024-04-25T12:37:33.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.355+0000 7f0f49e00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f2c0744e0 0x7f0f2c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:33.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.356+0000 7f0f49e00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f2c0744e0 0x7f0f2c0769f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0f38034000 tx=0x7f0f3802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:33.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.359+0000 7f0f42a00640 1 -- 172.21.15.123:0/1806363521 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f3c05e5a0 con 0x7f0f4c15ac10 2024-04-25T12:37:33.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.563+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f4c154d20 con 0x7f0f4c15ac10 2024-04-25T12:37:33.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.566+0000 7f0f42a00640 1 -- 172.21.15.123:0/1806363521 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+19058 (secure 0 0 0) 0x7f0f3c0628a0 con 0x7f0f4c15ac10 2024-04-25T12:37:33.566 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:33.566 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":93,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:37:21.396436+0000","last_up_change":"2024-04-25T12:35:59.575545+0000","last_in_change":"2024-04-25T12:37:00.163243+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":39,"up_from":47,"up_thru":76,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6827","nonce":4270711740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6829","nonce":4270711740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6833","nonce":4270711740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6831","nonce":4270711740}]},"public_addr":"172.21.15.153:6827/4270711740","cluster_addr":"172.21.15.153:6829/4270711740","heartbeat_back_addr":"172.21.15.153:6833/4270711740","heartbeat_front_addr":"172.21.15.153:6831/4270711740","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":40},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[3,1]},{"pgid":"4.15","osds":[1,0,7,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":5,"to":4}]},{"pgid":"2.c","mappings":[{"from":0,"to":2}]},{"pgid":"3.3","mappings":[{"from":0,"to":7}]},{"pgid":"3.c","mappings":[{"from":0,"to":1}]},{"pgid":"3.13","mappings":[{"from":0,"to":1}]},{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"3.17","mappings":[{"from":3,"to":7}]},{"pgid":"3.26","mappings":[{"from":3,"to":1}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f2c0744e0 msgr2=0x7f0f2c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f2c0744e0 0x7f0f2c0769f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0f38034000 tx=0x7f0f3802e000 comp rx=0 tx=0).stop 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f4c15ac10 msgr2=0x7f0f4c1567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f4c15ac10 0x7f0f4c1567c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c440 tx=0x7f0f3c00c900 comp rx=0 tx=0).stop 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 shutdown_connections 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f2c0744e0 0x7f0f2c0769f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f4c163ec0 0x7f0f4c15f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:33.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f4c161540 0x7f0f4c1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:33.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 --2- 172.21.15.123:0/1806363521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f4c15ac10 0x7f0f4c1567c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:33.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 >> 172.21.15.123:0/1806363521 conn(0x7f0f4c0cd600 msgr2=0x7f0f4c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:33.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 shutdown_connections 2024-04-25T12:37:33.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:33.569+0000 7f0f50a00640 1 -- 172.21.15.123:0/1806363521 wait complete. 2024-04-25T12:37:34.595 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T12:37:34.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:34.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.886+0000 7fd78f200640 1 Processor -- start 2024-04-25T12:37:34.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.886+0000 7fd78f200640 1 -- start start 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd78f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd788070250 0x7fd7881578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd78f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd78815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd78f200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd788074600 con 0x7fd7880c4cc0 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd78f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd7880c7270 con 0x7fd788070250 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd78f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd7880c9fb0 con 0x7fd78806f7c0 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd787e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd788070250 0x7fd7881578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd787e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd788070250 0x7fd7881578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59960/0 (socket says 172.21.15.123:59960) 2024-04-25T12:37:34.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd787e00640 1 -- 172.21.15.123:0/280920128 learned_addr learned my addr 172.21.15.123:0/280920128 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:34.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.888+0000 7fd78d200640 1 --2- 172.21.15.123:0/280920128 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd78815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:34.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.889+0000 7fd787e00640 1 -- 172.21.15.123:0/280920128 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 msgr2=0x7fd78815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:34.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.889+0000 7fd787e00640 1 --2- 172.21.15.123:0/280920128 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd78815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:34.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.889+0000 7fd787e00640 1 -- 172.21.15.123:0/280920128 >> v1:172.21.15.153:6789/0 conn(0x7fd7880c4cc0 legacy=0x7fd7880c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:34.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.889+0000 7fd787e00640 1 -- 172.21.15.123:0/280920128 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd788157e00 con 0x7fd788070250 2024-04-25T12:37:34.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.889+0000 7fd78d200640 1 --2- 172.21.15.123:0/280920128 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd78815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:34.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.889+0000 7fd787400640 1 -- 172.21.15.123:0/280920128 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185669054 0 0) 0x7fd788074600 con 0x7fd7880c4cc0 2024-04-25T12:37:34.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.890+0000 7fd787e00640 1 --2- 172.21.15.123:0/280920128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd788070250 0x7fd7881578c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd778009e50 tx=0x7fd77802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab8df3e93ae4319 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:34.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.890+0000 7fd787400640 1 -- 172.21.15.123:0/280920128 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778036070 con 0x7fd788070250 2024-04-25T12:37:34.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.890+0000 7fd787400640 1 -- 172.21.15.123:0/280920128 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd778031600 con 0x7fd788070250 2024-04-25T12:37:34.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.890+0000 7fd787400640 1 -- 172.21.15.123:0/280920128 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7780319d0 con 0x7fd788070250 2024-04-25T12:37:34.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 -- 172.21.15.123:0/280920128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd788070250 msgr2=0x7fd7881578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:34.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 --2- 172.21.15.123:0/280920128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd788070250 0x7fd7881578c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd778009e50 tx=0x7fd77802a7a0 comp rx=0 tx=0).stop 2024-04-25T12:37:34.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 -- 172.21.15.123:0/280920128 shutdown_connections 2024-04-25T12:37:34.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 --2- 172.21.15.123:0/280920128 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd78815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:34.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 --2- 172.21.15.123:0/280920128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd788070250 0x7fd7881578c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:34.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 -- 172.21.15.123:0/280920128 >> 172.21.15.123:0/280920128 conn(0x7fd78806c570 msgr2=0x7fd78806c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:34.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 -- 172.21.15.123:0/280920128 shutdown_connections 2024-04-25T12:37:34.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.891+0000 7fd78f200640 1 -- 172.21.15.123:0/280920128 wait complete. 2024-04-25T12:37:34.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.893+0000 7fd78f200640 1 Processor -- start 2024-04-25T12:37:34.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.894+0000 7fd78f200640 1 -- start start 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd78f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd7881f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd78f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd788070250 0x7fd7881f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd78f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7880c4cc0 0x7fd788159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd78f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7880c9960 con 0x7fd7880c4cc0 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd78f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd788074b50 con 0x7fd788070250 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd78f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd7880c9c40 con 0x7fd78806f7c0 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd787e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd7881f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd787e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd7881f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:44852/0 (socket says 172.21.15.123:44852) 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd787e00640 1 -- 172.21.15.123:0/2207746497 learned_addr learned my addr 172.21.15.123:0/2207746497 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd78d200640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7880c4cc0 0x7fd788159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.895+0000 7fd77f400640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd788070250 0x7fd7881f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:34.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd787e00640 1 -- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd788070250 msgr2=0x7fd7881f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:34.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd787e00640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd788070250 0x7fd7881f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:34.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd787e00640 1 -- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7880c4cc0 msgr2=0x7fd788159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:34.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd787e00640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7880c4cc0 0x7fd788159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:34.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd787e00640 1 -- 172.21.15.123:0/2207746497 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7881f70b0 con 0x7fd78806f7c0 2024-04-25T12:37:34.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd787e00640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd7881f6b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd778009e50 tx=0x7fd778004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:34.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778041020 con 0x7fd78806f7c0 2024-04-25T12:37:34.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7780077a0 con 0x7fd78806f7c0 2024-04-25T12:37:34.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778036040 con 0x7fd78806f7c0 2024-04-25T12:37:34.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.896+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7880c2980 con 0x7fd78806f7c0 2024-04-25T12:37:34.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.897+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7880c2f20 con 0x7fd78806f7c0 2024-04-25T12:37:34.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.898+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7881f2f40 con 0x7fd78806f7c0 2024-04-25T12:37:34.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.899+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd778044020 con 0x7fd78806f7c0 2024-04-25T12:37:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.900+0000 7fd784c00640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd7640744b0 0x7fd7640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.900+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fd7780b6270 con 0x7fd78806f7c0 2024-04-25T12:37:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.900+0000 7fd77f400640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd7640744b0 0x7fd7640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.900+0000 7fd784c00640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd76407e190 0x7fd764080680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.900+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd764080d50 con 0x7fd76407e190 2024-04-25T12:37:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.900+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fd7780bd050 con 0x7fd78806f7c0 2024-04-25T12:37:34.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.900+0000 7fd78d200640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd76407e190 0x7fd764080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:34.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.901+0000 7fd77f400640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd7640744b0 0x7fd7640769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd770000f60 tx=0x7fd770009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:34.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.901+0000 7fd78d200640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd76407e190 0x7fd764080680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:34.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.902+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== osd.0 v2:172.21.15.123:6816/1405705428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd764080d50 con 0x7fd76407e190 2024-04-25T12:37:34.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.916+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 --> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd788076700 con 0x7fd76407e190 2024-04-25T12:37:34.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd784c00640 1 -- 172.21.15.123:0/2207746497 <== osd.0 v2:172.21.15.123:6816/1405705428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd788076700 con 0x7fd76407e190 2024-04-25T12:37:34.916 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:34.916 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:34.916 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:34.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd76407e190 msgr2=0x7fd764080680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd78f200640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.123:6816/1405705428,v1:172.21.15.123:6817/1405705428] conn(0x7fd76407e190 0x7fd764080680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd7640744b0 msgr2=0x7fd7640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd78f200640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd7640744b0 0x7fd7640769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd770000f60 tx=0x7fd770009040 comp rx=0 tx=0).stop 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 msgr2=0x7fd7881f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd78f200640 1 --2- 172.21.15.123:0/2207746497 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd78806f7c0 0x7fd7881f6b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd778009e50 tx=0x7fd778004a40 comp rx=0 tx=0).stop 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.917+0000 7fd78d200640 1 -- 172.21.15.123:0/2207746497 reap_dead start 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.918+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 shutdown_connections 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.918+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 >> 172.21.15.123:0/2207746497 conn(0x7fd78806c570 msgr2=0x7fd7880c8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.918+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 shutdown_connections 2024-04-25T12:37:34.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:34.918+0000 7fd78f200640 1 -- 172.21.15.123:0/2207746497 wait complete. 2024-04-25T12:37:34.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T12:37:35.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.219+0000 7f56c8a00640 1 Processor -- start 2024-04-25T12:37:35.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.220+0000 7f56c8a00640 1 -- start start 2024-04-25T12:37:35.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c8a00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f56c40c4d50 0x7f56c4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c8a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f56c415afa0 0x7f56c40cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c8a00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c415b3b0 con 0x7f56c4074870 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c8a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f56c40c61f0 con 0x7f56c415afa0 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c8a00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f56c40c87d0 con 0x7f56c40c4d50 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c1e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f56c40c4d50 0x7f56c4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56bbe00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f56c4074870 0x7f56c40c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34760/0 (socket says 172.21.15.123:34760) 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c1e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f56c40c4d50 0x7f56c4154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41664/0 (socket says 172.21.15.123:41664) 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56bbe00640 1 -- 172.21.15.123:0/2355075040 learned_addr learned my addr 172.21.15.123:0/2355075040 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.222+0000 7f56c2800640 1 --2- 172.21.15.123:0/2355075040 >> v2:172.21.15.123:3300/0 conn(0x7f56c415afa0 0x7f56c40cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.223+0000 7f56c1e00640 1 -- 172.21.15.123:0/2355075040 >> v2:172.21.15.123:3300/0 conn(0x7f56c415afa0 msgr2=0x7f56c40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:35.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.223+0000 7f56c1e00640 1 --2- 172.21.15.123:0/2355075040 >> v2:172.21.15.123:3300/0 conn(0x7f56c415afa0 0x7f56c40cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.223+0000 7f56c1e00640 1 -- 172.21.15.123:0/2355075040 >> v1:172.21.15.153:6789/0 conn(0x7f56c4074870 legacy=0x7f56c40c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.223+0000 7f56c1e00640 1 -- 172.21.15.123:0/2355075040 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c4073ce0 con 0x7f56c40c4d50 2024-04-25T12:37:35.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.223+0000 7f56c1400640 1 -- 172.21.15.123:0/2355075040 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405772793 0 0) 0x7f56c415b3b0 con 0x7f56c4074870 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.224+0000 7f56c1e00640 1 --2- 172.21.15.123:0/2355075040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c40c4d50 0x7f56c4154890 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f56c40ce6d0 tx=0x7f56b402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906f723b47add766 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.224+0000 7f56c1400640 1 -- 172.21.15.123:0/2355075040 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4037070 con 0x7f56c40c4d50 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.224+0000 7f56c1400640 1 -- 172.21.15.123:0/2355075040 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b4032440 con 0x7f56c40c4d50 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.224+0000 7f56c1400640 1 -- 172.21.15.123:0/2355075040 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4032810 con 0x7f56c40c4d50 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 -- 172.21.15.123:0/2355075040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c40c4d50 msgr2=0x7f56c4154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 --2- 172.21.15.123:0/2355075040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c40c4d50 0x7f56c4154890 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f56c40ce6d0 tx=0x7f56b402a230 comp rx=0 tx=0).stop 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 -- 172.21.15.123:0/2355075040 shutdown_connections 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 --2- 172.21.15.123:0/2355075040 >> v2:172.21.15.123:3300/0 conn(0x7f56c415afa0 0x7f56c40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 --2- 172.21.15.123:0/2355075040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c40c4d50 0x7f56c4154890 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 -- 172.21.15.123:0/2355075040 >> 172.21.15.123:0/2355075040 conn(0x7f56c406c750 msgr2=0x7f56c406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:35.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 -- 172.21.15.123:0/2355075040 shutdown_connections 2024-04-25T12:37:35.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.225+0000 7f56c8a00640 1 -- 172.21.15.123:0/2355075040 wait complete. 2024-04-25T12:37:35.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.228+0000 7f56c8a00640 1 Processor -- start 2024-04-25T12:37:35.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.228+0000 7f56c8a00640 1 -- start start 2024-04-25T12:37:35.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c8a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56c4074870 0x7f56c4158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c4d50 0x7f56c414d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c415afa0 0x7f56c41574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40c8460 con 0x7f56c40c4d50 2024-04-25T12:37:35.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c8a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40742a0 con 0x7f56c415afa0 2024-04-25T12:37:35.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c8a00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f56c40c9370 con 0x7f56c4074870 2024-04-25T12:37:35.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c4d50 0x7f56c414d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c2800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c415afa0 0x7f56c41574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c4d50 0x7f56c414d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59978/0 (socket says 172.21.15.123:59978) 2024-04-25T12:37:35.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56bbe00640 1 -- 172.21.15.123:0/650678131 learned_addr learned my addr 172.21.15.123:0/650678131 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:35.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.230+0000 7f56c1e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56c4074870 0x7f56c4158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56c2800640 1 -- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56c4074870 msgr2=0x7f56c4158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56c2800640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56c4074870 0x7f56c4158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56c2800640 1 -- 172.21.15.123:0/650678131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c4d50 msgr2=0x7f56c414d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56c2800640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c4d50 0x7f56c414d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56c2800640 1 -- 172.21.15.123:0/650678131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c4158ab0 con 0x7f56c415afa0 2024-04-25T12:37:35.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56c2800640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c415afa0 0x7f56c41574d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00eb70 tx=0x7f56bc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:35.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56ba000640 1 -- 172.21.15.123:0/650678131 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc018020 con 0x7f56c415afa0 2024-04-25T12:37:35.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56ba000640 1 -- 172.21.15.123:0/650678131 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc00cd20 con 0x7f56c415afa0 2024-04-25T12:37:35.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56ba000640 1 -- 172.21.15.123:0/650678131 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0146c0 con 0x7f56c415afa0 2024-04-25T12:37:35.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.231+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c41f08e0 con 0x7f56c415afa0 2024-04-25T12:37:35.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.232+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c41f0ea0 con 0x7f56c415afa0 2024-04-25T12:37:35.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.233+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c4076700 con 0x7f56c415afa0 2024-04-25T12:37:35.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.234+0000 7f56ba000640 1 -- 172.21.15.123:0/650678131 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f56bc01b020 con 0x7f56c415afa0 2024-04-25T12:37:35.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.235+0000 7f56ba000640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f56a00744b0 0x7f56a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.235+0000 7f56ba000640 1 -- 172.21.15.123:0/650678131 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f56bc096320 con 0x7f56c415afa0 2024-04-25T12:37:35.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.235+0000 7f56c1e00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f56a00744b0 0x7f56a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.236+0000 7f56c1e00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f56a00744b0 0x7f56a00769c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034000 tx=0x7f56b402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:35.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.240+0000 7f56ba000640 1 -- 172.21.15.123:0/650678131 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56bc022020 con 0x7f56c415afa0 2024-04-25T12:37:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.441+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f56c40c6dc0 con 0x7f56a00744b0 2024-04-25T12:37:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.442+0000 7f56ba000640 1 -- 172.21.15.123:0/650678131 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f56c40c6dc0 con 0x7f56a00744b0 2024-04-25T12:37:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.445+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f56a00744b0 msgr2=0x7f56a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.445+0000 7f56c8a00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f56a00744b0 0x7f56a00769c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034000 tx=0x7f56b402e000 comp rx=0 tx=0).stop 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.445+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c415afa0 msgr2=0x7f56c41574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.445+0000 7f56c8a00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c415afa0 0x7f56c41574d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00eb70 tx=0x7f56bc00c490 comp rx=0 tx=0).stop 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 shutdown_connections 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f56a00744b0 0x7f56a00769c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c415afa0 0x7f56c41574d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c4d50 0x7f56c414d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 --2- 172.21.15.123:0/650678131 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56c4074870 0x7f56c4158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 >> 172.21.15.123:0/650678131 conn(0x7f56c406c750 msgr2=0x7f56c406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:35.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 shutdown_connections 2024-04-25T12:37:35.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.446+0000 7f56c8a00640 1 -- 172.21.15.123:0/650678131 wait complete. 2024-04-25T12:37:35.473 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T12:37:35.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:35.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.739+0000 7f3a67200640 1 Processor -- start 2024-04-25T12:37:35.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.739+0000 7f3a67200640 1 -- start start 2024-04-25T12:37:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.741+0000 7f3a67200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a68072d80 0x7f3a68073190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.741+0000 7f3a67200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a6815dbf0 0x7f3a6815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.741+0000 7f3a67200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6815a170 con 0x7f3a68154160 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.741+0000 7f3a67200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6815d760 con 0x7f3a68072d80 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.741+0000 7f3a67200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3a680736d0 con 0x7f3a6815dbf0 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.741+0000 7f3a65e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3a68154160 0x7f3a680c6b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60882/0 (socket says 172.21.15.123:60882) 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.741+0000 7f3a65e00640 1 -- 172.21.15.123:0/2740547257 learned_addr learned my addr 172.21.15.123:0/2740547257 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a65400640 1 --2- 172.21.15.123:0/2740547257 >> v2:172.21.15.153:3300/0 conn(0x7f3a68072d80 0x7f3a68073190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a66800640 1 --2- 172.21.15.123:0/2740547257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a6815dbf0 0x7f3a6815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a64a00640 1 -- 172.21.15.123:0/2740547257 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880071165 0 0) 0x7f3a6815a170 con 0x7f3a68154160 2024-04-25T12:37:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a64a00640 1 -- 172.21.15.123:0/2740547257 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a4c003520 con 0x7f3a68154160 2024-04-25T12:37:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a66800640 1 -- 172.21.15.123:0/2740547257 >> v2:172.21.15.153:3300/0 conn(0x7f3a68072d80 msgr2=0x7f3a68073190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a66800640 1 --2- 172.21.15.123:0/2740547257 >> v2:172.21.15.153:3300/0 conn(0x7f3a68072d80 0x7f3a68073190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a66800640 1 -- 172.21.15.123:0/2740547257 >> v1:172.21.15.123:6789/0 conn(0x7f3a68154160 legacy=0x7f3a680c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a66800640 1 -- 172.21.15.123:0/2740547257 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6815a410 con 0x7f3a6815dbf0 2024-04-25T12:37:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.742+0000 7f3a65400640 1 --2- 172.21.15.123:0/2740547257 >> v2:172.21.15.153:3300/0 conn(0x7f3a68072d80 0x7f3a68073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.743+0000 7f3a66800640 1 --2- 172.21.15.123:0/2740547257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a6815dbf0 0x7f3a6815e000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a6815d8e0 tx=0x7f3a6002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9a3c9a519bd5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a64a00640 1 -- 172.21.15.123:0/2740547257 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60007740 con 0x7f3a6815dbf0 2024-04-25T12:37:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a64a00640 1 -- 172.21.15.123:0/2740547257 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a600078e0 con 0x7f3a6815dbf0 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a64a00640 1 -- 172.21.15.123:0/2740547257 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60007c90 con 0x7f3a6815dbf0 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a67200640 1 -- 172.21.15.123:0/2740547257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a6815dbf0 msgr2=0x7f3a6815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a67200640 1 --2- 172.21.15.123:0/2740547257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a6815dbf0 0x7f3a6815e000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a6815d8e0 tx=0x7f3a6002a6d0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a67200640 1 -- 172.21.15.123:0/2740547257 shutdown_connections 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a67200640 1 --2- 172.21.15.123:0/2740547257 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a6815dbf0 0x7f3a6815e000 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a67200640 1 --2- 172.21.15.123:0/2740547257 >> v2:172.21.15.153:3300/0 conn(0x7f3a68072d80 0x7f3a68073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.744+0000 7f3a67200640 1 -- 172.21.15.123:0/2740547257 >> 172.21.15.123:0/2740547257 conn(0x7f3a680c9f60 msgr2=0x7f3a680ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.745+0000 7f3a67200640 1 -- 172.21.15.123:0/2740547257 shutdown_connections 2024-04-25T12:37:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.745+0000 7f3a67200640 1 -- 172.21.15.123:0/2740547257 wait complete. 2024-04-25T12:37:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.747+0000 7f3a67200640 1 Processor -- start 2024-04-25T12:37:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.747+0000 7f3a67200640 1 -- start start 2024-04-25T12:37:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.748+0000 7f3a67200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a68072d80 0x7f3a6815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.748+0000 7f3a67200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a68154160 0x7f3a681f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.748+0000 7f3a67200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6815dbf0 0x7f3a6815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.748+0000 7f3a67200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6815acb0 con 0x7f3a68154160 2024-04-25T12:37:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.748+0000 7f3a67200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6815a9d0 con 0x7f3a6815dbf0 2024-04-25T12:37:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.748+0000 7f3a67200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6815af90 con 0x7f3a68072d80 2024-04-25T12:37:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a68154160 0x7f3a681f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a68154160 0x7f3a681f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59992/0 (socket says 172.21.15.123:59992) 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 -- 172.21.15.123:0/1154755695 learned_addr learned my addr 172.21.15.123:0/1154755695 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65e00640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a68072d80 0x7f3a6815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a66800640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6815dbf0 0x7f3a6815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 -- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a68072d80 msgr2=0x7f3a6815ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3a68072d80 0x7f3a6815ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 -- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6815dbf0 msgr2=0x7f3a6815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6815dbf0 0x7f3a6815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.749+0000 7f3a65400640 1 -- 172.21.15.123:0/1154755695 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a680ccee0 con 0x7f3a68154160 2024-04-25T12:37:35.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.750+0000 7f3a65400640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a68154160 0x7f3a681f36f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400e990 tx=0x7f3a5400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.750+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017070 con 0x7f3a68154160 2024-04-25T12:37:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.750+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5400c9e0 con 0x7f3a68154160 2024-04-25T12:37:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.750+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400cdb0 con 0x7f3a68154160 2024-04-25T12:37:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.750+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6815bff0 con 0x7f3a68154160 2024-04-25T12:37:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.750+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6815c290 con 0x7f3a68154160 2024-04-25T12:37:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.751+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a68076da0 con 0x7f3a68154160 2024-04-25T12:37:35.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.752+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a5401a070 con 0x7f3a68154160 2024-04-25T12:37:35.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.753+0000 7f3a5e000640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a480744b0 0x7f3a480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.753+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f3a540955c0 con 0x7f3a68154160 2024-04-25T12:37:35.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.753+0000 7f3a5e000640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f3a4807e190 0x7f3a48080680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.753+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a48080d50 con 0x7f3a4807e190 2024-04-25T12:37:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.753+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f3a5409c050 con 0x7f3a68154160 2024-04-25T12:37:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.754+0000 7f3a65e00640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a480744b0 0x7f3a480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.754+0000 7f3a66800640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f3a4807e190 0x7f3a48080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.754+0000 7f3a66800640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f3a4807e190 0x7f3a48080680 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.754+0000 7f3a65e00640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a480744b0 0x7f3a480769c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3a58005880 tx=0x7f3a58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:35.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.755+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== osd.1 v2:172.21.15.123:6800/1234958824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a48080d50 con 0x7f3a4807e190 2024-04-25T12:37:35.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.769+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 --> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3a681e4060 con 0x7f3a4807e190 2024-04-25T12:37:35.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.769+0000 7f3a5e000640 1 -- 172.21.15.123:0/1154755695 <== osd.1 v2:172.21.15.123:6800/1234958824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3a681e4060 con 0x7f3a4807e190 2024-04-25T12:37:35.768 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:35.769 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:35.769 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f3a4807e190 msgr2=0x7f3a48080680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:6800/1234958824,v1:172.21.15.123:6801/1234958824] conn(0x7f3a4807e190 0x7f3a48080680 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a480744b0 msgr2=0x7f3a480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f3a480744b0 0x7f3a480769c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3a58005880 tx=0x7f3a58009040 comp rx=0 tx=0).stop 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a68154160 msgr2=0x7f3a681f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 --2- 172.21.15.123:0/1154755695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a68154160 0x7f3a681f36f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400e990 tx=0x7f3a5400ee50 comp rx=0 tx=0).stop 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a66800640 1 -- 172.21.15.123:0/1154755695 reap_dead start 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 shutdown_connections 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 >> 172.21.15.123:0/1154755695 conn(0x7f3a680c9f60 msgr2=0x7f3a681e5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 shutdown_connections 2024-04-25T12:37:35.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:35.770+0000 7f3a67200640 1 -- 172.21.15.123:0/1154755695 wait complete. 2024-04-25T12:37:35.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T12:37:36.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.063+0000 7f170e000640 1 Processor -- start 2024-04-25T12:37:36.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.064+0000 7f170e000640 1 -- start start 2024-04-25T12:37:36.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170e000640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f170815ca70 0x7f170815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170e000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 0x7f17080c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170e000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170806ecd0 con 0x7f17081567b0 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170e000640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f170806efb0 con 0x7f170815ca70 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170e000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f170806f290 con 0x7f17080c82b0 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170cc00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f17081567b0 0x7f170806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60896/0 (socket says 172.21.15.123:60896) 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170cc00640 1 -- 172.21.15.123:0/1884124083 learned_addr learned my addr 172.21.15.123:0/1884124083 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f170d600640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 0x7f17080c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.066+0000 7f1707e00640 1 --2- 172.21.15.123:0/1884124083 >> v2:172.21.15.153:3300/0 conn(0x7f170815ca70 0x7f170815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502403406 0 0) 0x7f170806ecd0 con 0x7f17081567b0 2024-04-25T12:37:36.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16ec003520 con 0x7f17081567b0 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1559499446 0 0) 0x7f16ec003520 con 0x7f17081567b0 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 msgr2=0x7f17080c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 0x7f17080c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 >> v2:172.21.15.153:3300/0 conn(0x7f170815ca70 msgr2=0x7f170815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 --2- 172.21.15.123:0/1884124083 >> v2:172.21.15.153:3300/0 conn(0x7f170815ca70 0x7f170815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170806f570 con 0x7f17081567b0 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1134209144 0 0) 0x7f16f80019f0 con 0x7f17081567b0 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707e00640 1 --2- 172.21.15.123:0/1884124083 >> v2:172.21.15.153:3300/0 conn(0x7f170815ca70 0x7f170815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 >> v1:172.21.15.123:6789/0 conn(0x7f17081567b0 legacy=0x7f170806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16ec0039b0 0x7f16ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec004800 0x7f16ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 0x7f16ec005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f170806efb0 con 0x7f16ec0039b0 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f170806f290 con 0x7f16ec004800 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.067+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f16ec00ba40 con 0x7f17080c82b0 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f1707e00640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec004800 0x7f16ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f170cc00640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16ec0039b0 0x7f16ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f170d600640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 0x7f16ec005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f170cc00640 1 -- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 msgr2=0x7f16ec005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f170cc00640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f17080c82b0 0x7f16ec005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f170cc00640 1 -- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec004800 msgr2=0x7f16ec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f170cc00640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec004800 0x7f16ec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f170cc00640 1 -- 172.21.15.123:0/1884124083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec003520 con 0x7f16ec0039b0 2024-04-25T12:37:36.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.068+0000 7f1707e00640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec004800 0x7f16ec004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:36.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.069+0000 7f170cc00640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16ec0039b0 0x7f16ec003e20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16f800a920 tx=0x7f16ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b70f60ebc9231bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.069+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f800a4c0 con 0x7f16ec0039b0 2024-04-25T12:37:36.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.069+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f802f070 con 0x7f16ec0039b0 2024-04-25T12:37:36.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.069+0000 7f1707400640 1 -- 172.21.15.123:0/1884124083 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f802bd00 con 0x7f16ec0039b0 2024-04-25T12:37:36.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.070+0000 7f170e000640 1 -- 172.21.15.123:0/1884124083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16ec0039b0 msgr2=0x7f16ec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.070+0000 7f170e000640 1 --2- 172.21.15.123:0/1884124083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16ec0039b0 0x7f16ec003e20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16f800a920 tx=0x7f16ec005830 comp rx=0 tx=0).stop 2024-04-25T12:37:36.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.070+0000 7f170d600640 1 -- 172.21.15.123:0/1884124083 reap_dead start 2024-04-25T12:37:36.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.070+0000 7f170e000640 1 -- 172.21.15.123:0/1884124083 shutdown_connections 2024-04-25T12:37:36.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.070+0000 7f170e000640 1 -- 172.21.15.123:0/1884124083 >> 172.21.15.123:0/1884124083 conn(0x7f17080cd600 msgr2=0x7f17080cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:36.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.070+0000 7f170e000640 1 -- 172.21.15.123:0/1884124083 shutdown_connections 2024-04-25T12:37:36.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.070+0000 7f170e000640 1 -- 172.21.15.123:0/1884124083 wait complete. 2024-04-25T12:37:36.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.073+0000 7f170e000640 1 Processor -- start 2024-04-25T12:37:36.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.073+0000 7f170e000640 1 -- start start 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.074+0000 7f170e000640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16ec0039b0 0x7f17081f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.074+0000 7f170e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17081f48a0 0x7f170815eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.074+0000 7f170e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f170815a3e0 0x7f170815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.074+0000 7f170e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17080c9d20 con 0x7f170815a3e0 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.074+0000 7f170e000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f170815d3c0 con 0x7f17081f48a0 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.074+0000 7f170e000640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1708156bc0 con 0x7f16ec0039b0 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f170815a3e0 0x7f170815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f170815a3e0 0x7f170815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60010/0 (socket says 172.21.15.123:60010) 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 -- 172.21.15.123:0/322085015 learned_addr learned my addr 172.21.15.123:0/322085015 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170cc00640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16ec0039b0 0x7f17081f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f1707e00640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17081f48a0 0x7f170815eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 -- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16ec0039b0 msgr2=0x7f17081f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16ec0039b0 0x7f17081f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 -- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17081f48a0 msgr2=0x7f170815eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17081f48a0 0x7f170815eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 -- 172.21.15.123:0/322085015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170815bdb0 con 0x7f170815a3e0 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.075+0000 7f170d600640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f170815a3e0 0x7f170815a7f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f170815faf0 tx=0x7f1700011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.076+0000 7f1704c00640 1 -- 172.21.15.123:0/322085015 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0009860 con 0x7f170815a3e0 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.076+0000 7f1704c00640 1 -- 172.21.15.123:0/322085015 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170001b070 con 0x7f170815a3e0 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.076+0000 7f1704c00640 1 -- 172.21.15.123:0/322085015 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700004230 con 0x7f170815a3e0 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.076+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170815c070 con 0x7f170815a3e0 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.076+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17081f40a0 con 0x7f170815a3e0 2024-04-25T12:37:36.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.077+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1708076da0 con 0x7f170815a3e0 2024-04-25T12:37:36.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.078+0000 7f1704c00640 1 -- 172.21.15.123:0/322085015 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1700012040 con 0x7f170815a3e0 2024-04-25T12:37:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.080+0000 7f1704c00640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f16e40744b0 0x7f16e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.080+0000 7f1704c00640 1 -- 172.21.15.123:0/322085015 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f1700095250 con 0x7f170815a3e0 2024-04-25T12:37:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.080+0000 7f170cc00640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f16e40744b0 0x7f16e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.081+0000 7f170cc00640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f16e40744b0 0x7f16e40769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f16f800a920 tx=0x7f16f8007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.085+0000 7f1704c00640 1 -- 172.21.15.123:0/322085015 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f170005f530 con 0x7f170815a3e0 2024-04-25T12:37:36.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.286+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f170806fe00 con 0x7f16e40744b0 2024-04-25T12:37:36.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.288+0000 7f1704c00640 1 -- 172.21.15.123:0/322085015 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f170806fe00 con 0x7f16e40744b0 2024-04-25T12:37:36.287 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f16e40744b0 msgr2=0x7f16e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f16e40744b0 0x7f16e40769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f16f800a920 tx=0x7f16f8007000 comp rx=0 tx=0).stop 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f170815a3e0 msgr2=0x7f170815a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f170815a3e0 0x7f170815a7f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f170815faf0 tx=0x7f1700011c30 comp rx=0 tx=0).stop 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 shutdown_connections 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f16e40744b0 0x7f16e40769c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f170815a3e0 0x7f170815a7f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17081f48a0 0x7f170815eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 --2- 172.21.15.123:0/322085015 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16ec0039b0 0x7f17081f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 >> 172.21.15.123:0/322085015 conn(0x7f17080cd600 msgr2=0x7f1708155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:36.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 shutdown_connections 2024-04-25T12:37:36.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.291+0000 7f170e000640 1 -- 172.21.15.123:0/322085015 wait complete. 2024-04-25T12:37:36.317 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T12:37:36.317 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:36.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.573+0000 7fcf3b200640 1 Processor -- start 2024-04-25T12:37:36.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.574+0000 7fcf3b200640 1 -- start start 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf3b200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcf3415c380 0x7fcf340ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf3b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3415c950 con 0x7fcf340c6700 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf3b200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3415cc30 con 0x7fcf340cd3e0 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf3b200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3415cf10 con 0x7fcf3415c380 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf39400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcf3415c380 0x7fcf340ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf39e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcf340c6700 0x7fcf34160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60900/0 (socket says 172.21.15.123:60900) 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf39e00640 1 -- 172.21.15.123:0/1508549333 learned_addr learned my addr 172.21.15.123:0/1508549333 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:36.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf3a800640 1 --1- 172.21.15.123:0/1508549333 >> v1:172.21.15.153:6789/0 conn(0x7fcf340cd3e0 0x7fcf34166d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34770/0 (socket says 172.21.15.123:34770) 2024-04-25T12:37:36.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf39400640 1 -- 172.21.15.123:0/1508549333 >> v1:172.21.15.153:6789/0 conn(0x7fcf340cd3e0 legacy=0x7fcf34166d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.576+0000 7fcf39400640 1 -- 172.21.15.123:0/1508549333 >> v1:172.21.15.123:6789/0 conn(0x7fcf340c6700 legacy=0x7fcf34160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.577+0000 7fcf39400640 1 -- 172.21.15.123:0/1508549333 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3415d1f0 con 0x7fcf3415c380 2024-04-25T12:37:36.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.578+0000 7fcf39400640 1 --2- 172.21.15.123:0/1508549333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf3415c380 0x7fcf340ccea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcf240097a0 tx=0x7fcf2402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e94cd64fc2b000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.578+0000 7fcf38a00640 1 -- 172.21.15.123:0/1508549333 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24037070 con 0x7fcf3415c380 2024-04-25T12:37:36.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.578+0000 7fcf38a00640 1 -- 172.21.15.123:0/1508549333 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf24032640 con 0x7fcf3415c380 2024-04-25T12:37:36.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.578+0000 7fcf38a00640 1 -- 172.21.15.123:0/1508549333 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24032a10 con 0x7fcf3415c380 2024-04-25T12:37:36.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.579+0000 7fcf3b200640 1 -- 172.21.15.123:0/1508549333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf3415c380 msgr2=0x7fcf340ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.579+0000 7fcf3b200640 1 --2- 172.21.15.123:0/1508549333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf3415c380 0x7fcf340ccea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcf240097a0 tx=0x7fcf2402a2c0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.579+0000 7fcf3b200640 1 -- 172.21.15.123:0/1508549333 shutdown_connections 2024-04-25T12:37:36.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.579+0000 7fcf3b200640 1 --2- 172.21.15.123:0/1508549333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf3415c380 0x7fcf340ccea0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.579+0000 7fcf3b200640 1 -- 172.21.15.123:0/1508549333 >> 172.21.15.123:0/1508549333 conn(0x7fcf3406db20 msgr2=0x7fcf3406df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:36.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.579+0000 7fcf3b200640 1 -- 172.21.15.123:0/1508549333 shutdown_connections 2024-04-25T12:37:36.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.579+0000 7fcf3b200640 1 -- 172.21.15.123:0/1508549333 wait complete. 2024-04-25T12:37:36.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.582+0000 7fcf3b200640 1 Processor -- start 2024-04-25T12:37:36.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.582+0000 7fcf3b200640 1 -- start start 2024-04-25T12:37:36.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3b200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf340c6700 0x7fcf341e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf340cd3e0 0x7fcf341e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3415c380 0x7fcf341512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34160880 con 0x7fcf3415c380 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3415fc50 con 0x7fcf340cd3e0 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3b200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcf340c6f20 con 0x7fcf340c6700 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3415c380 0x7fcf341512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf39e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf340c6700 0x7fcf341e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf39400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf340cd3e0 0x7fcf341e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf39400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf340cd3e0 0x7fcf341e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41738/0 (socket says 172.21.15.123:41738) 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf39400640 1 -- 172.21.15.123:0/494120879 learned_addr learned my addr 172.21.15.123:0/494120879 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.584+0000 7fcf3a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3415c380 0x7fcf341512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60016/0 (socket says 172.21.15.123:60016) 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.585+0000 7fcf39400640 1 -- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf340c6700 msgr2=0x7fcf341e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.585+0000 7fcf39400640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcf340c6700 0x7fcf341e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.585+0000 7fcf39400640 1 -- 172.21.15.123:0/494120879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3415c380 msgr2=0x7fcf341512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.585+0000 7fcf39400640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3415c380 0x7fcf341512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.585+0000 7fcf39400640 1 -- 172.21.15.123:0/494120879 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf341e8590 con 0x7fcf340cd3e0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.585+0000 7fcf39400640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf340cd3e0 0x7fcf341e8050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcf24035040 tx=0x7fcf240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.586+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24041030 con 0x7fcf340cd3e0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.586+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf24030750 con 0x7fcf340cd3e0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.586+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf341e5bb0 con 0x7fcf340cd3e0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.586+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24037040 con 0x7fcf340cd3e0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.586+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf341e6170 con 0x7fcf340cd3e0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.587+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf34076da0 con 0x7fcf340cd3e0 2024-04-25T12:37:36.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.588+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf24044070 con 0x7fcf340cd3e0 2024-04-25T12:37:36.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.589+0000 7fcf2a000640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcf180744b0 0x7fcf180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.589+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fcf240b65d0 con 0x7fcf340cd3e0 2024-04-25T12:37:36.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.590+0000 7fcf39e00640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcf180744b0 0x7fcf180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.590+0000 7fcf2a000640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcf1807e190 0x7fcf18080680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.590+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf18080d50 con 0x7fcf1807e190 2024-04-25T12:37:36.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.590+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fcf240b6980 con 0x7fcf340cd3e0 2024-04-25T12:37:36.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.590+0000 7fcf3a800640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcf1807e190 0x7fcf18080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.590+0000 7fcf39e00640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcf180744b0 0x7fcf180769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005c00 tx=0x7fcf2c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.591+0000 7fcf3a800640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcf1807e190 0x7fcf18080680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.591+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== osd.2 v2:172.21.15.123:6808/2222173849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf18080d50 con 0x7fcf1807e190 2024-04-25T12:37:36.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.605+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 --> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcf341e5960 con 0x7fcf1807e190 2024-04-25T12:37:36.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.606+0000 7fcf2a000640 1 -- 172.21.15.123:0/494120879 <== osd.2 v2:172.21.15.123:6808/2222173849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcf341e5960 con 0x7fcf1807e190 2024-04-25T12:37:36.606 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:36.606 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:36.606 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.606+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcf1807e190 msgr2=0x7fcf18080680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.606+0000 7fcf3b200640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.123:6808/2222173849,v1:172.21.15.123:6809/2222173849] conn(0x7fcf1807e190 0x7fcf18080680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.606+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcf180744b0 msgr2=0x7fcf180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.606+0000 7fcf3b200640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fcf180744b0 0x7fcf180769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005c00 tx=0x7fcf2c009040 comp rx=0 tx=0).stop 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.607+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf340cd3e0 msgr2=0x7fcf341e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.607+0000 7fcf3b200640 1 --2- 172.21.15.123:0/494120879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf340cd3e0 0x7fcf341e8050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcf24035040 tx=0x7fcf240046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.607+0000 7fcf3a800640 1 -- 172.21.15.123:0/494120879 reap_dead start 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.607+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 shutdown_connections 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.607+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 >> 172.21.15.123:0/494120879 conn(0x7fcf3406db20 msgr2=0x7fcf34152620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.607+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 shutdown_connections 2024-04-25T12:37:36.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.607+0000 7fcf3b200640 1 -- 172.21.15.123:0/494120879 wait complete. 2024-04-25T12:37:36.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T12:37:36.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.901+0000 7f771da00640 1 Processor -- start 2024-04-25T12:37:36.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.901+0000 7f771da00640 1 -- start start 2024-04-25T12:37:36.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.903+0000 7f771da00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f77180c9080 0x7f77181546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.903+0000 7f771da00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771815a330 con 0x7f77180c82b0 2024-04-25T12:37:36.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.903+0000 7f771da00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7718073b10 con 0x7f771806c570 2024-04-25T12:37:36.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.903+0000 7f771da00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7718073df0 con 0x7f77180c9080 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.903+0000 7f7717e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f77180c9080 0x7f77181546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f7717e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f77180c9080 0x7f77181546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41742/0 (socket says 172.21.15.123:41742) 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f7717e00640 1 -- 172.21.15.123:0/825934988 learned_addr learned my addr 172.21.15.123:0/825934988 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f771d000640 1 --1- 172.21.15.123:0/825934988 >> v1:172.21.15.153:6789/0 conn(0x7f771806c570 0x7f7718164cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34774/0 (socket says 172.21.15.123:34774) 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f770f400640 1 --1- 172.21.15.123:0/825934988 >> v1:172.21.15.123:6789/0 conn(0x7f77180c82b0 0x7f771815e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60914/0 (socket says 172.21.15.123:60914) 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f7717e00640 1 -- 172.21.15.123:0/825934988 >> v1:172.21.15.153:6789/0 conn(0x7f771806c570 legacy=0x7f7718164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f7717400640 1 -- 172.21.15.123:0/825934988 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858877888 0 0) 0x7f771815a330 con 0x7f77180c82b0 2024-04-25T12:37:36.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f7717e00640 1 -- 172.21.15.123:0/825934988 >> v1:172.21.15.123:6789/0 conn(0x7f77180c82b0 legacy=0x7f771815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.904+0000 7f7717e00640 1 -- 172.21.15.123:0/825934988 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180740d0 con 0x7f77180c9080 2024-04-25T12:37:36.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.905+0000 7f7717e00640 1 --2- 172.21.15.123:0/825934988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c9080 0x7f77181546d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79076a0dc7a2b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.905+0000 7f7717400640 1 -- 172.21.15.123:0/825934988 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708037070 con 0x7f77180c9080 2024-04-25T12:37:36.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.905+0000 7f7717400640 1 -- 172.21.15.123:0/825934988 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708032440 con 0x7f77180c9080 2024-04-25T12:37:36.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.905+0000 7f7717400640 1 -- 172.21.15.123:0/825934988 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708032810 con 0x7f77180c9080 2024-04-25T12:37:36.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.906+0000 7f771da00640 1 -- 172.21.15.123:0/825934988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c9080 msgr2=0x7f77181546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.906+0000 7f771da00640 1 --2- 172.21.15.123:0/825934988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c9080 0x7f77181546d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770802a270 comp rx=0 tx=0).stop 2024-04-25T12:37:36.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.906+0000 7f771da00640 1 -- 172.21.15.123:0/825934988 shutdown_connections 2024-04-25T12:37:36.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.906+0000 7f771da00640 1 --2- 172.21.15.123:0/825934988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c9080 0x7f77181546d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.906+0000 7f771da00640 1 -- 172.21.15.123:0/825934988 >> 172.21.15.123:0/825934988 conn(0x7f77180cd600 msgr2=0x7f77180cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:36.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.906+0000 7f771da00640 1 -- 172.21.15.123:0/825934988 shutdown_connections 2024-04-25T12:37:36.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.906+0000 7f771da00640 1 -- 172.21.15.123:0/825934988 wait complete. 2024-04-25T12:37:36.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.909+0000 7f771da00640 1 Processor -- start 2024-04-25T12:37:36.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.909+0000 7f771da00640 1 -- start start 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.910+0000 7f771da00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f771806c570 0x7f7718153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.910+0000 7f771da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c82b0 0x7f77181f0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.910+0000 7f771da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77180c9080 0x7f771815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.910+0000 7f771da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f771815e760 con 0x7f77180c9080 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.910+0000 7f771da00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f771815db30 con 0x7f77180c82b0 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.910+0000 7f771da00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7718073760 con 0x7f771806c570 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.910+0000 7f771d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77180c9080 0x7f771815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f771806c570 0x7f7718153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f771d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77180c9080 0x7f771815cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60018/0 (socket says 172.21.15.123:60018) 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f771806c570 0x7f7718153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:44912/0 (socket says 172.21.15.123:44912) 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f771d000640 1 -- 172.21.15.123:0/1660603282 learned_addr learned my addr 172.21.15.123:0/1660603282 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f770f400640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c82b0 0x7f77181f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 -- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c82b0 msgr2=0x7f77181f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c82b0 0x7f77181f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 -- 172.21.15.123:0/1660603282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77180c9080 msgr2=0x7f771815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77180c9080 0x7f771815cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 -- 172.21.15.123:0/1660603282 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77181f0ed0 con 0x7f771806c570 2024-04-25T12:37:36.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.911+0000 7f7717e00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f771806c570 0x7f7718153300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7708031000 tx=0x7f77080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.912+0000 7f7714c00640 1 -- 172.21.15.123:0/1660603282 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708041020 con 0x7f771806c570 2024-04-25T12:37:36.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.912+0000 7f7714c00640 1 -- 172.21.15.123:0/1660603282 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708030970 con 0x7f771806c570 2024-04-25T12:37:36.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.912+0000 7f7714c00640 1 -- 172.21.15.123:0/1660603282 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708037040 con 0x7f771806c570 2024-04-25T12:37:36.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.912+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7718153ce0 con 0x7f771806c570 2024-04-25T12:37:36.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.912+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77181f10e0 con 0x7f771806c570 2024-04-25T12:37:36.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.913+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718153a70 con 0x7f771806c570 2024-04-25T12:37:36.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.914+0000 7f7714c00640 1 -- 172.21.15.123:0/1660603282 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7708044020 con 0x7f771806c570 2024-04-25T12:37:36.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.915+0000 7f7714c00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76f40744b0 0x7f76f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:36.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.915+0000 7f7714c00640 1 -- 172.21.15.123:0/1660603282 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f77080b5f60 con 0x7f771806c570 2024-04-25T12:37:36.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.915+0000 7f770f400640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76f40744b0 0x7f76f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:36.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.916+0000 7f770f400640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76f40744b0 0x7f76f40769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7700007bf0 tx=0x7f7700007b40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:36.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:36.920+0000 7f7714c00640 1 -- 172.21.15.123:0/1660603282 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f770804b030 con 0x7f771806c570 2024-04-25T12:37:37.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.123+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f771806f6a0 con 0x7f76f40744b0 2024-04-25T12:37:37.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.125+0000 7f7714c00640 1 -- 172.21.15.123:0/1660603282 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f771806f6a0 con 0x7f76f40744b0 2024-04-25T12:37:37.125 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76f40744b0 msgr2=0x7f76f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76f40744b0 0x7f76f40769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7700007bf0 tx=0x7f7700007b40 comp rx=0 tx=0).stop 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f771806c570 msgr2=0x7f7718153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f771806c570 0x7f7718153300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7708031000 tx=0x7f77080046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 shutdown_connections 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f76f40744b0 0x7f76f40769c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77180c9080 0x7f771815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77180c82b0 0x7f77181f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 --2- 172.21.15.123:0/1660603282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f771806c570 0x7f7718153300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 >> 172.21.15.123:0/1660603282 conn(0x7f77180cd600 msgr2=0x7f7718075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 shutdown_connections 2024-04-25T12:37:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.128+0000 7f771da00640 1 -- 172.21.15.123:0/1660603282 wait complete. 2024-04-25T12:37:37.153 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T12:37:37.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:37.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.436+0000 7feda7e00640 1 Processor -- start 2024-04-25T12:37:37.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.436+0000 7feda7e00640 1 -- start start 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda7e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda00c8990 con 0x7feda006f3f0 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda7e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda00c8c30 con 0x7feda0074750 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feda00c8f10 con 0x7feda0152f20 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda5e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7feda006f3f0 0x7feda006f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60918/0 (socket says 172.21.15.123:60918) 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda5400640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7feda0074750 0x7feda0159440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34782/0 (socket says 172.21.15.123:34782) 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda5e00640 1 -- 172.21.15.123:0/2542068267 learned_addr learned my addr 172.21.15.123:0/2542068267 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.439+0000 7feda4a00640 1 --2- 172.21.15.123:0/2542068267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.440+0000 7fed9fe00640 1 -- 172.21.15.123:0/2542068267 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368588081 0 0) 0x7feda00c8c30 con 0x7feda0074750 2024-04-25T12:37:37.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.440+0000 7fed9fe00640 1 -- 172.21.15.123:0/2542068267 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed88003520 con 0x7feda0074750 2024-04-25T12:37:37.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.440+0000 7fed9fe00640 1 -- 172.21.15.123:0/2542068267 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557390514 0 0) 0x7feda00c8990 con 0x7feda006f3f0 2024-04-25T12:37:37.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.440+0000 7feda4a00640 1 -- 172.21.15.123:0/2542068267 >> v1:172.21.15.153:6789/0 conn(0x7feda0074750 legacy=0x7feda0159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.440+0000 7feda4a00640 1 -- 172.21.15.123:0/2542068267 >> v1:172.21.15.123:6789/0 conn(0x7feda006f3f0 legacy=0x7feda006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.440+0000 7feda4a00640 1 -- 172.21.15.123:0/2542068267 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda0158c40 con 0x7feda0152f20 2024-04-25T12:37:37.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.441+0000 7feda4a00640 1 --2- 172.21.15.123:0/2542068267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda006eeb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0097d0 tx=0x7fed8c02a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda25501626f26a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:37.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.441+0000 7fed9fe00640 1 -- 172.21.15.123:0/2542068267 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c037070 con 0x7feda0152f20 2024-04-25T12:37:37.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.441+0000 7fed9fe00640 1 -- 172.21.15.123:0/2542068267 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8c032640 con 0x7feda0152f20 2024-04-25T12:37:37.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.441+0000 7fed9fe00640 1 -- 172.21.15.123:0/2542068267 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0329f0 con 0x7feda0152f20 2024-04-25T12:37:37.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.442+0000 7feda7e00640 1 -- 172.21.15.123:0/2542068267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 msgr2=0x7feda006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.442+0000 7feda7e00640 1 --2- 172.21.15.123:0/2542068267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda006eeb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0097d0 tx=0x7fed8c02a2c0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.442+0000 7feda7e00640 1 -- 172.21.15.123:0/2542068267 shutdown_connections 2024-04-25T12:37:37.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.442+0000 7feda7e00640 1 --2- 172.21.15.123:0/2542068267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda006eeb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.442+0000 7feda7e00640 1 -- 172.21.15.123:0/2542068267 >> 172.21.15.123:0/2542068267 conn(0x7feda006c570 msgr2=0x7feda006c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:37.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.442+0000 7feda7e00640 1 -- 172.21.15.123:0/2542068267 shutdown_connections 2024-04-25T12:37:37.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.442+0000 7feda7e00640 1 -- 172.21.15.123:0/2542068267 wait complete. 2024-04-25T12:37:37.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.444+0000 7feda7e00640 1 Processor -- start 2024-04-25T12:37:37.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.444+0000 7feda7e00640 1 -- start start 2024-04-25T12:37:37.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda006f3f0 0x7feda01f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda0074750 0x7feda00ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda7e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda00ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feda006ff10 con 0x7feda0074750 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda7e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feda0158fb0 con 0x7feda006f3f0 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda7e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feda00c81c0 con 0x7feda0152f20 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda0074750 0x7feda00ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda0074750 0x7feda00ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60020/0 (socket says 172.21.15.123:60020) 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda4a00640 1 -- 172.21.15.123:0/3423571946 learned_addr learned my addr 172.21.15.123:0/3423571946 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda5400640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda006f3f0 0x7feda01f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.446+0000 7feda5e00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda00ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7feda4a00640 1 -- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 msgr2=0x7feda00ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7feda4a00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feda0152f20 0x7feda00ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7feda4a00640 1 -- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda006f3f0 msgr2=0x7feda01f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7feda4a00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda006f3f0 0x7feda01f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7feda4a00640 1 -- 172.21.15.123:0/3423571946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda0149cc0 con 0x7feda0074750 2024-04-25T12:37:37.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7feda4a00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda0074750 0x7feda00ca2c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fed8c000c00 tx=0x7fed8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:37.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c041070 con 0x7feda0074750 2024-04-25T12:37:37.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.447+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8c03c870 con 0x7feda0074750 2024-04-25T12:37:37.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.448+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda015c080 con 0x7feda0074750 2024-04-25T12:37:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.448+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c037040 con 0x7feda0074750 2024-04-25T12:37:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.448+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda015c640 con 0x7feda0074750 2024-04-25T12:37:37.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.448+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feda00cb1d0 con 0x7feda0074750 2024-04-25T12:37:37.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.450+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed8c044070 con 0x7feda0074750 2024-04-25T12:37:37.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.451+0000 7fed9d600640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fed800744b0 0x7fed800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.451+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fed8c0b6630 con 0x7feda0074750 2024-04-25T12:37:37.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.452+0000 7feda5400640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fed800744b0 0x7fed800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.452+0000 7fed9d600640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fed8007e190 0x7fed80080680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.452+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed80080d50 con 0x7fed8007e190 2024-04-25T12:37:37.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.452+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fed8c0b6a90 con 0x7feda0074750 2024-04-25T12:37:37.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.452+0000 7feda5e00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fed8007e190 0x7fed80080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.452+0000 7feda5400640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fed800744b0 0x7fed800769c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fed940011e0 tx=0x7fed94007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:37.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.452+0000 7feda5e00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fed8007e190 0x7fed80080680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:37.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.453+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== osd.3 v2:172.21.15.123:6824/1491071423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed80080d50 con 0x7fed8007e190 2024-04-25T12:37:37.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.468+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 --> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feda0149a70 con 0x7fed8007e190 2024-04-25T12:37:37.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.468+0000 7fed9d600640 1 -- 172.21.15.123:0/3423571946 <== osd.3 v2:172.21.15.123:6824/1491071423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feda0149a70 con 0x7fed8007e190 2024-04-25T12:37:37.468 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:37.468 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:37.468 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:37.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fed8007e190 msgr2=0x7fed80080680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:6824/1491071423,v1:172.21.15.123:6825/1491071423] conn(0x7fed8007e190 0x7fed80080680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fed800744b0 msgr2=0x7fed800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fed800744b0 0x7fed800769c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fed940011e0 tx=0x7fed94007d30 comp rx=0 tx=0).stop 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda0074750 msgr2=0x7feda00ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 --2- 172.21.15.123:0/3423571946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda0074750 0x7feda00ca2c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fed8c000c00 tx=0x7fed8c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda5e00640 1 -- 172.21.15.123:0/3423571946 reap_dead start 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 shutdown_connections 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 >> 172.21.15.123:0/3423571946 conn(0x7feda006c570 msgr2=0x7feda006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.469+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 shutdown_connections 2024-04-25T12:37:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.470+0000 7feda7e00640 1 -- 172.21.15.123:0/3423571946 wait complete. 2024-04-25T12:37:37.496 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T12:37:37.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.767+0000 7fe17be00640 1 Processor -- start 2024-04-25T12:37:37.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.767+0000 7fe17be00640 1 -- start start 2024-04-25T12:37:37.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.769+0000 7fe17be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe17c15ca70 0x7fe17c15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.769+0000 7fe17be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c0c82b0 0x7fe17c0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.769+0000 7fe17be00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe17c06ecd0 con 0x7fe17c1567b0 2024-04-25T12:37:37.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.769+0000 7fe17be00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe17c06efb0 con 0x7fe17c15ca70 2024-04-25T12:37:37.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.769+0000 7fe17be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe17c06f290 con 0x7fe17c0c82b0 2024-04-25T12:37:37.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.769+0000 7fe17a000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe17c15ca70 0x7fe17c15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.769+0000 7fe17b400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c0c82b0 0x7fe17c0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.770+0000 7fe17aa00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe17c1567b0 0x7fe17c06c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34784/0 (socket says 172.21.15.123:34784) 2024-04-25T12:37:37.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.770+0000 7fe17aa00640 1 -- 172.21.15.123:0/1241998609 learned_addr learned my addr 172.21.15.123:0/1241998609 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:37.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.770+0000 7fe179600640 1 -- 172.21.15.123:0/1241998609 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751201246 0 0) 0x7fe17c06ecd0 con 0x7fe17c1567b0 2024-04-25T12:37:37.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.770+0000 7fe17b400640 1 -- 172.21.15.123:0/1241998609 >> v2:172.21.15.123:3300/0 conn(0x7fe17c15ca70 msgr2=0x7fe17c15ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:37.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.770+0000 7fe17b400640 1 --2- 172.21.15.123:0/1241998609 >> v2:172.21.15.123:3300/0 conn(0x7fe17c15ca70 0x7fe17c15ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.770+0000 7fe17b400640 1 -- 172.21.15.123:0/1241998609 >> v1:172.21.15.153:6789/0 conn(0x7fe17c1567b0 legacy=0x7fe17c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.770+0000 7fe17b400640 1 -- 172.21.15.123:0/1241998609 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c06f570 con 0x7fe17c0c82b0 2024-04-25T12:37:37.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.771+0000 7fe17b400640 1 --2- 172.21.15.123:0/1241998609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c0c82b0 0x7fe17c0c86c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097a0 tx=0x7fe16c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4855a346be8b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:37.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.771+0000 7fe179600640 1 -- 172.21.15.123:0/1241998609 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c037070 con 0x7fe17c0c82b0 2024-04-25T12:37:37.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.771+0000 7fe179600640 1 -- 172.21.15.123:0/1241998609 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16c032440 con 0x7fe17c0c82b0 2024-04-25T12:37:37.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.771+0000 7fe179600640 1 -- 172.21.15.123:0/1241998609 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c032810 con 0x7fe17c0c82b0 2024-04-25T12:37:37.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 -- 172.21.15.123:0/1241998609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c0c82b0 msgr2=0x7fe17c0c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 --2- 172.21.15.123:0/1241998609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c0c82b0 0x7fe17c0c86c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097a0 tx=0x7fe16c02a210 comp rx=0 tx=0).stop 2024-04-25T12:37:37.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 -- 172.21.15.123:0/1241998609 shutdown_connections 2024-04-25T12:37:37.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 --2- 172.21.15.123:0/1241998609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c0c82b0 0x7fe17c0c86c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 --2- 172.21.15.123:0/1241998609 >> v2:172.21.15.123:3300/0 conn(0x7fe17c15ca70 0x7fe17c15ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 -- 172.21.15.123:0/1241998609 >> 172.21.15.123:0/1241998609 conn(0x7fe17c0cd600 msgr2=0x7fe17c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:37.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 -- 172.21.15.123:0/1241998609 shutdown_connections 2024-04-25T12:37:37.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.772+0000 7fe17be00640 1 -- 172.21.15.123:0/1241998609 wait complete. 2024-04-25T12:37:37.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.775+0000 7fe17be00640 1 Processor -- start 2024-04-25T12:37:37.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.775+0000 7fe17be00640 1 -- start start 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.776+0000 7fe17be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe17c0c82b0 0x7fe17c155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.776+0000 7fe17be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe17c1567b0 0x7fe17c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.776+0000 7fe17be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c15ca70 0x7fe17c14c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c0c8c00 con 0x7fe17c1567b0 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c15d3c0 con 0x7fe17c0c82b0 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe17c0c9d20 con 0x7fe17c15ca70 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe17c1567b0 0x7fe17c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe17c1567b0 0x7fe17c0cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60048/0 (socket says 172.21.15.123:60048) 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17a000640 1 -- 172.21.15.123:0/704257840 learned_addr learned my addr 172.21.15.123:0/704257840 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:37.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17b400640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c15ca70 0x7fe17c14c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17aa00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe17c0c82b0 0x7fe17c155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17b400640 1 -- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe17c0c82b0 msgr2=0x7fe17c155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17b400640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe17c0c82b0 0x7fe17c155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17b400640 1 -- 172.21.15.123:0/704257840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe17c1567b0 msgr2=0x7fe17c0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17b400640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe17c1567b0 0x7fe17c0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17b400640 1 -- 172.21.15.123:0/704257840 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c155dd0 con 0x7fe17c15ca70 2024-04-25T12:37:37.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.777+0000 7fe17a000640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe17c1567b0 0x7fe17c0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:37.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.778+0000 7fe17b400640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c15ca70 0x7fe17c14c0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe17c06f410 tx=0x7fe16c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:37.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.778+0000 7fe172a00640 1 -- 172.21.15.123:0/704257840 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c041030 con 0x7fe17c15ca70 2024-04-25T12:37:37.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.778+0000 7fe172a00640 1 -- 172.21.15.123:0/704257840 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16c03d4e0 con 0x7fe17c15ca70 2024-04-25T12:37:37.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.778+0000 7fe172a00640 1 -- 172.21.15.123:0/704257840 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c037040 con 0x7fe17c15ca70 2024-04-25T12:37:37.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.778+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17c1f20d0 con 0x7fe17c15ca70 2024-04-25T12:37:37.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.778+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17c158ed0 con 0x7fe17c15ca70 2024-04-25T12:37:37.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.779+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17c076da0 con 0x7fe17c15ca70 2024-04-25T12:37:37.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.780+0000 7fe172a00640 1 -- 172.21.15.123:0/704257840 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe16c044070 con 0x7fe17c15ca70 2024-04-25T12:37:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.781+0000 7fe172a00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe1580744e0 0x7fe1580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.782+0000 7fe172a00640 1 -- 172.21.15.123:0/704257840 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fe16c0b5be0 con 0x7fe17c15ca70 2024-04-25T12:37:37.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.782+0000 7fe17aa00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe1580744e0 0x7fe1580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:37.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.782+0000 7fe17aa00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe1580744e0 0x7fe1580769f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe174007fd0 tx=0x7fe174007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:37.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.786+0000 7fe172a00640 1 -- 172.21.15.123:0/704257840 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe16c07f870 con 0x7fe17c15ca70 2024-04-25T12:37:37.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.989+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe17c0c8e60 con 0x7fe1580744e0 2024-04-25T12:37:37.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.990+0000 7fe172a00640 1 -- 172.21.15.123:0/704257840 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe17c0c8e60 con 0x7fe1580744e0 2024-04-25T12:37:37.991 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.993+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe1580744e0 msgr2=0x7fe1580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.993+0000 7fe17be00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe1580744e0 0x7fe1580769f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe174007fd0 tx=0x7fe174007de0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.993+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c15ca70 msgr2=0x7fe17c14c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.993+0000 7fe17be00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c15ca70 0x7fe17c14c0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe17c06f410 tx=0x7fe16c0046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 shutdown_connections 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fe1580744e0 0x7fe1580769f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe17c15ca70 0x7fe17c14c0c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe17c1567b0 0x7fe17c0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 --2- 172.21.15.123:0/704257840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe17c0c82b0 0x7fe17c155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:37.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 >> 172.21.15.123:0/704257840 conn(0x7fe17c0cd600 msgr2=0x7fe17c0c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:37.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 shutdown_connections 2024-04-25T12:37:37.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:37.994+0000 7fe17be00640 1 -- 172.21.15.123:0/704257840 wait complete. 2024-04-25T12:37:38.020 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T12:37:38.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:38.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:37:38.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:37:38.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.298+0000 7f100ac00640 1 Processor -- start 2024-04-25T12:37:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.299+0000 7f100ac00640 1 -- start start 2024-04-25T12:37:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f100ac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1004070250 0x7f10041578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f100ac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10040c4cc0 0x7f10040c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f100ac00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1004074600 con 0x7f100406f7c0 2024-04-25T12:37:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f100ac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f10040c7270 con 0x7f10040c4cc0 2024-04-25T12:37:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f100ac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f10040c9fb0 con 0x7f1004070250 2024-04-25T12:37:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f1003400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10040c4cc0 0x7f10040c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f1003400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10040c4cc0 0x7f10040c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60054/0 (socket says 172.21.15.123:60054) 2024-04-25T12:37:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f1003400640 1 -- 172.21.15.123:0/711038382 learned_addr learned my addr 172.21.15.123:0/711038382 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f1008c00640 1 --1- 172.21.15.123:0/711038382 >> v1:172.21.15.123:6789/0 conn(0x7f100406f7c0 0x7f100415dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60922/0 (socket says 172.21.15.123:60922) 2024-04-25T12:37:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.301+0000 7f1003e00640 1 --2- 172.21.15.123:0/711038382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1004070250 0x7f10041578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.302+0000 7f1003400640 1 -- 172.21.15.123:0/711038382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1004070250 msgr2=0x7f10041578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.302+0000 7f1003400640 1 --2- 172.21.15.123:0/711038382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1004070250 0x7f10041578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.302+0000 7f1003400640 1 -- 172.21.15.123:0/711038382 >> v1:172.21.15.123:6789/0 conn(0x7f100406f7c0 legacy=0x7f100415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.302+0000 7f1003400640 1 -- 172.21.15.123:0/711038382 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1004157e00 con 0x7f10040c4cc0 2024-04-25T12:37:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.303+0000 7f1003400640 1 --2- 172.21.15.123:0/711038382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f10040c6690 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009870 tx=0x7f0fec02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfacbb940c2653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.303+0000 7f1002a00640 1 -- 172.21.15.123:0/711038382 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec037070 con 0x7f10040c4cc0 2024-04-25T12:37:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.303+0000 7f1002a00640 1 -- 172.21.15.123:0/711038382 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec02ae50 con 0x7f10040c4cc0 2024-04-25T12:37:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.303+0000 7f1002a00640 1 -- 172.21.15.123:0/711038382 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec032860 con 0x7f10040c4cc0 2024-04-25T12:37:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 -- 172.21.15.123:0/711038382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 msgr2=0x7f10040c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 --2- 172.21.15.123:0/711038382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f10040c6690 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009870 tx=0x7f0fec02a300 comp rx=0 tx=0).stop 2024-04-25T12:37:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 -- 172.21.15.123:0/711038382 shutdown_connections 2024-04-25T12:37:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 --2- 172.21.15.123:0/711038382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f10040c6690 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 --2- 172.21.15.123:0/711038382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1004070250 0x7f10041578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 -- 172.21.15.123:0/711038382 >> 172.21.15.123:0/711038382 conn(0x7f100406c570 msgr2=0x7f100406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 -- 172.21.15.123:0/711038382 shutdown_connections 2024-04-25T12:37:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.304+0000 7f100ac00640 1 -- 172.21.15.123:0/711038382 wait complete. 2024-04-25T12:37:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.307+0000 7f100ac00640 1 Processor -- start 2024-04-25T12:37:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.307+0000 7f100ac00640 1 -- start start 2024-04-25T12:37:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.308+0000 7f100ac00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f100406f7c0 0x7f10041f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.308+0000 7f100ac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004070250 0x7f1004151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.308+0000 7f100ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f1004152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.308+0000 7f100ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10040c9960 con 0x7f10040c4cc0 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.308+0000 7f100ac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1004074b50 con 0x7f1004070250 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.308+0000 7f100ac00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f10040c9c40 con 0x7f100406f7c0 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1008c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f1004152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1008c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f1004152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60068/0 (socket says 172.21.15.123:60068) 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1008c00640 1 -- 172.21.15.123:0/2508458244 learned_addr learned my addr 172.21.15.123:0/2508458244 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003400640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004070250 0x7f1004151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003e00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f100406f7c0 0x7f10041f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003400640 1 -- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f100406f7c0 msgr2=0x7f10041f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003400640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f100406f7c0 0x7f10041f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003400640 1 -- 172.21.15.123:0/2508458244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 msgr2=0x7f1004152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003400640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f1004152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003400640 1 -- 172.21.15.123:0/2508458244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10040b21f0 con 0x7f1004070250 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1003e00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f100406f7c0 0x7f10041f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.309+0000 7f1008c00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10040c4cc0 0x7f1004152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.310+0000 7f1003400640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004070250 0x7f1004151f40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0fec031000 tx=0x7f0fec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.310+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec041070 con 0x7f1004070250 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.310+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec03c680 con 0x7f1004070250 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.310+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1004157090 con 0x7f1004070250 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.310+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec037040 con 0x7f1004070250 2024-04-25T12:37:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.310+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10040b0920 con 0x7f1004070250 2024-04-25T12:37:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.311+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1004156b50 con 0x7f1004070250 2024-04-25T12:37:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.313+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0fec044070 con 0x7f1004070250 2024-04-25T12:37:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.314+0000 7f0ffbe00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fe00744b0 0x7f0fe00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.314+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f0fec066e00 con 0x7f1004070250 2024-04-25T12:37:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.314+0000 7f1003e00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fe00744b0 0x7f0fe00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.314+0000 7f0ffbe00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0fe007e1a0 0x7f0fe0080690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.314+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0fec041e70 con 0x7f0fe007e1a0 2024-04-25T12:37:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.314+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f0fec0b6440 con 0x7f1004070250 2024-04-25T12:37:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.314+0000 7f1008c00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0fe007e1a0 0x7f0fe0080690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.315+0000 7f1003e00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fe00744b0 0x7f0fe00769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009fd0 tx=0x7f0ff400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.315+0000 7f1008c00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0fe007e1a0 0x7f0fe0080690 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.316+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== osd.4 v2:172.21.15.153:6802/3135982807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0fec041e70 con 0x7f0fe007e1a0 2024-04-25T12:37:38.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.330+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 --> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1004076700 con 0x7f0fe007e1a0 2024-04-25T12:37:38.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.331+0000 7f0ffbe00640 1 -- 172.21.15.123:0/2508458244 <== osd.4 v2:172.21.15.153:6802/3135982807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1004076700 con 0x7f0fe007e1a0 2024-04-25T12:37:38.331 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:38.331 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:38.331 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0fe007e1a0 msgr2=0x7f0fe0080690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6802/3135982807,v1:172.21.15.153:6803/3135982807] conn(0x7f0fe007e1a0 0x7f0fe0080690 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fe00744b0 msgr2=0x7f0fe00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0fe00744b0 0x7f0fe00769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009fd0 tx=0x7f0ff400b040 comp rx=0 tx=0).stop 2024-04-25T12:37:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004070250 msgr2=0x7f1004151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 --2- 172.21.15.123:0/2508458244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004070250 0x7f1004151f40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0fec031000 tx=0x7f0fec0046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f1008c00640 1 -- 172.21.15.123:0/2508458244 reap_dead start 2024-04-25T12:37:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 shutdown_connections 2024-04-25T12:37:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 >> 172.21.15.123:0/2508458244 conn(0x7f100406c570 msgr2=0x7f1004072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.332+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 shutdown_connections 2024-04-25T12:37:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.333+0000 7f100ac00640 1 -- 172.21.15.123:0/2508458244 wait complete. 2024-04-25T12:37:38.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T12:37:38.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.636+0000 7f722a600640 1 Processor -- start 2024-04-25T12:37:38.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.636+0000 7f722a600640 1 -- start start 2024-04-25T12:37:38.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f722a600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f72240c9080 0x7f72241546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f722a600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f722406c570 0x7f722406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f722a600640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722415a330 con 0x7f72240c82b0 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f722a600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7224073b10 con 0x7f722406c570 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f722a600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7224073df0 con 0x7f72240c9080 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f7229200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f72240c9080 0x7f72241546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f7229c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f722406c570 0x7f722406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f721be00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f72240c82b0 0x7f722415e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34796/0 (socket says 172.21.15.123:34796) 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.638+0000 7f721be00640 1 -- 172.21.15.123:0/1122059714 learned_addr learned my addr 172.21.15.123:0/1122059714 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:38.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.639+0000 7f7229c00640 1 -- 172.21.15.123:0/1122059714 >> v2:172.21.15.153:3300/0 conn(0x7f72240c9080 msgr2=0x7f72241546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:38.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.639+0000 7f7229c00640 1 --2- 172.21.15.123:0/1122059714 >> v2:172.21.15.153:3300/0 conn(0x7f72240c9080 0x7f72241546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.639+0000 7f7229c00640 1 -- 172.21.15.123:0/1122059714 >> v1:172.21.15.153:6789/0 conn(0x7f72240c82b0 legacy=0x7f722415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.639+0000 7f7223e00640 1 -- 172.21.15.123:0/1122059714 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107214062 0 0) 0x7f722415a330 con 0x7f72240c82b0 2024-04-25T12:37:38.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.639+0000 7f7229c00640 1 -- 172.21.15.123:0/1122059714 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72240740d0 con 0x7f722406c570 2024-04-25T12:37:38.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.640+0000 7f7229c00640 1 --2- 172.21.15.123:0/1122059714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722406c980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f72240ce520 tx=0x7f721c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c79ecd13f829c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:38.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.640+0000 7f7223e00640 1 -- 172.21.15.123:0/1122059714 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c007740 con 0x7f722406c570 2024-04-25T12:37:38.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.640+0000 7f7223e00640 1 -- 172.21.15.123:0/1122059714 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f721c0078e0 con 0x7f722406c570 2024-04-25T12:37:38.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.640+0000 7f7223e00640 1 -- 172.21.15.123:0/1122059714 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c007cb0 con 0x7f722406c570 2024-04-25T12:37:38.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 -- 172.21.15.123:0/1122059714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 msgr2=0x7f722406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 --2- 172.21.15.123:0/1122059714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722406c980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f72240ce520 tx=0x7f721c02a5f0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 -- 172.21.15.123:0/1122059714 shutdown_connections 2024-04-25T12:37:38.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 --2- 172.21.15.123:0/1122059714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722406c980 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 --2- 172.21.15.123:0/1122059714 >> v2:172.21.15.153:3300/0 conn(0x7f72240c9080 0x7f72241546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 -- 172.21.15.123:0/1122059714 >> 172.21.15.123:0/1122059714 conn(0x7f72240cd600 msgr2=0x7f72240cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:38.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 -- 172.21.15.123:0/1122059714 shutdown_connections 2024-04-25T12:37:38.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.641+0000 7f722a600640 1 -- 172.21.15.123:0/1122059714 wait complete. 2024-04-25T12:37:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.643+0000 7f722a600640 1 Processor -- start 2024-04-25T12:37:38.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.644+0000 7f722a600640 1 -- start start 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f722a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f722a600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72240c82b0 0x7f72241f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f722a600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72240c9080 0x7f72240cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f722a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f722415e760 con 0x7f722406c570 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f722a600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f722415db30 con 0x7f72240c9080 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f722a600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7224073760 con 0x7f72240c82b0 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f7229200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f7229200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722415d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60086/0 (socket says 172.21.15.123:60086) 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f7229200640 1 -- 172.21.15.123:0/874838223 learned_addr learned my addr 172.21.15.123:0/874838223 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f721be00640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72240c82b0 0x7f72241f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.645+0000 7f7229c00640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72240c9080 0x7f72240cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7229200640 1 -- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72240c82b0 msgr2=0x7f72241f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7229200640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72240c82b0 0x7f72241f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7229200640 1 -- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72240c9080 msgr2=0x7f72240cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7229200640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72240c9080 0x7f72240cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7229200640 1 -- 172.21.15.123:0/874838223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72241f0590 con 0x7f722406c570 2024-04-25T12:37:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7229200640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722415d100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f721400e8a0 tx=0x7f721400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7221600640 1 -- 172.21.15.123:0/874838223 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214017070 con 0x7f722406c570 2024-04-25T12:37:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7221600640 1 -- 172.21.15.123:0/874838223 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f721400c940 con 0x7f722406c570 2024-04-25T12:37:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f7221600640 1 -- 172.21.15.123:0/874838223 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721400cd30 con 0x7f722406c570 2024-04-25T12:37:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.646+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72241f3560 con 0x7f722406c570 2024-04-25T12:37:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.647+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72241f3820 con 0x7f722406c570 2024-04-25T12:37:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.648+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7224076da0 con 0x7f722406c570 2024-04-25T12:37:38.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.649+0000 7f7221600640 1 -- 172.21.15.123:0/874838223 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f721401a070 con 0x7f722406c570 2024-04-25T12:37:38.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.650+0000 7f7221600640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f72000744b0 0x7f72000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.650+0000 7f7221600640 1 -- 172.21.15.123:0/874838223 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f7214095880 con 0x7f722406c570 2024-04-25T12:37:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.650+0000 7f721be00640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f72000744b0 0x7f72000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.651+0000 7f721be00640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f72000744b0 0x7f72000769c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f72240cd510 tx=0x7f720c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.655+0000 7f7221600640 1 -- 172.21.15.123:0/874838223 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f721405f550 con 0x7f722406c570 2024-04-25T12:37:38.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.856+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f72240c8e60 con 0x7f72000744b0 2024-04-25T12:37:38.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.857+0000 7f7221600640 1 -- 172.21.15.123:0/874838223 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f72240c8e60 con 0x7f72000744b0 2024-04-25T12:37:38.857 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f72000744b0 msgr2=0x7f72000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f72000744b0 0x7f72000769c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f72240cd510 tx=0x7f720c009040 comp rx=0 tx=0).stop 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 msgr2=0x7f722415d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722415d100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f721400e8a0 tx=0x7f721400ed60 comp rx=0 tx=0).stop 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 shutdown_connections 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f72000744b0 0x7f72000769c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72240c9080 0x7f72240cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72240c82b0 0x7f72241f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 --2- 172.21.15.123:0/874838223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722406c570 0x7f722415d100 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:38.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.859+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 >> 172.21.15.123:0/874838223 conn(0x7f72240cd600 msgr2=0x7f72240c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:38.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.860+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 shutdown_connections 2024-04-25T12:37:38.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:38.860+0000 7f722a600640 1 -- 172.21.15.123:0/874838223 wait complete. 2024-04-25T12:37:38.888 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T12:37:38.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:39.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.168+0000 7f8f5b600640 1 Processor -- start 2024-04-25T12:37:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.168+0000 7f8f5b600640 1 -- start start 2024-04-25T12:37:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f5b600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c4400 0x7f8f540b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f5b600640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8f540c92e0 0x7f8f54160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f5b600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f540c2870 0x7f8f5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f5b600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8f540c96f0 con 0x7f8f540c2870 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f5b600640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8f540c8b70 con 0x7f8f540c92e0 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f5b600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8f540c4020 con 0x7f8f540c4400 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f59600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f540c2870 0x7f8f5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f58c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c4400 0x7f8f540b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f59600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f540c2870 0x7f8f5406edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60094/0 (socket says 172.21.15.123:60094) 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f58c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c4400 0x7f8f540b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:44972/0 (socket says 172.21.15.123:44972) 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f59600640 1 -- 172.21.15.123:0/3232011856 learned_addr learned my addr 172.21.15.123:0/3232011856 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.170+0000 7f8f53e00640 1 --2- 172.21.15.123:0/3232011856 >> v2:172.21.15.153:3300/0 conn(0x7f8f540c92e0 0x7f8f54160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.171+0000 7f8f58c00640 1 -- 172.21.15.123:0/3232011856 >> v2:172.21.15.153:3300/0 conn(0x7f8f540c92e0 msgr2=0x7f8f54160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.171+0000 7f8f58c00640 1 --2- 172.21.15.123:0/3232011856 >> v2:172.21.15.153:3300/0 conn(0x7f8f540c92e0 0x7f8f54160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.171+0000 7f8f58c00640 1 -- 172.21.15.123:0/3232011856 >> v2:172.21.15.123:3300/0 conn(0x7f8f540c2870 msgr2=0x7f8f5406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.171+0000 7f8f58c00640 1 --2- 172.21.15.123:0/3232011856 >> v2:172.21.15.123:3300/0 conn(0x7f8f540c2870 0x7f8f5406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.171+0000 7f8f58c00640 1 -- 172.21.15.123:0/3232011856 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f540c48b0 con 0x7f8f540c4400 2024-04-25T12:37:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.172+0000 7f8f58c00640 1 --2- 172.21.15.123:0/3232011856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c4400 0x7f8f540b3680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f54072cb0 tx=0x7f8f4402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8721b67950210021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.172+0000 7f8f53400640 1 -- 172.21.15.123:0/3232011856 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44037070 con 0x7f8f540c4400 2024-04-25T12:37:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.172+0000 7f8f53400640 1 -- 172.21.15.123:0/3232011856 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4402ae50 con 0x7f8f540c4400 2024-04-25T12:37:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.172+0000 7f8f53400640 1 -- 172.21.15.123:0/3232011856 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44032660 con 0x7f8f540c4400 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.173+0000 7f8f5b600640 1 -- 172.21.15.123:0/3232011856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c4400 msgr2=0x7f8f540b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.173+0000 7f8f5b600640 1 --2- 172.21.15.123:0/3232011856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c4400 0x7f8f540b3680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f54072cb0 tx=0x7f8f4402a100 comp rx=0 tx=0).stop 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.173+0000 7f8f5b600640 1 -- 172.21.15.123:0/3232011856 shutdown_connections 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.173+0000 7f8f5b600640 1 --2- 172.21.15.123:0/3232011856 >> v2:172.21.15.123:3300/0 conn(0x7f8f540c2870 0x7f8f5406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.173+0000 7f8f5b600640 1 --2- 172.21.15.123:0/3232011856 >> v2:172.21.15.153:3300/0 conn(0x7f8f540c92e0 0x7f8f54160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.173+0000 7f8f5b600640 1 --2- 172.21.15.123:0/3232011856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c4400 0x7f8f540b3680 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.173+0000 7f8f5b600640 1 -- 172.21.15.123:0/3232011856 >> 172.21.15.123:0/3232011856 conn(0x7f8f5406c570 msgr2=0x7f8f5406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.174+0000 7f8f5b600640 1 -- 172.21.15.123:0/3232011856 shutdown_connections 2024-04-25T12:37:39.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.174+0000 7f8f5b600640 1 -- 172.21.15.123:0/3232011856 wait complete. 2024-04-25T12:37:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.177+0000 7f8f5b600640 1 Processor -- start 2024-04-25T12:37:39.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.177+0000 7f8f5b600640 1 -- start start 2024-04-25T12:37:39.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f5b600640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c2870 0x7f8f54157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f5b600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f540c4400 0x7f8f540c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f5b600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f540c92e0 0x7f8f5414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f5b600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f540c8780 con 0x7f8f540c92e0 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f5b600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f540c84a0 con 0x7f8f540c4400 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f5b600640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8f540c8dc0 con 0x7f8f540c2870 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f53e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f540c4400 0x7f8f540c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f58c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c2870 0x7f8f54157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f58c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c2870 0x7f8f54157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.123:44988/0 (socket says 172.21.15.123:44988) 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f58c00640 1 -- 172.21.15.123:0/442802469 learned_addr learned my addr 172.21.15.123:0/442802469 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.179+0000 7f8f59600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f540c92e0 0x7f8f5414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f58c00640 1 -- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f540c4400 msgr2=0x7f8f540c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f58c00640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f540c4400 0x7f8f540c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f58c00640 1 -- 172.21.15.123:0/442802469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f540c92e0 msgr2=0x7f8f5414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f58c00640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f540c92e0 0x7f8f5414e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f58c00640 1 -- 172.21.15.123:0/442802469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f540c5f60 con 0x7f8f540c2870 2024-04-25T12:37:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f58c00640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c2870 0x7f8f54157720 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8f44000c00 tx=0x7f8f440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44042070 con 0x7f8f540c2870 2024-04-25T12:37:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4403e460 con 0x7f8f540c2870 2024-04-25T12:37:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44037040 con 0x7f8f540c2870 2024-04-25T12:37:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.180+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f540c7060 con 0x7f8f540c2870 2024-04-25T12:37:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.181+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f540c4f40 con 0x7f8f540c2870 2024-04-25T12:37:39.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.181+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f540cb150 con 0x7f8f540c2870 2024-04-25T12:37:39.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.183+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f44045070 con 0x7f8f540c2870 2024-04-25T12:37:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.184+0000 7f8f50c00640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f300744b0 0x7f8f300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.184+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f8f440b6500 con 0x7f8f540c2870 2024-04-25T12:37:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.184+0000 7f8f53e00640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f300744b0 0x7f8f300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.184+0000 7f8f50c00640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8f3007e190 0x7f8f30080680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.184+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f30080d50 con 0x7f8f3007e190 2024-04-25T12:37:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.184+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f8f440b68b0 con 0x7f8f540c2870 2024-04-25T12:37:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.185+0000 7f8f59600640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8f3007e190 0x7f8f30080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.185+0000 7f8f53e00640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f300744b0 0x7f8f300769c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8f541f84a0 tx=0x7f8f3c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.185+0000 7f8f59600640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8f3007e190 0x7f8f30080680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:39.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.186+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== osd.5 v2:172.21.15.153:6826/4270711740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f30080d50 con 0x7f8f3007e190 2024-04-25T12:37:39.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.201+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 --> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8f5415ebb0 con 0x7f8f3007e190 2024-04-25T12:37:39.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.201+0000 7f8f50c00640 1 -- 172.21.15.123:0/442802469 <== osd.5 v2:172.21.15.153:6826/4270711740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8f5415ebb0 con 0x7f8f3007e190 2024-04-25T12:37:39.201 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:39.201 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:39.202 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8f3007e190 msgr2=0x7f8f30080680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6826/4270711740,v1:172.21.15.153:6827/4270711740] conn(0x7f8f3007e190 0x7f8f30080680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f300744b0 msgr2=0x7f8f300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f8f300744b0 0x7f8f300769c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8f541f84a0 tx=0x7f8f3c00b040 comp rx=0 tx=0).stop 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c2870 msgr2=0x7f8f54157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 --2- 172.21.15.123:0/442802469 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f540c2870 0x7f8f54157720 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8f44000c00 tx=0x7f8f440046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f59600640 1 -- 172.21.15.123:0/442802469 reap_dead start 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 shutdown_connections 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.202+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 >> 172.21.15.123:0/442802469 conn(0x7f8f5406c570 msgr2=0x7f8f54072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:39.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.203+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 shutdown_connections 2024-04-25T12:37:39.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.203+0000 7f8f5b600640 1 -- 172.21.15.123:0/442802469 wait complete. 2024-04-25T12:37:39.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T12:37:39.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.506+0000 7fd3c9400640 1 Processor -- start 2024-04-25T12:37:39.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.506+0000 7fd3c9400640 1 -- start start 2024-04-25T12:37:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c9400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd3c40cb990 0x7fd3c415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c9400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c41600e0 0x7fd3c4166d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c9400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c406ecc0 con 0x7fd3c40c6450 2024-04-25T12:37:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c9400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c415d220 con 0x7fd3c40cb990 2024-04-25T12:37:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c9400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c415c610 con 0x7fd3c41600e0 2024-04-25T12:37:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c8a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c41600e0 0x7fd3c4166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c3e00640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd3c40c6450 0x7fd3c40cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:34800/0 (socket says 172.21.15.123:34800) 2024-04-25T12:37:39.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c3e00640 1 -- 172.21.15.123:0/3353884459 learned_addr learned my addr 172.21.15.123:0/3353884459 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:39.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.509+0000 7fd3c3400640 1 --2- 172.21.15.123:0/3353884459 >> v2:172.21.15.153:3300/0 conn(0x7fd3c40cb990 0x7fd3c415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.510+0000 7fd3c3400640 1 -- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c41600e0 msgr2=0x7fd3c4166d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:39.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.510+0000 7fd3c3400640 1 --2- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c41600e0 0x7fd3c4166d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.510+0000 7fd3c3400640 1 -- 172.21.15.123:0/3353884459 >> v1:172.21.15.153:6789/0 conn(0x7fd3c40c6450 legacy=0x7fd3c40cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.510+0000 7fd3c3400640 1 -- 172.21.15.123:0/3353884459 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c415c8f0 con 0x7fd3c40cb990 2024-04-25T12:37:39.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.510+0000 7fd3c8a00640 1 --2- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c41600e0 0x7fd3c4166d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:39.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.511+0000 7fd3c3400640 1 --2- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40cb990 0x7fd3c415fba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00097a0 tx=0x7fd3b002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d98a33ee9a92fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:39.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.514+0000 7fd3c2a00640 1 -- 172.21.15.123:0/3353884459 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0037070 con 0x7fd3c40cb990 2024-04-25T12:37:39.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.514+0000 7fd3c2a00640 1 -- 172.21.15.123:0/3353884459 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0032440 con 0x7fd3c40cb990 2024-04-25T12:37:39.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.514+0000 7fd3c2a00640 1 -- 172.21.15.123:0/3353884459 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0032810 con 0x7fd3c40cb990 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.515+0000 7fd3c9400640 1 -- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40cb990 msgr2=0x7fd3c415fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.515+0000 7fd3c9400640 1 --2- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40cb990 0x7fd3c415fba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00097a0 tx=0x7fd3b002a180 comp rx=0 tx=0).stop 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.516+0000 7fd3c9400640 1 -- 172.21.15.123:0/3353884459 shutdown_connections 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.516+0000 7fd3c9400640 1 --2- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c41600e0 0x7fd3c4166d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.516+0000 7fd3c9400640 1 --2- 172.21.15.123:0/3353884459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c40cb990 0x7fd3c415fba0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.516+0000 7fd3c9400640 1 -- 172.21.15.123:0/3353884459 >> 172.21.15.123:0/3353884459 conn(0x7fd3c4072cb0 msgr2=0x7fd3c40730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.516+0000 7fd3c9400640 1 -- 172.21.15.123:0/3353884459 shutdown_connections 2024-04-25T12:37:39.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.516+0000 7fd3c9400640 1 -- 172.21.15.123:0/3353884459 wait complete. 2024-04-25T12:37:39.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.518+0000 7fd3c9400640 1 Processor -- start 2024-04-25T12:37:39.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.518+0000 7fd3c9400640 1 -- start start 2024-04-25T12:37:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c40c6450 0x7fd3c4151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c9400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40cb990 0x7fd3c414a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c41600e0 0x7fd3c40b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c9400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c41604f0 con 0x7fd3c40c6450 2024-04-25T12:37:39.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c9400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c415ceb0 con 0x7fd3c41600e0 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c9400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c40ca2b0 con 0x7fd3c40cb990 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c40c6450 0x7fd3c4151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c40c6450 0x7fd3c4151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60112/0 (socket says 172.21.15.123:60112) 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c3e00640 1 -- 172.21.15.123:0/2534773406 learned_addr learned my addr 172.21.15.123:0/2534773406 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c3400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40cb990 0x7fd3c414a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.520+0000 7fd3c8a00640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c41600e0 0x7fd3c40b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c3e00640 1 -- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40cb990 msgr2=0x7fd3c414a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40cb990 0x7fd3c414a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c3e00640 1 -- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c41600e0 msgr2=0x7fd3c40b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c41600e0 0x7fd3c40b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c3e00640 1 -- 172.21.15.123:0/2534773406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c4151870 con 0x7fd3c40c6450 2024-04-25T12:37:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c3400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40cb990 0x7fd3c414a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:39.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c8a00640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c41600e0 0x7fd3c40b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c3e00640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c40c6450 0x7fd3c4151330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400c3f0 tx=0x7fd3b400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3bbe00640 1 -- 172.21.15.123:0/2534773406 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4017070 con 0x7fd3c40c6450 2024-04-25T12:37:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3bbe00640 1 -- 172.21.15.123:0/2534773406 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b4005ce0 con 0x7fd3c40c6450 2024-04-25T12:37:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3bbe00640 1 -- 172.21.15.123:0/2534773406 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40072f0 con 0x7fd3c40c6450 2024-04-25T12:37:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.521+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c415b780 con 0x7fd3c40c6450 2024-04-25T12:37:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.522+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c415bd40 con 0x7fd3c40c6450 2024-04-25T12:37:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.523+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c4076da0 con 0x7fd3c40c6450 2024-04-25T12:37:39.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.524+0000 7fd3bbe00640 1 -- 172.21.15.123:0/2534773406 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3b401a070 con 0x7fd3c40c6450 2024-04-25T12:37:39.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.525+0000 7fd3bbe00640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a40744b0 0x7fd3a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:39.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.525+0000 7fd3bbe00640 1 -- 172.21.15.123:0/2534773406 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fd3b40948e0 con 0x7fd3c40c6450 2024-04-25T12:37:39.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.525+0000 7fd3c3400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a40744b0 0x7fd3a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:39.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.526+0000 7fd3c3400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a40744b0 0x7fd3a40769c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0034000 tx=0x7fd3b002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:39.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.530+0000 7fd3bbe00640 1 -- 172.21.15.123:0/2534773406 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3b405e570 con 0x7fd3c40c6450 2024-04-25T12:37:39.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.730+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fd3c415b530 con 0x7fd3a40744b0 2024-04-25T12:37:39.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.731+0000 7fd3bbe00640 1 -- 172.21.15.123:0/2534773406 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd3c415b530 con 0x7fd3a40744b0 2024-04-25T12:37:39.731 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.733+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a40744b0 msgr2=0x7fd3a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.733+0000 7fd3c9400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a40744b0 0x7fd3a40769c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0034000 tx=0x7fd3b002e000 comp rx=0 tx=0).stop 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.733+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c40c6450 msgr2=0x7fd3c4151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.733+0000 7fd3c9400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c40c6450 0x7fd3c4151330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400c3f0 tx=0x7fd3b400c7f0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 shutdown_connections 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd3a40744b0 0x7fd3a40769c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3c41600e0 0x7fd3c40b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd3c40cb990 0x7fd3c414a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 --2- 172.21.15.123:0/2534773406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c40c6450 0x7fd3c4151330 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:39.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 >> 172.21.15.123:0/2534773406 conn(0x7fd3c4072cb0 msgr2=0x7fd3c4153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:39.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 shutdown_connections 2024-04-25T12:37:39.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:39.734+0000 7fd3c9400640 1 -- 172.21.15.123:0/2534773406 wait complete. 2024-04-25T12:37:39.759 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T12:37:39.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:40.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.034+0000 7f0f5be00640 1 Processor -- start 2024-04-25T12:37:40.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.035+0000 7f0f5be00640 1 -- start start 2024-04-25T12:37:40.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.036+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.036+0000 7f0f5be00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0f5c0c93c0 0x7f0f5c0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.036+0000 7f0f5be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f5c159110 0x7f0f5c159520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.036+0000 7f0f5be00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5c15c700 con 0x7f0f5c159110 2024-04-25T12:37:40.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f5be00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5c159a60 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f5be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c0c9040 con 0x7f0f5c0c85b0 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f53e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0f5c0c93c0 0x7f0f5c0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f5b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f5c159110 0x7f0f5c159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f5aa00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f53e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0f5c0c93c0 0x7f0f5c0c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41846/0 (socket says 172.21.15.123:41846) 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f5b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f5c159110 0x7f0f5c159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60120/0 (socket says 172.21.15.123:60120) 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f53e00640 1 -- 172.21.15.123:0/3008002946 learned_addr learned my addr 172.21.15.123:0/3008002946 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f53e00640 1 -- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 msgr2=0x7f0f5c0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f53e00640 1 --2- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c0c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f53e00640 1 -- 172.21.15.123:0/3008002946 >> v2:172.21.15.123:3300/0 conn(0x7f0f5c159110 msgr2=0x7f0f5c159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.037+0000 7f0f53e00640 1 --2- 172.21.15.123:0/3008002946 >> v2:172.21.15.123:3300/0 conn(0x7f0f5c159110 0x7f0f5c159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.038+0000 7f0f53e00640 1 -- 172.21.15.123:0/3008002946 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c15c950 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.038+0000 7f0f5aa00640 1 --2- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c0c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.039+0000 7f0f53e00640 1 --2- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 0x7f0f5c0c97d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c06fcf0 tx=0x7f0f4402c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe35a3aaee6c604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.039+0000 7f0f5a000640 1 -- 172.21.15.123:0/3008002946 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44036020 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.039+0000 7f0f5a000640 1 -- 172.21.15.123:0/3008002946 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f44032820 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.039+0000 7f0f5a000640 1 -- 172.21.15.123:0/3008002946 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44032bf0 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 -- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 msgr2=0x7f0f5c0c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 --2- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 0x7f0f5c0c97d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c06fcf0 tx=0x7f0f4402c920 comp rx=0 tx=0).stop 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 -- 172.21.15.123:0/3008002946 shutdown_connections 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 --2- 172.21.15.123:0/3008002946 >> v2:172.21.15.123:3300/0 conn(0x7f0f5c159110 0x7f0f5c159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 --2- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 0x7f0f5c0c97d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 --2- 172.21.15.123:0/3008002946 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c0c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 -- 172.21.15.123:0/3008002946 >> 172.21.15.123:0/3008002946 conn(0x7f0f5c0c9f60 msgr2=0x7f0f5c0ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:40.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 -- 172.21.15.123:0/3008002946 shutdown_connections 2024-04-25T12:37:40.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.040+0000 7f0f5be00640 1 -- 172.21.15.123:0/3008002946 wait complete. 2024-04-25T12:37:40.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.042+0000 7f0f5be00640 1 Processor -- start 2024-04-25T12:37:40.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.042+0000 7f0f5be00640 1 -- start start 2024-04-25T12:37:40.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 0x7f0f5c0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5c159110 0x7f0f5c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c15d1f0 con 0x7f0f5c159110 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c15cf10 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5be00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c0c80d0 con 0x7f0f5c0c85b0 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5c159110 0x7f0f5c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5c159110 0x7f0f5c0cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60128/0 (socket says 172.21.15.123:60128) 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5b400640 1 -- 172.21.15.123:0/2297345195 learned_addr learned my addr 172.21.15.123:0/2297345195 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 0x7f0f5c0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.044+0000 7f0f5aa00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f53e00640 1 -- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 msgr2=0x7f0f5c148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f53e00640 1 -- 172.21.15.123:0/2297345195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5c159110 msgr2=0x7f0f5c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5c159110 0x7f0f5c0cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f53e00640 1 -- 172.21.15.123:0/2297345195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c1f2ba0 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f5aa00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0f5c0c85b0 0x7f0f5c148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:40.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f5b400640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f5c159110 0x7f0f5c0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 0x7f0f5c0cc050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f44034000 tx=0x7f0f44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44042070 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f44032e20 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c1f2e10 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.045+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44036040 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.046+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c1f38f0 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.046+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f5c1f34d0 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.048+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f44045070 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.049+0000 7f0f52a00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f380744e0 0x7f0f380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.049+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f0f440b65e0 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.049+0000 7f0f52a00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0f3807e1c0 0x7f0f380806b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.049+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f38080d80 con 0x7f0f3807e1c0 2024-04-25T12:37:40.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.049+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f0f440b6990 con 0x7f0f5c0c93c0 2024-04-25T12:37:40.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.049+0000 7f0f5aa00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f380744e0 0x7f0f380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.049+0000 7f0f5b400640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0f3807e1c0 0x7f0f380806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.050+0000 7f0f5aa00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f380744e0 0x7f0f380769f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009580 tx=0x7f0f4c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.050+0000 7f0f5b400640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0f3807e1c0 0x7f0f380806b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.051+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== osd.6 v2:172.21.15.153:6810/3626282289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f38080d80 con 0x7f0f3807e1c0 2024-04-25T12:37:40.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.065+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 --> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0f5c15a780 con 0x7f0f3807e1c0 2024-04-25T12:37:40.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f52a00640 1 -- 172.21.15.123:0/2297345195 <== osd.6 v2:172.21.15.153:6810/3626282289 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0f5c15a780 con 0x7f0f3807e1c0 2024-04-25T12:37:40.066 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:40.066 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:40.066 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:40.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0f3807e1c0 msgr2=0x7f0f380806b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f5be00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6810/3626282289,v1:172.21.15.153:6811/3626282289] conn(0x7f0f3807e1c0 0x7f0f380806b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f380744e0 msgr2=0x7f0f380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f5be00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0f380744e0 0x7f0f380769f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009580 tx=0x7f0f4c009190 comp rx=0 tx=0).stop 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 msgr2=0x7f0f5c0cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f5be00640 1 --2- 172.21.15.123:0/2297345195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f5c0c93c0 0x7f0f5c0cc050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f44034000 tx=0x7f0f44004730 comp rx=0 tx=0).stop 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.066+0000 7f0f5b400640 1 -- 172.21.15.123:0/2297345195 reap_dead start 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.067+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 shutdown_connections 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.067+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 >> 172.21.15.123:0/2297345195 conn(0x7f0f5c0c9f60 msgr2=0x7f0f5c0ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.067+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 shutdown_connections 2024-04-25T12:37:40.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.067+0000 7f0f5be00640 1 -- 172.21.15.123:0/2297345195 wait complete. 2024-04-25T12:37:40.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T12:37:40.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.364+0000 7fd196e00640 1 Processor -- start 2024-04-25T12:37:40.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.365+0000 7fd196e00640 1 -- start start 2024-04-25T12:37:40.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.366+0000 7fd196e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd1900c8250 0x7fd1900c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.366+0000 7fd196e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd190156e60 con 0x7fd190159820 2024-04-25T12:37:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.366+0000 7fd196e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1900c7130 con 0x7fd1900c9b80 2024-04-25T12:37:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.366+0000 7fd196e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd1900749c0 con 0x7fd1900c8250 2024-04-25T12:37:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd194e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd190159820 0x7fd190159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60930/0 (socket says 172.21.15.123:60930) 2024-04-25T12:37:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd194e00640 1 -- 172.21.15.123:0/2596140314 learned_addr learned my addr 172.21.15.123:0/2596140314 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd187e00640 1 --2- 172.21.15.123:0/2596140314 >> v2:172.21.15.153:3300/0 conn(0x7fd1900c8250 0x7fd1900c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd18f400640 1 -- 172.21.15.123:0/2596140314 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162182506 0 0) 0x7fd1900c7130 con 0x7fd1900c9b80 2024-04-25T12:37:40.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd18f400640 1 -- 172.21.15.123:0/2596140314 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd174003520 con 0x7fd1900c9b80 2024-04-25T12:37:40.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd18f400640 1 -- 172.21.15.123:0/2596140314 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 590322305 0 0) 0x7fd190156e60 con 0x7fd190159820 2024-04-25T12:37:40.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd187e00640 1 -- 172.21.15.123:0/2596140314 >> v1:172.21.15.153:6789/0 conn(0x7fd1900c9b80 legacy=0x7fd1900742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd187e00640 1 -- 172.21.15.123:0/2596140314 >> v1:172.21.15.123:6789/0 conn(0x7fd190159820 legacy=0x7fd190159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.367+0000 7fd187e00640 1 -- 172.21.15.123:0/2596140314 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19015ca70 con 0x7fd1900c8250 2024-04-25T12:37:40.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.368+0000 7fd187e00640 1 --2- 172.21.15.123:0/2596140314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1900c8250 0x7fd1900c8660 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1780097a0 tx=0x7fd17802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36eced859e08b4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.369+0000 7fd18f400640 1 -- 172.21.15.123:0/2596140314 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178037070 con 0x7fd1900c8250 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.369+0000 7fd18f400640 1 -- 172.21.15.123:0/2596140314 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd178032440 con 0x7fd1900c8250 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.369+0000 7fd18f400640 1 -- 172.21.15.123:0/2596140314 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178032810 con 0x7fd1900c8250 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.369+0000 7fd196e00640 1 -- 172.21.15.123:0/2596140314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1900c8250 msgr2=0x7fd1900c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.369+0000 7fd196e00640 1 --2- 172.21.15.123:0/2596140314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1900c8250 0x7fd1900c8660 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1780097a0 tx=0x7fd17802a230 comp rx=0 tx=0).stop 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.370+0000 7fd196e00640 1 -- 172.21.15.123:0/2596140314 shutdown_connections 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.370+0000 7fd196e00640 1 --2- 172.21.15.123:0/2596140314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1900c8250 0x7fd1900c8660 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.370+0000 7fd196e00640 1 -- 172.21.15.123:0/2596140314 >> 172.21.15.123:0/2596140314 conn(0x7fd19006c750 msgr2=0x7fd19006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.370+0000 7fd196e00640 1 -- 172.21.15.123:0/2596140314 shutdown_connections 2024-04-25T12:37:40.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.370+0000 7fd196e00640 1 -- 172.21.15.123:0/2596140314 wait complete. 2024-04-25T12:37:40.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.372+0000 7fd196e00640 1 Processor -- start 2024-04-25T12:37:40.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.372+0000 7fd196e00640 1 -- start start 2024-04-25T12:37:40.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd196e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1900c8250 0x7fd1901f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd196e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1900c9b80 0x7fd1900b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd196e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190159820 0x7fd190158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd196e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd19015d030 con 0x7fd1900c9b80 2024-04-25T12:37:40.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd196e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1900c8ba0 con 0x7fd190159820 2024-04-25T12:37:40.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd196e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd19015d310 con 0x7fd1900c8250 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1900c9b80 0x7fd1900b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1900c9b80 0x7fd1900b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60142/0 (socket says 172.21.15.123:60142) 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 -- 172.21.15.123:0/240144911 learned_addr learned my addr 172.21.15.123:0/240144911 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd194e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190159820 0x7fd190158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd18fe00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1900c8250 0x7fd1901f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 -- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1900c8250 msgr2=0x7fd1901f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1900c8250 0x7fd1901f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 -- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190159820 msgr2=0x7fd190158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190159820 0x7fd190158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.374+0000 7fd187e00640 1 -- 172.21.15.123:0/240144911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1901f6ff0 con 0x7fd1900c9b80 2024-04-25T12:37:40.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.375+0000 7fd187e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1900c9b80 0x7fd1900b1510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd178035040 tx=0x7fd1780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.375+0000 7fd18cc00640 1 -- 172.21.15.123:0/240144911 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178041020 con 0x7fd1900c9b80 2024-04-25T12:37:40.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.375+0000 7fd18cc00640 1 -- 172.21.15.123:0/240144911 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17803d6e0 con 0x7fd1900c9b80 2024-04-25T12:37:40.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.375+0000 7fd18cc00640 1 -- 172.21.15.123:0/240144911 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178037040 con 0x7fd1900c9b80 2024-04-25T12:37:40.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.375+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19014e190 con 0x7fd1900c9b80 2024-04-25T12:37:40.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.375+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1901f35f0 con 0x7fd1900c9b80 2024-04-25T12:37:40.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.377+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1900c5550 con 0x7fd1900c9b80 2024-04-25T12:37:40.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.378+0000 7fd18cc00640 1 -- 172.21.15.123:0/240144911 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd178044020 con 0x7fd1900c9b80 2024-04-25T12:37:40.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.378+0000 7fd18cc00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd16c0744b0 0x7fd16c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.379+0000 7fd18cc00640 1 -- 172.21.15.123:0/240144911 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7fd1780b5c40 con 0x7fd1900c9b80 2024-04-25T12:37:40.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.379+0000 7fd18fe00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd16c0744b0 0x7fd16c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.380+0000 7fd18fe00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd16c0744b0 0x7fd16c0769c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd180007fd0 tx=0x7fd180009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.384+0000 7fd18cc00640 1 -- 172.21.15.123:0/240144911 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1900c5550 con 0x7fd1900c9b80 2024-04-25T12:37:40.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.586+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd190076700 con 0x7fd16c0744b0 2024-04-25T12:37:40.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.587+0000 7fd18cc00640 1 -- 172.21.15.123:0/240144911 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd190076700 con 0x7fd16c0744b0 2024-04-25T12:37:40.587 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.590+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd16c0744b0 msgr2=0x7fd16c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.590+0000 7fd196e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd16c0744b0 0x7fd16c0769c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd180007fd0 tx=0x7fd180009040 comp rx=0 tx=0).stop 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.590+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1900c9b80 msgr2=0x7fd1900b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.590+0000 7fd196e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1900c9b80 0x7fd1900b1510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd178035040 tx=0x7fd1780046c0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.590+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 shutdown_connections 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.591+0000 7fd196e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7fd16c0744b0 0x7fd16c0769c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.591+0000 7fd196e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190159820 0x7fd190158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.591+0000 7fd196e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1900c9b80 0x7fd1900b1510 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.591+0000 7fd196e00640 1 --2- 172.21.15.123:0/240144911 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1900c8250 0x7fd1901f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.591+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 >> 172.21.15.123:0/240144911 conn(0x7fd19006c750 msgr2=0x7fd1900707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:40.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.591+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 shutdown_connections 2024-04-25T12:37:40.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.591+0000 7fd196e00640 1 -- 172.21.15.123:0/240144911 wait complete. 2024-04-25T12:37:40.616 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T12:37:40.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T12:37:40.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.895+0000 7f736de00640 1 Processor -- start 2024-04-25T12:37:40.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.895+0000 7f736de00640 1 -- start start 2024-04-25T12:37:40.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.897+0000 7f736de00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f73681578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.897+0000 7f736de00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f736806f7c0 0x7f736815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.897+0000 7f736de00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7368074600 con 0x7f73680c4cc0 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.897+0000 7f736de00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f73680c7270 con 0x7f736806f7c0 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.897+0000 7f736de00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f73680c9fb0 con 0x7f7368070250 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f735fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f73680c4cc0 0x7f73680c6690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60934/0 (socket says 172.21.15.123:60934) 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f735fe00640 1 -- 172.21.15.123:0/3728014246 learned_addr learned my addr 172.21.15.123:0/3728014246 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7366a00640 1 --2- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f73681578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7367400640 1 --2- 172.21.15.123:0/3728014246 >> v2:172.21.15.153:3300/0 conn(0x7f736806f7c0 0x7f736815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7366000640 1 -- 172.21.15.123:0/3728014246 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606958931 0 0) 0x7f7368074600 con 0x7f73680c4cc0 2024-04-25T12:37:40.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7366000640 1 -- 172.21.15.123:0/3728014246 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f734c003520 con 0x7f73680c4cc0 2024-04-25T12:37:40.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7367400640 1 -- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 msgr2=0x7f73681578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:40.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7367400640 1 --2- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f73681578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7367400640 1 -- 172.21.15.123:0/3728014246 >> v1:172.21.15.123:6789/0 conn(0x7f73680c4cc0 legacy=0x7f73680c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7367400640 1 -- 172.21.15.123:0/3728014246 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7368157e00 con 0x7f736806f7c0 2024-04-25T12:37:40.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.898+0000 7f7366a00640 1 --2- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f73681578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:37:40.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.900+0000 7f7367400640 1 --2- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f736815dbc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7360009d90 tx=0x7f736002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb6f5f64cc35d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.900+0000 7f7366000640 1 -- 172.21.15.123:0/3728014246 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360007740 con 0x7f736806f7c0 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.900+0000 7f7366000640 1 -- 172.21.15.123:0/3728014246 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73600078e0 con 0x7f736806f7c0 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.900+0000 7f7366000640 1 -- 172.21.15.123:0/3728014246 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360007cb0 con 0x7f736806f7c0 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 -- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 msgr2=0x7f736815dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 --2- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f736815dbc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7360009d90 tx=0x7f736002a6f0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 -- 172.21.15.123:0/3728014246 shutdown_connections 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 --2- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f736815dbc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 --2- 172.21.15.123:0/3728014246 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f73681578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 -- 172.21.15.123:0/3728014246 >> 172.21.15.123:0/3728014246 conn(0x7f736806c570 msgr2=0x7f736806c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 -- 172.21.15.123:0/3728014246 shutdown_connections 2024-04-25T12:37:40.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.901+0000 7f736de00640 1 -- 172.21.15.123:0/3728014246 wait complete. 2024-04-25T12:37:40.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.904+0000 7f736de00640 1 Processor -- start 2024-04-25T12:37:40.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.904+0000 7f736de00640 1 -- start start 2024-04-25T12:37:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f736de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f73680cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f736de00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f736813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f736de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73680c4cc0 0x7f73680cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f736de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73680c9960 con 0x7f73680c4cc0 2024-04-25T12:37:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f736de00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7368074b50 con 0x7f736806f7c0 2024-04-25T12:37:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f736de00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f73680c9c40 con 0x7f7368070250 2024-04-25T12:37:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f7366a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f73680cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f735fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f736813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f7366a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f73680cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:41902/0 (socket says 172.21.15.123:41902) 2024-04-25T12:37:40.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f7366a00640 1 -- 172.21.15.123:0/1018084582 learned_addr learned my addr 172.21.15.123:0/1018084582 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:40.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.906+0000 7f7367400640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73680c4cc0 0x7f73680cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f7366a00640 1 -- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 msgr2=0x7f736813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f7366a00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7368070250 0x7f736813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f7366a00640 1 -- 172.21.15.123:0/1018084582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73680c4cc0 msgr2=0x7f73680cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f7366a00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73680c4cc0 0x7f73680cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f7366a00640 1 -- 172.21.15.123:0/1018084582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73680cc210 con 0x7f736806f7c0 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f7366a00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f73680cbcd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f735800e940 tx=0x7f735800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7358017070 con 0x7f736806f7c0 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735800ca00 con 0x7f736806f7c0 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735800cdd0 con 0x7f736806f7c0 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.907+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73680c4680 con 0x7f736806f7c0 2024-04-25T12:37:40.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.908+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736814aaf0 con 0x7f736806f7c0 2024-04-25T12:37:40.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.909+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f736806a260 con 0x7f736806f7c0 2024-04-25T12:37:40.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.910+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f735801a070 con 0x7f736806f7c0 2024-04-25T12:37:40.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.911+0000 7f735f400640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f73440744b0 0x7f73440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.911+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f7358095bd0 con 0x7f736806f7c0 2024-04-25T12:37:40.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.912+0000 7f735f400640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f734407e190 0x7f7344080680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:40.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.912+0000 7f735fe00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f73440744b0 0x7f73440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.912+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7344080d50 con 0x7f734407e190 2024-04-25T12:37:40.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.912+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f7358096060 con 0x7f736806f7c0 2024-04-25T12:37:40.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.912+0000 7f7367400640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f734407e190 0x7f7344080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:40.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.912+0000 7f735fe00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f73440744b0 0x7f73440769c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7368140990 tx=0x7f7350009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.913+0000 7f7367400640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f734407e190 0x7f7344080680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:40.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.914+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== osd.7 v2:172.21.15.153:6818/3086604284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7344080d50 con 0x7f734407e190 2024-04-25T12:37:40.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.929+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 --> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7368076700 con 0x7f734407e190 2024-04-25T12:37:40.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f735f400640 1 -- 172.21.15.123:0/1018084582 <== osd.7 v2:172.21.15.153:6818/3086604284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7368076700 con 0x7f734407e190 2024-04-25T12:37:40.930 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T12:37:40.930 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T12:37:40.930 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f734407e190 msgr2=0x7f7344080680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f736de00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6818/3086604284,v1:172.21.15.153:6819/3086604284] conn(0x7f734407e190 0x7f7344080680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f73440744b0 msgr2=0x7f73440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f736de00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f73440744b0 0x7f73440769c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7368140990 tx=0x7f7350009040 comp rx=0 tx=0).stop 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 msgr2=0x7f73680cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f736de00640 1 --2- 172.21.15.123:0/1018084582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f736806f7c0 0x7f73680cbcd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f735800e940 tx=0x7f735800ee00 comp rx=0 tx=0).stop 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.930+0000 7f7367400640 1 -- 172.21.15.123:0/1018084582 reap_dead start 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.931+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 shutdown_connections 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.931+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 >> 172.21.15.123:0/1018084582 conn(0x7f736806c570 msgr2=0x7f73680c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.931+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 shutdown_connections 2024-04-25T12:37:40.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:40.931+0000 7f736de00640 1 -- 172.21.15.123:0/1018084582 wait complete. 2024-04-25T12:37:40.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T12:37:41.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.231+0000 7f913f200640 1 Processor -- start 2024-04-25T12:37:41.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.231+0000 7f913f200640 1 -- start start 2024-04-25T12:37:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.233+0000 7f913f200640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91400c94f0 0x7f91400c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91401593c0 con 0x7f91400c2720 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913f200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91401596a0 con 0x7f9140072d70 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913f200640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9140159fd0 con 0x7f91400c94f0 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913e800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f91400c2720 0x7f91400c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60940/0 (socket says 172.21.15.123:60940) 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913e800640 1 -- 172.21.15.123:0/2341893574 learned_addr learned my addr 172.21.15.123:0/2341893574 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913d400640 1 --2- 172.21.15.123:0/2341893574 >> v2:172.21.15.153:3300/0 conn(0x7f91400c94f0 0x7f91400c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913ca00640 1 -- 172.21.15.123:0/2341893574 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847864538 0 0) 0x7f91401593c0 con 0x7f91400c2720 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.234+0000 7f913ca00640 1 -- 172.21.15.123:0/2341893574 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9124003520 con 0x7f91400c2720 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.235+0000 7f913ca00640 1 -- 172.21.15.123:0/2341893574 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993771883 0 0) 0x7f91401596a0 con 0x7f9140072d70 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.235+0000 7f913d400640 1 -- 172.21.15.123:0/2341893574 >> v1:172.21.15.153:6789/0 conn(0x7f9140072d70 legacy=0x7f91401420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.235+0000 7f913d400640 1 -- 172.21.15.123:0/2341893574 >> v1:172.21.15.123:6789/0 conn(0x7f91400c2720 legacy=0x7f91400c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.235+0000 7f913d400640 1 -- 172.21.15.123:0/2341893574 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9140159980 con 0x7f91400c94f0 2024-04-25T12:37:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.236+0000 7f913d400640 1 --2- 172.21.15.123:0/2341893574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c94f0 0x7f91400c9900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f912c0097a0 tx=0x7f912c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4440ec07738d5d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.236+0000 7f913ca00640 1 -- 172.21.15.123:0/2341893574 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c037070 con 0x7f91400c94f0 2024-04-25T12:37:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.236+0000 7f913ca00640 1 -- 172.21.15.123:0/2341893574 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c032640 con 0x7f91400c94f0 2024-04-25T12:37:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.236+0000 7f913ca00640 1 -- 172.21.15.123:0/2341893574 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c032a10 con 0x7f91400c94f0 2024-04-25T12:37:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.237+0000 7f913f200640 1 -- 172.21.15.123:0/2341893574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c94f0 msgr2=0x7f91400c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.237+0000 7f913f200640 1 --2- 172.21.15.123:0/2341893574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c94f0 0x7f91400c9900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f912c0097a0 tx=0x7f912c02a2c0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.237+0000 7f913f200640 1 -- 172.21.15.123:0/2341893574 shutdown_connections 2024-04-25T12:37:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.237+0000 7f913f200640 1 --2- 172.21.15.123:0/2341893574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c94f0 0x7f91400c9900 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.237+0000 7f913f200640 1 -- 172.21.15.123:0/2341893574 >> 172.21.15.123:0/2341893574 conn(0x7f91400cd600 msgr2=0x7f91400cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.237+0000 7f913f200640 1 -- 172.21.15.123:0/2341893574 shutdown_connections 2024-04-25T12:37:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.237+0000 7f913f200640 1 -- 172.21.15.123:0/2341893574 wait complete. 2024-04-25T12:37:41.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.240+0000 7f913f200640 1 Processor -- start 2024-04-25T12:37:41.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.240+0000 7f913f200640 1 -- start start 2024-04-25T12:37:41.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.242+0000 7f913f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9140072d70 0x7f91401f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.242+0000 7f913f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c2720 0x7f91400cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.242+0000 7f913f200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91400c94f0 0x7f914015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.242+0000 7f913f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91400c8990 con 0x7f9140072d70 2024-04-25T12:37:41.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.242+0000 7f913f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91400c8810 con 0x7f91400c2720 2024-04-25T12:37:41.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.242+0000 7f913f200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f91401f26c0 con 0x7f91400c94f0 2024-04-25T12:37:41.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9140072d70 0x7f91401f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9140072d70 0x7f91401f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60162/0 (socket says 172.21.15.123:60162) 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 -- 172.21.15.123:0/2040696552 learned_addr learned my addr 172.21.15.123:0/2040696552 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913e800640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91400c94f0 0x7f914015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913d400640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c2720 0x7f91400cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 -- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91400c94f0 msgr2=0x7f914015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91400c94f0 0x7f914015c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 -- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c2720 msgr2=0x7f91400cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c2720 0x7f91400cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.243+0000 7f913de00640 1 -- 172.21.15.123:0/2040696552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914015c230 con 0x7f9140072d70 2024-04-25T12:37:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.244+0000 7f913de00640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9140072d70 0x7f91401f1cf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f913000c3f0 tx=0x7f913000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.244+0000 7f9136000640 1 -- 172.21.15.123:0/2040696552 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130017070 con 0x7f9140072d70 2024-04-25T12:37:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.244+0000 7f9136000640 1 -- 172.21.15.123:0/2040696552 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91300035f0 con 0x7f9140072d70 2024-04-25T12:37:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.244+0000 7f9136000640 1 -- 172.21.15.123:0/2040696552 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91300039e0 con 0x7f9140072d70 2024-04-25T12:37:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.244+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91401f5200 con 0x7f9140072d70 2024-04-25T12:37:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.244+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401f57c0 con 0x7f9140072d70 2024-04-25T12:37:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.245+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9140076da0 con 0x7f9140072d70 2024-04-25T12:37:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.246+0000 7f9136000640 1 -- 172.21.15.123:0/2040696552 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f913001a070 con 0x7f9140072d70 2024-04-25T12:37:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.247+0000 7f9136000640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91200744b0 0x7f91200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.247+0000 7f9136000640 1 -- 172.21.15.123:0/2040696552 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f9130094a20 con 0x7f9140072d70 2024-04-25T12:37:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.248+0000 7f913d400640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91200744b0 0x7f91200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.248+0000 7f913d400640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91200744b0 0x7f91200769c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f912c034000 tx=0x7f912c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:41.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.252+0000 7f9136000640 1 -- 172.21.15.123:0/2040696552 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f913005e6b0 con 0x7f9140072d70 2024-04-25T12:37:41.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.456+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f91400cdd90 con 0x7f91200744b0 2024-04-25T12:37:41.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.457+0000 7f9136000640 1 -- 172.21.15.123:0/2040696552 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f91400cdd90 con 0x7f91200744b0 2024-04-25T12:37:41.457 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91200744b0 msgr2=0x7f91200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91200744b0 0x7f91200769c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f912c034000 tx=0x7f912c02e000 comp rx=0 tx=0).stop 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9140072d70 msgr2=0x7f91401f1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9140072d70 0x7f91401f1cf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f913000c3f0 tx=0x7f913000c8b0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 shutdown_connections 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f91200744b0 0x7f91200769c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f91400c94f0 0x7f914015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91400c2720 0x7f91400cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 --2- 172.21.15.123:0/2040696552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9140072d70 0x7f91401f1cf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.459+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 >> 172.21.15.123:0/2040696552 conn(0x7f91400cd600 msgr2=0x7f91400b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.460+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 shutdown_connections 2024-04-25T12:37:41.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.460+0000 7f913f200640 1 -- 172.21.15.123:0/2040696552 wait complete. 2024-04-25T12:37:41.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:37:41.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.740+0000 7f0e38e00640 1 Processor -- start 2024-04-25T12:37:41.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.740+0000 7f0e38e00640 1 -- start start 2024-04-25T12:37:41.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e38e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e340c9080 0x7f0e341546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e38e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e3406c570 0x7f0e3406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e38e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3415a330 con 0x7f0e340c82b0 2024-04-25T12:37:41.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e38e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0e34073b10 con 0x7f0e3406c570 2024-04-25T12:37:41.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e38e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0e34073df0 con 0x7f0e340c9080 2024-04-25T12:37:41.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e33e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e3406c570 0x7f0e3406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e33e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e3406c570 0x7f0e3406c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60170/0 (socket says 172.21.15.123:60170) 2024-04-25T12:37:41.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e33e00640 1 -- 172.21.15.123:0/2037746648 learned_addr learned my addr 172.21.15.123:0/2037746648 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:41.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.742+0000 7f0e33400640 1 --2- 172.21.15.123:0/2037746648 >> v2:172.21.15.153:3300/0 conn(0x7f0e340c9080 0x7f0e341546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.743+0000 7f0e33e00640 1 -- 172.21.15.123:0/2037746648 >> v2:172.21.15.153:3300/0 conn(0x7f0e340c9080 msgr2=0x7f0e341546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:37:41.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.743+0000 7f0e33e00640 1 --2- 172.21.15.123:0/2037746648 >> v2:172.21.15.153:3300/0 conn(0x7f0e340c9080 0x7f0e341546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.743+0000 7f0e32000640 1 -- 172.21.15.123:0/2037746648 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 542547994 0 0) 0x7f0e3415a330 con 0x7f0e340c82b0 2024-04-25T12:37:41.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.743+0000 7f0e33e00640 1 -- 172.21.15.123:0/2037746648 >> v1:172.21.15.153:6789/0 conn(0x7f0e340c82b0 legacy=0x7f0e3415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.743+0000 7f0e33e00640 1 -- 172.21.15.123:0/2037746648 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e340740d0 con 0x7f0e3406c570 2024-04-25T12:37:41.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.744+0000 7f0e33e00640 1 --2- 172.21.15.123:0/2037746648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e3406c980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e340ce520 tx=0x7f0e2c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94fa7e553f4cab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.744+0000 7f0e32000640 1 -- 172.21.15.123:0/2037746648 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c036070 con 0x7f0e3406c570 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.744+0000 7f0e32000640 1 -- 172.21.15.123:0/2037746648 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2c031600 con 0x7f0e3406c570 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.744+0000 7f0e32000640 1 -- 172.21.15.123:0/2037746648 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0319d0 con 0x7f0e3406c570 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 -- 172.21.15.123:0/2037746648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 msgr2=0x7f0e3406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 --2- 172.21.15.123:0/2037746648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e3406c980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e340ce520 tx=0x7f0e2c02a6a0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 -- 172.21.15.123:0/2037746648 shutdown_connections 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 --2- 172.21.15.123:0/2037746648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e3406c980 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 --2- 172.21.15.123:0/2037746648 >> v2:172.21.15.153:3300/0 conn(0x7f0e340c9080 0x7f0e341546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 -- 172.21.15.123:0/2037746648 >> 172.21.15.123:0/2037746648 conn(0x7f0e340cd600 msgr2=0x7f0e340cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 -- 172.21.15.123:0/2037746648 shutdown_connections 2024-04-25T12:37:41.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.745+0000 7f0e38e00640 1 -- 172.21.15.123:0/2037746648 wait complete. 2024-04-25T12:37:41.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.747+0000 7f0e38e00640 1 Processor -- start 2024-04-25T12:37:41.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.748+0000 7f0e38e00640 1 -- start start 2024-04-25T12:37:41.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e38e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e341efd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e38e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e340c82b0 0x7f0e3415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e38e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e340c9080 0x7f0e3414b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e38e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3415e760 con 0x7f0e3406c570 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e38e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3415db30 con 0x7f0e340c9080 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e38e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0e34073760 con 0x7f0e340c82b0 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e33400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e341efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e33400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e341efd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60184/0 (socket says 172.21.15.123:60184) 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e33400640 1 -- 172.21.15.123:0/386137168 learned_addr learned my addr 172.21.15.123:0/386137168 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e32a00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e340c82b0 0x7f0e3415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.749+0000 7f0e33e00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e340c9080 0x7f0e3414b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e33400640 1 -- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e340c82b0 msgr2=0x7f0e3415b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e33400640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e340c82b0 0x7f0e3415b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e33400640 1 -- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e340c9080 msgr2=0x7f0e3414b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e33400640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e340c9080 0x7f0e3414b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e33400640 1 -- 172.21.15.123:0/386137168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e340ccea0 con 0x7f0e3406c570 2024-04-25T12:37:41.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e33400640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e341efd90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e8a0 tx=0x7f0e2400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:41.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e2b400640 1 -- 172.21.15.123:0/386137168 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24017070 con 0x7f0e3406c570 2024-04-25T12:37:41.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e2b400640 1 -- 172.21.15.123:0/386137168 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2400c940 con 0x7f0e3406c570 2024-04-25T12:37:41.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3414edb0 con 0x7f0e3406c570 2024-04-25T12:37:41.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.750+0000 7f0e2b400640 1 -- 172.21.15.123:0/386137168 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400cd30 con 0x7f0e3406c570 2024-04-25T12:37:41.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.751+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e340cc6a0 con 0x7f0e3406c570 2024-04-25T12:37:41.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.752+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e34076da0 con 0x7f0e3406c570 2024-04-25T12:37:41.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.753+0000 7f0e2b400640 1 -- 172.21.15.123:0/386137168 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e2401a070 con 0x7f0e3406c570 2024-04-25T12:37:41.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.754+0000 7f0e2b400640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e140744e0 0x7f0e140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:37:41.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.754+0000 7f0e2b400640 1 -- 172.21.15.123:0/386137168 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f0e24095480 con 0x7f0e3406c570 2024-04-25T12:37:41.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.754+0000 7f0e32a00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e140744e0 0x7f0e140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:37:41.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.755+0000 7f0e32a00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e140744e0 0x7f0e140769f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0e341f0ec0 tx=0x7f0e20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:37:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.758+0000 7f0e2b400640 1 -- 172.21.15.123:0/386137168 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e2405f120 con 0x7f0e3406c570 2024-04-25T12:37:41.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.957+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e340c8e60 con 0x7f0e140744e0 2024-04-25T12:37:41.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.968+0000 7f0e2b400640 1 -- 172.21.15.123:0/386137168 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332405 (secure 0 0 0) 0x7f0e340c8e60 con 0x7f0e140744e0 2024-04-25T12:37:41.972 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:37:41.974 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.972+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e140744e0 msgr2=0x7f0e140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.972+0000 7f0e38e00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e140744e0 0x7f0e140769f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0e341f0ec0 tx=0x7f0e20009040 comp rx=0 tx=0).stop 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.972+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 msgr2=0x7f0e341efd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.972+0000 7f0e38e00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e341efd90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e8a0 tx=0x7f0e2400ed60 comp rx=0 tx=0).stop 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 shutdown_connections 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f0e140744e0 0x7f0e140769f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e340c9080 0x7f0e3414b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e340c82b0 0x7f0e3415b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 --2- 172.21.15.123:0/386137168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3406c570 0x7f0e341efd90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 >> 172.21.15.123:0/386137168 conn(0x7f0e340cd600 msgr2=0x7f0e340c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:37:41.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 shutdown_connections 2024-04-25T12:37:41.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:37:41.973+0000 7f0e38e00640 1 -- 172.21.15.123:0/386137168 wait complete. 2024-04-25T12:37:42.001 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-04-25T12:37:40.845850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631669,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":274,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":731,"ondisk_log_size":731,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238704,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":229739,"kb_avail":754736016,"statfs":{"total":773094113280,"available":772849680384,"internally_reserved":0,"allocated":9093120,"data_stored":5890226,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":235253715},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004200"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569452+0000","last_change":"2024-04-25T12:34:12.912238+0000","last_active":"2024-04-25T12:37:21.569452+0000","last_peered":"2024-04-25T12:37:21.569452+0000","last_clean":"2024-04-25T12:37:21.569452+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:21.569452+0000","last_undegraded":"2024-04-25T12:37:21.569452+0000","last_fullsized":"2024-04-25T12:37:21.569452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:41.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.064960+0000","last_change":"2024-04-25T12:36:00.630269+0000","last_active":"2024-04-25T12:37:31.064960+0000","last_peered":"2024-04-25T12:37:31.064960+0000","last_clean":"2024-04-25T12:37:31.064960+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:31.064960+0000","last_undegraded":"2024-04-25T12:37:31.064960+0000","last_fullsized":"2024-04-25T12:37:31.064960+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:21.807271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402490+0000","last_change":"2024-04-25T12:37:01.222445+0000","last_active":"2024-04-25T12:37:21.402490+0000","last_peered":"2024-04-25T12:37:21.402490+0000","last_clean":"2024-04-25T12:37:21.402490+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:21.402490+0000","last_undegraded":"2024-04-25T12:37:21.402490+0000","last_fullsized":"2024-04-25T12:37:21.402490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907880+0000","last_change":"2024-04-25T12:36:00.628990+0000","last_active":"2024-04-25T12:37:21.907880+0000","last_peered":"2024-04-25T12:37:21.907880+0000","last_clean":"2024-04-25T12:37:21.907880+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:21.907880+0000","last_undegraded":"2024-04-25T12:37:21.907880+0000","last_fullsized":"2024-04-25T12:37:21.907880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.754242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":114,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.063519+0000","last_change":"2024-04-25T12:36:00.629376+0000","last_active":"2024-04-25T12:37:31.063519+0000","last_peered":"2024-04-25T12:37:31.063519+0000","last_clean":"2024-04-25T12:37:31.063519+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:31.063519+0000","last_undegraded":"2024-04-25T12:37:31.063519+0000","last_fullsized":"2024-04-25T12:37:31.063519+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:52.856469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404380+0000","last_change":"2024-04-25T12:37:01.216562+0000","last_active":"2024-04-25T12:37:21.404380+0000","last_peered":"2024-04-25T12:37:21.404380+0000","last_clean":"2024-04-25T12:37:21.404380+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:21.404380+0000","last_undegraded":"2024-04-25T12:37:21.404380+0000","last_fullsized":"2024-04-25T12:37:21.404380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:28.812215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"70'5","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910373+0000","last_change":"2024-04-25T12:34:12.929127+0000","last_active":"2024-04-25T12:37:21.910373+0000","last_peered":"2024-04-25T12:37:21.910373+0000","last_clean":"2024-04-25T12:37:21.910373+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:21.910373+0000","last_undegraded":"2024-04-25T12:37:21.910373+0000","last_fullsized":"2024-04-25T12:37:21.910373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.077582+0000","last_change":"2024-04-25T12:34:13.977308+0000","last_active":"2024-04-25T12:37:31.077582+0000","last_peered":"2024-04-25T12:37:31.077582+0000","last_clean":"2024-04-25T12:37:31.077582+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:31.077582+0000","last_undegraded":"2024-04-25T12:37:31.077582+0000","last_fullsized":"2024-04-25T12:37:31.077582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405114+0000","last_change":"2024-04-25T12:37:01.226633+0000","last_active":"2024-04-25T12:37:21.405114+0000","last_peered":"2024-04-25T12:37:21.405114+0000","last_clean":"2024-04-25T12:37:21.405114+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:21.405114+0000","last_undegraded":"2024-04-25T12:37:21.405114+0000","last_fullsized":"2024-04-25T12:37:21.405114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.771833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"93'3","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:37.196340+0000","last_change":"2024-04-25T12:37:37.196393+0000","last_active":"2024-04-25T12:37:37.196340+0000","last_peered":"2024-04-25T12:37:37.196340+0000","last_clean":"2024-04-25T12:37:37.196340+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:37.196340+0000","last_undegraded":"2024-04-25T12:37:37.196340+0000","last_fullsized":"2024-04-25T12:37:37.196340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:37.196244+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:37.196244+0000","last_clean_scrub_stamp":"2024-04-25T12:37:37.196244+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:49.970014+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.087212+0000","last_change":"2024-04-25T12:34:13.977816+0000","last_active":"2024-04-25T12:37:31.087212+0000","last_peered":"2024-04-25T12:37:31.087212+0000","last_clean":"2024-04-25T12:37:31.087212+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:31.087212+0000","last_undegraded":"2024-04-25T12:37:31.087212+0000","last_fullsized":"2024-04-25T12:37:31.087212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:22.072117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404697+0000","last_change":"2024-04-25T12:37:01.221650+0000","last_active":"2024-04-25T12:37:21.404697+0000","last_peered":"2024-04-25T12:37:21.404697+0000","last_clean":"2024-04-25T12:37:21.404697+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:21.404697+0000","last_undegraded":"2024-04-25T12:37:21.404697+0000","last_fullsized":"2024-04-25T12:37:21.404697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.084165+0000","last_change":"2024-04-25T12:34:13.971548+0000","last_active":"2024-04-25T12:37:31.084165+0000","last_peered":"2024-04-25T12:37:31.084165+0000","last_clean":"2024-04-25T12:37:31.084165+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:31.084165+0000","last_undegraded":"2024-04-25T12:37:31.084165+0000","last_fullsized":"2024-04-25T12:37:31.084165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:33.712902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423223+0000","last_change":"2024-04-25T12:34:12.923724+0000","last_active":"2024-04-25T12:37:21.423223+0000","last_peered":"2024-04-25T12:37:21.423223+0000","last_clean":"2024-04-25T12:37:21.423223+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:21.423223+0000","last_undegraded":"2024-04-25T12:37:21.423223+0000","last_fullsized":"2024-04-25T12:37:21.423223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:00.653280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":154,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.909805+0000","last_change":"2024-04-25T12:36:00.640119+0000","last_active":"2024-04-25T12:37:21.909805+0000","last_peered":"2024-04-25T12:37:21.909805+0000","last_clean":"2024-04-25T12:37:21.909805+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:21.909805+0000","last_undegraded":"2024-04-25T12:37:21.909805+0000","last_fullsized":"2024-04-25T12:37:21.909805+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.044483+0000","last_change":"2024-04-25T12:37:01.209490+0000","last_active":"2024-04-25T12:37:31.044483+0000","last_peered":"2024-04-25T12:37:31.044483+0000","last_clean":"2024-04-25T12:37:31.044483+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:31.044483+0000","last_undegraded":"2024-04-25T12:37:31.044483+0000","last_fullsized":"2024-04-25T12:37:31.044483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:46.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908543+0000","last_change":"2024-04-25T12:37:01.230485+0000","last_active":"2024-04-25T12:37:21.908543+0000","last_peered":"2024-04-25T12:37:21.908543+0000","last_clean":"2024-04-25T12:37:21.908543+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:21.908543+0000","last_undegraded":"2024-04-25T12:37:21.908543+0000","last_fullsized":"2024-04-25T12:37:21.908543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:47.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910133+0000","last_change":"2024-04-25T12:37:01.222195+0000","last_active":"2024-04-25T12:37:21.910133+0000","last_peered":"2024-04-25T12:37:21.910133+0000","last_clean":"2024-04-25T12:37:21.910133+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:21.910133+0000","last_undegraded":"2024-04-25T12:37:21.910133+0000","last_fullsized":"2024-04-25T12:37:21.910133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.038851+0000","last_change":"2024-04-25T12:34:13.935345+0000","last_active":"2024-04-25T12:37:31.038851+0000","last_peered":"2024-04-25T12:37:31.038851+0000","last_clean":"2024-04-25T12:37:31.038851+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:31.038851+0000","last_undegraded":"2024-04-25T12:37:31.038851+0000","last_fullsized":"2024-04-25T12:37:31.038851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:37.411199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569070+0000","last_change":"2024-04-25T12:34:12.918854+0000","last_active":"2024-04-25T12:37:21.569070+0000","last_peered":"2024-04-25T12:37:21.569070+0000","last_clean":"2024-04-25T12:37:21.569070+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:21.569070+0000","last_undegraded":"2024-04-25T12:37:21.569070+0000","last_fullsized":"2024-04-25T12:37:21.569070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:55.791067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403488+0000","last_change":"2024-04-25T12:34:18.016224+0000","last_active":"2024-04-25T12:37:21.403488+0000","last_peered":"2024-04-25T12:37:21.403488+0000","last_clean":"2024-04-25T12:37:21.403488+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:21.403488+0000","last_undegraded":"2024-04-25T12:37:21.403488+0000","last_fullsized":"2024-04-25T12:37:21.403488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:36.377341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":154,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.018224+0000","last_change":"2024-04-25T12:34:13.967757+0000","last_active":"2024-04-25T12:37:31.018224+0000","last_peered":"2024-04-25T12:37:31.018224+0000","last_clean":"2024-04-25T12:37:31.018224+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:31.018224+0000","last_undegraded":"2024-04-25T12:37:31.018224+0000","last_fullsized":"2024-04-25T12:37:31.018224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:47.865411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422292+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:37:21.422292+0000","last_peered":"2024-04-25T12:37:21.422292+0000","last_clean":"2024-04-25T12:37:21.422292+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:37:21.422292+0000","last_undegraded":"2024-04-25T12:37:21.422292+0000","last_fullsized":"2024-04-25T12:37:21.422292+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:16.988943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405968+0000","last_change":"2024-04-25T12:34:18.008204+0000","last_active":"2024-04-25T12:37:21.405968+0000","last_peered":"2024-04-25T12:37:21.405968+0000","last_clean":"2024-04-25T12:37:21.405968+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:21.405968+0000","last_undegraded":"2024-04-25T12:37:21.405968+0000","last_fullsized":"2024-04-25T12:37:21.405968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.038670+0000","last_change":"2024-04-25T12:34:13.935954+0000","last_active":"2024-04-25T12:37:31.038670+0000","last_peered":"2024-04-25T12:37:31.038670+0000","last_clean":"2024-04-25T12:37:31.038670+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:31.038670+0000","last_undegraded":"2024-04-25T12:37:31.038670+0000","last_fullsized":"2024-04-25T12:37:31.038670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:33.186450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"70'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424634+0000","last_change":"2024-04-25T12:34:12.921868+0000","last_active":"2024-04-25T12:37:21.424634+0000","last_peered":"2024-04-25T12:37:21.424634+0000","last_clean":"2024-04-25T12:37:21.424634+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:21.424634+0000","last_undegraded":"2024-04-25T12:37:21.424634+0000","last_fullsized":"2024-04-25T12:37:21.424634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.306082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404952+0000","last_change":"2024-04-25T12:37:01.226160+0000","last_active":"2024-04-25T12:37:21.404952+0000","last_peered":"2024-04-25T12:37:21.404952+0000","last_clean":"2024-04-25T12:37:21.404952+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:21.404952+0000","last_undegraded":"2024-04-25T12:37:21.404952+0000","last_fullsized":"2024-04-25T12:37:21.404952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:43.475095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.038094+0000","last_change":"2024-04-25T12:37:01.206319+0000","last_active":"2024-04-25T12:37:31.038094+0000","last_peered":"2024-04-25T12:37:31.038094+0000","last_clean":"2024-04-25T12:37:31.038094+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:31.038094+0000","last_undegraded":"2024-04-25T12:37:31.038094+0000","last_fullsized":"2024-04-25T12:37:31.038094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"93'13","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569402+0000","last_change":"2024-04-25T12:34:12.918589+0000","last_active":"2024-04-25T12:37:21.569402+0000","last_peered":"2024-04-25T12:37:21.569402+0000","last_clean":"2024-04-25T12:37:21.569402+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:21.569402+0000","last_undegraded":"2024-04-25T12:37:21.569402+0000","last_fullsized":"2024-04-25T12:37:21.569402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:12.536208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568614+0000","last_change":"2024-04-25T12:37:01.225992+0000","last_active":"2024-04-25T12:37:21.568614+0000","last_peered":"2024-04-25T12:37:21.568614+0000","last_clean":"2024-04-25T12:37:21.568614+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:21.568614+0000","last_undegraded":"2024-04-25T12:37:21.568614+0000","last_fullsized":"2024-04-25T12:37:21.568614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:05.029213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.047473+0000","last_change":"2024-04-25T12:37:01.208764+0000","last_active":"2024-04-25T12:37:31.047473+0000","last_peered":"2024-04-25T12:37:31.047473+0000","last_clean":"2024-04-25T12:37:31.047473+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:31.047473+0000","last_undegraded":"2024-04-25T12:37:31.047473+0000","last_fullsized":"2024-04-25T12:37:31.047473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:38.538923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424069+0000","last_change":"2024-04-25T12:34:12.900723+0000","last_active":"2024-04-25T12:37:21.424069+0000","last_peered":"2024-04-25T12:37:21.424069+0000","last_clean":"2024-04-25T12:37:21.424069+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:21.424069+0000","last_undegraded":"2024-04-25T12:37:21.424069+0000","last_fullsized":"2024-04-25T12:37:21.424069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:39.059634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908596+0000","last_change":"2024-04-25T12:37:01.225006+0000","last_active":"2024-04-25T12:37:21.908596+0000","last_peered":"2024-04-25T12:37:21.908596+0000","last_clean":"2024-04-25T12:37:21.908596+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:21.908596+0000","last_undegraded":"2024-04-25T12:37:21.908596+0000","last_fullsized":"2024-04-25T12:37:21.908596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:22.621929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.054034+0000","last_change":"2024-04-25T12:37:01.222179+0000","last_active":"2024-04-25T12:37:31.054034+0000","last_peered":"2024-04-25T12:37:31.054034+0000","last_clean":"2024-04-25T12:37:31.054034+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:31.054034+0000","last_undegraded":"2024-04-25T12:37:31.054034+0000","last_fullsized":"2024-04-25T12:37:31.054034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907696+0000","last_change":"2024-04-25T12:37:01.211908+0000","last_active":"2024-04-25T12:37:21.907696+0000","last_peered":"2024-04-25T12:37:21.907696+0000","last_clean":"2024-04-25T12:37:21.907696+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:21.907696+0000","last_undegraded":"2024-04-25T12:37:21.907696+0000","last_fullsized":"2024-04-25T12:37:21.907696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:23.523737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908482+0000","last_change":"2024-04-25T12:37:01.229773+0000","last_active":"2024-04-25T12:37:21.908482+0000","last_peered":"2024-04-25T12:37:21.908482+0000","last_clean":"2024-04-25T12:37:21.908482+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:21.908482+0000","last_undegraded":"2024-04-25T12:37:21.908482+0000","last_fullsized":"2024-04-25T12:37:21.908482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:55.680417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052653+0000","last_change":"2024-04-25T12:37:01.223560+0000","last_active":"2024-04-25T12:37:31.052653+0000","last_peered":"2024-04-25T12:37:31.052653+0000","last_clean":"2024-04-25T12:37:31.052653+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:31.052653+0000","last_undegraded":"2024-04-25T12:37:31.052653+0000","last_fullsized":"2024-04-25T12:37:31.052653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:18.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424227+0000","last_change":"2024-04-25T12:37:01.227566+0000","last_active":"2024-04-25T12:37:21.424227+0000","last_peered":"2024-04-25T12:37:21.424227+0000","last_clean":"2024-04-25T12:37:21.424227+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:37:21.424227+0000","last_undegraded":"2024-04-25T12:37:21.424227+0000","last_fullsized":"2024-04-25T12:37:21.424227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:43.265214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908845+0000","last_change":"2024-04-25T12:37:01.226991+0000","last_active":"2024-04-25T12:37:21.908845+0000","last_peered":"2024-04-25T12:37:21.908845+0000","last_clean":"2024-04-25T12:37:21.908845+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:21.908845+0000","last_undegraded":"2024-04-25T12:37:21.908845+0000","last_fullsized":"2024-04-25T12:37:21.908845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:23.065480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":116,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052400+0000","last_change":"2024-04-25T12:36:00.622385+0000","last_active":"2024-04-25T12:37:31.052400+0000","last_peered":"2024-04-25T12:37:31.052400+0000","last_clean":"2024-04-25T12:37:31.052400+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:31.052400+0000","last_undegraded":"2024-04-25T12:37:31.052400+0000","last_fullsized":"2024-04-25T12:37:31.052400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:05.435118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'7","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402648+0000","last_change":"2024-04-25T12:36:00.625934+0000","last_active":"2024-04-25T12:37:21.402648+0000","last_peered":"2024-04-25T12:37:21.402648+0000","last_clean":"2024-04-25T12:37:21.402648+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:21.402648+0000","last_undegraded":"2024-04-25T12:37:21.402648+0000","last_fullsized":"2024-04-25T12:37:21.402648+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:28.578445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907983+0000","last_change":"2024-04-25T12:37:01.222254+0000","last_active":"2024-04-25T12:37:21.907983+0000","last_peered":"2024-04-25T12:37:21.907983+0000","last_clean":"2024-04-25T12:37:21.907983+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:21.907983+0000","last_undegraded":"2024-04-25T12:37:21.907983+0000","last_fullsized":"2024-04-25T12:37:21.907983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:50.699807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052986+0000","last_change":"2024-04-25T12:37:01.224799+0000","last_active":"2024-04-25T12:37:31.052986+0000","last_peered":"2024-04-25T12:37:31.052986+0000","last_clean":"2024-04-25T12:37:31.052986+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:31.052986+0000","last_undegraded":"2024-04-25T12:37:31.052986+0000","last_fullsized":"2024-04-25T12:37:31.052986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:24.737805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423362+0000","last_change":"2024-04-25T12:36:00.636542+0000","last_active":"2024-04-25T12:37:21.423362+0000","last_peered":"2024-04-25T12:37:21.423362+0000","last_clean":"2024-04-25T12:37:21.423362+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:21.423362+0000","last_undegraded":"2024-04-25T12:37:21.423362+0000","last_fullsized":"2024-04-25T12:37:21.423362+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.867390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908375+0000","last_change":"2024-04-25T12:37:01.226465+0000","last_active":"2024-04-25T12:37:21.908375+0000","last_peered":"2024-04-25T12:37:21.908375+0000","last_clean":"2024-04-25T12:37:21.908375+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:21.908375+0000","last_undegraded":"2024-04-25T12:37:21.908375+0000","last_fullsized":"2024-04-25T12:37:21.908375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:26.218787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.054718+0000","last_change":"2024-04-25T12:34:13.958859+0000","last_active":"2024-04-25T12:37:31.054718+0000","last_peered":"2024-04-25T12:37:31.054718+0000","last_clean":"2024-04-25T12:37:31.054718+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:31.054718+0000","last_undegraded":"2024-04-25T12:37:31.054718+0000","last_fullsized":"2024-04-25T12:37:31.054718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:32.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423614+0000","last_change":"2024-04-25T12:34:12.921113+0000","last_active":"2024-04-25T12:37:21.423614+0000","last_peered":"2024-04-25T12:37:21.423614+0000","last_clean":"2024-04-25T12:37:21.423614+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:21.423614+0000","last_undegraded":"2024-04-25T12:37:21.423614+0000","last_fullsized":"2024-04-25T12:37:21.423614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.194510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":263,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911041+0000","last_change":"2024-04-25T12:35:16.912902+0000","last_active":"2024-04-25T12:37:21.911041+0000","last_peered":"2024-04-25T12:37:21.911041+0000","last_clean":"2024-04-25T12:37:21.911041+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:21.911041+0000","last_undegraded":"2024-04-25T12:37:21.911041+0000","last_fullsized":"2024-04-25T12:37:21.911041+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:07.013466+0000","last_clean_scrub_stamp":"2024-04-25T12:34:07.013466+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423841+0000","last_change":"2024-04-25T12:34:18.012048+0000","last_active":"2024-04-25T12:37:21.423841+0000","last_peered":"2024-04-25T12:37:21.423841+0000","last_clean":"2024-04-25T12:37:21.423841+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:21.423841+0000","last_undegraded":"2024-04-25T12:37:21.423841+0000","last_fullsized":"2024-04-25T12:37:21.423841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.193343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.088496+0000","last_change":"2024-04-25T12:34:13.968759+0000","last_active":"2024-04-25T12:37:31.088496+0000","last_peered":"2024-04-25T12:37:31.088496+0000","last_clean":"2024-04-25T12:37:31.088496+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:31.088496+0000","last_undegraded":"2024-04-25T12:37:31.088496+0000","last_fullsized":"2024-04-25T12:37:31.088496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.223112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908962+0000","last_change":"2024-04-25T12:37:01.242232+0000","last_active":"2024-04-25T12:37:21.908962+0000","last_peered":"2024-04-25T12:37:21.908962+0000","last_clean":"2024-04-25T12:37:21.908962+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:21.908962+0000","last_undegraded":"2024-04-25T12:37:21.908962+0000","last_fullsized":"2024-04-25T12:37:21.908962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:59.941469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.404864+0000","last_change":"2024-04-25T12:34:18.008915+0000","last_active":"2024-04-25T12:37:21.404864+0000","last_peered":"2024-04-25T12:37:21.404864+0000","last_clean":"2024-04-25T12:37:21.404864+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:21.404864+0000","last_undegraded":"2024-04-25T12:37:21.404864+0000","last_fullsized":"2024-04-25T12:37:21.404864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:03.078376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.062893+0000","last_change":"2024-04-25T12:37:01.220247+0000","last_active":"2024-04-25T12:37:31.062893+0000","last_peered":"2024-04-25T12:37:31.062893+0000","last_clean":"2024-04-25T12:37:31.062893+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:31.062893+0000","last_undegraded":"2024-04-25T12:37:31.062893+0000","last_fullsized":"2024-04-25T12:37:31.062893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908088+0000","last_change":"2024-04-25T12:36:00.633583+0000","last_active":"2024-04-25T12:37:21.908088+0000","last_peered":"2024-04-25T12:37:21.908088+0000","last_clean":"2024-04-25T12:37:21.908088+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:21.908088+0000","last_undegraded":"2024-04-25T12:37:21.908088+0000","last_fullsized":"2024-04-25T12:37:21.908088+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.252431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":112,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907766+0000","last_change":"2024-04-25T12:36:00.642656+0000","last_active":"2024-04-25T12:37:21.907766+0000","last_peered":"2024-04-25T12:37:21.907766+0000","last_clean":"2024-04-25T12:37:21.907766+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:21.907766+0000","last_undegraded":"2024-04-25T12:37:21.907766+0000","last_fullsized":"2024-04-25T12:37:21.907766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.935100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.073776+0000","last_change":"2024-04-25T12:34:13.974440+0000","last_active":"2024-04-25T12:37:31.073776+0000","last_peered":"2024-04-25T12:37:31.073776+0000","last_clean":"2024-04-25T12:37:31.073776+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:31.073776+0000","last_undegraded":"2024-04-25T12:37:31.073776+0000","last_fullsized":"2024-04-25T12:37:31.073776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569231+0000","last_change":"2024-04-25T12:36:00.627125+0000","last_active":"2024-04-25T12:37:21.569231+0000","last_peered":"2024-04-25T12:37:21.569231+0000","last_clean":"2024-04-25T12:37:21.569231+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:21.569231+0000","last_undegraded":"2024-04-25T12:37:21.569231+0000","last_fullsized":"2024-04-25T12:37:21.569231+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:29.109027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422865+0000","last_change":"2024-04-25T12:34:18.007548+0000","last_active":"2024-04-25T12:37:21.422865+0000","last_peered":"2024-04-25T12:37:21.422865+0000","last_clean":"2024-04-25T12:37:21.422865+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:21.422865+0000","last_undegraded":"2024-04-25T12:37:21.422865+0000","last_fullsized":"2024-04-25T12:37:21.422865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:24.739842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.076775+0000","last_change":"2024-04-25T12:34:13.977230+0000","last_active":"2024-04-25T12:37:31.076775+0000","last_peered":"2024-04-25T12:37:31.076775+0000","last_clean":"2024-04-25T12:37:31.076775+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:31.076775+0000","last_undegraded":"2024-04-25T12:37:31.076775+0000","last_fullsized":"2024-04-25T12:37:31.076775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:06.979267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911881+0000","last_change":"2024-04-25T12:36:00.631358+0000","last_active":"2024-04-25T12:37:21.911881+0000","last_peered":"2024-04-25T12:37:21.911881+0000","last_clean":"2024-04-25T12:37:21.911881+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:21.911881+0000","last_undegraded":"2024-04-25T12:37:21.911881+0000","last_fullsized":"2024-04-25T12:37:21.911881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.910649+0000","last_change":"2024-04-25T12:36:00.644882+0000","last_active":"2024-04-25T12:37:21.910649+0000","last_peered":"2024-04-25T12:37:21.910649+0000","last_clean":"2024-04-25T12:37:21.910649+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:21.910649+0000","last_undegraded":"2024-04-25T12:37:21.910649+0000","last_fullsized":"2024-04-25T12:37:21.910649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.084311+0000","last_change":"2024-04-25T12:34:13.961149+0000","last_active":"2024-04-25T12:37:31.084311+0000","last_peered":"2024-04-25T12:37:31.084311+0000","last_clean":"2024-04-25T12:37:31.084311+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:31.084311+0000","last_undegraded":"2024-04-25T12:37:31.084311+0000","last_fullsized":"2024-04-25T12:37:31.084311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:21.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"75'16","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568873+0000","last_change":"2024-04-25T12:34:12.896717+0000","last_active":"2024-04-25T12:37:21.568873+0000","last_peered":"2024-04-25T12:37:21.568873+0000","last_clean":"2024-04-25T12:37:21.568873+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:21.568873+0000","last_undegraded":"2024-04-25T12:37:21.568873+0000","last_fullsized":"2024-04-25T12:37:21.568873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908280+0000","last_change":"2024-04-25T12:37:01.226590+0000","last_active":"2024-04-25T12:37:21.908280+0000","last_peered":"2024-04-25T12:37:21.908280+0000","last_clean":"2024-04-25T12:37:21.908280+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:21.908280+0000","last_undegraded":"2024-04-25T12:37:21.908280+0000","last_fullsized":"2024-04-25T12:37:21.908280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.581831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.058689+0000","last_change":"2024-04-25T12:36:00.638096+0000","last_active":"2024-04-25T12:37:31.058689+0000","last_peered":"2024-04-25T12:37:31.058689+0000","last_clean":"2024-04-25T12:37:31.058689+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:31.058689+0000","last_undegraded":"2024-04-25T12:37:31.058689+0000","last_fullsized":"2024-04-25T12:37:31.058689+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:36.180578+0000","last_change":"2024-04-25T12:37:36.180654+0000","last_active":"2024-04-25T12:37:36.180578+0000","last_peered":"2024-04-25T12:37:36.180578+0000","last_clean":"2024-04-25T12:37:36.180578+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:36.180578+0000","last_undegraded":"2024-04-25T12:37:36.180578+0000","last_fullsized":"2024-04-25T12:37:36.180578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:36.180353+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:36.180353+0000","last_clean_scrub_stamp":"2024-04-25T12:37:36.180353+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:38.779021+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908364+0000","last_change":"2024-04-25T12:37:01.220767+0000","last_active":"2024-04-25T12:37:21.908364+0000","last_peered":"2024-04-25T12:37:21.908364+0000","last_clean":"2024-04-25T12:37:21.908364+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:21.908364+0000","last_undegraded":"2024-04-25T12:37:21.908364+0000","last_fullsized":"2024-04-25T12:37:21.908364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:15.874258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.090198+0000","last_change":"2024-04-25T12:37:01.233439+0000","last_active":"2024-04-25T12:37:31.090198+0000","last_peered":"2024-04-25T12:37:31.090198+0000","last_clean":"2024-04-25T12:37:31.090198+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:37:31.090198+0000","last_undegraded":"2024-04-25T12:37:31.090198+0000","last_fullsized":"2024-04-25T12:37:31.090198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:11.957744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"70'4","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424042+0000","last_change":"2024-04-25T12:34:12.912150+0000","last_active":"2024-04-25T12:37:21.424042+0000","last_peered":"2024-04-25T12:37:21.424042+0000","last_clean":"2024-04-25T12:37:21.424042+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:21.424042+0000","last_undegraded":"2024-04-25T12:37:21.424042+0000","last_fullsized":"2024-04-25T12:37:21.424042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:42.736078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908903+0000","last_change":"2024-04-25T12:37:01.225914+0000","last_active":"2024-04-25T12:37:21.908903+0000","last_peered":"2024-04-25T12:37:21.908903+0000","last_clean":"2024-04-25T12:37:21.908903+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:21.908903+0000","last_undegraded":"2024-04-25T12:37:21.908903+0000","last_fullsized":"2024-04-25T12:37:21.908903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:34.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.044913+0000","last_change":"2024-04-25T12:37:01.229502+0000","last_active":"2024-04-25T12:37:31.044913+0000","last_peered":"2024-04-25T12:37:31.044913+0000","last_clean":"2024-04-25T12:37:31.044913+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:31.044913+0000","last_undegraded":"2024-04-25T12:37:31.044913+0000","last_fullsized":"2024-04-25T12:37:31.044913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:25.697423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403098+0000","last_change":"2024-04-25T12:34:12.916242+0000","last_active":"2024-04-25T12:37:21.403098+0000","last_peered":"2024-04-25T12:37:21.403098+0000","last_clean":"2024-04-25T12:37:21.403098+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:21.403098+0000","last_undegraded":"2024-04-25T12:37:21.403098+0000","last_fullsized":"2024-04-25T12:37:21.403098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:28.880476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.912228+0000","last_change":"2024-04-25T12:36:32.381600+0000","last_active":"2024-04-25T12:37:21.912228+0000","last_peered":"2024-04-25T12:37:21.912228+0000","last_clean":"2024-04-25T12:37:21.912228+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:21.912228+0000","last_undegraded":"2024-04-25T12:37:21.912228+0000","last_fullsized":"2024-04-25T12:37:21.912228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.042486+0000","last_change":"2024-04-25T12:37:01.207603+0000","last_active":"2024-04-25T12:37:31.042486+0000","last_peered":"2024-04-25T12:37:31.042486+0000","last_clean":"2024-04-25T12:37:31.042486+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:31.042486+0000","last_undegraded":"2024-04-25T12:37:31.042486+0000","last_fullsized":"2024-04-25T12:37:31.042486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:01.832860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"93'29","reported_seq":184,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.015655+0000","last_change":"2024-04-25T12:34:12.914620+0000","last_active":"2024-04-25T12:37:31.015655+0000","last_peered":"2024-04-25T12:37:31.015655+0000","last_clean":"2024-04-25T12:37:31.015655+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:31.015655+0000","last_undegraded":"2024-04-25T12:37:31.015655+0000","last_fullsized":"2024-04-25T12:37:31.015655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:47.037642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37279,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568703+0000","last_change":"2024-04-25T12:36:00.635972+0000","last_active":"2024-04-25T12:37:21.568703+0000","last_peered":"2024-04-25T12:37:21.568703+0000","last_clean":"2024-04-25T12:37:21.568703+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:21.568703+0000","last_undegraded":"2024-04-25T12:37:21.568703+0000","last_fullsized":"2024-04-25T12:37:21.568703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.084773+0000","last_change":"2024-04-25T12:34:13.961113+0000","last_active":"2024-04-25T12:37:31.084773+0000","last_peered":"2024-04-25T12:37:31.084773+0000","last_clean":"2024-04-25T12:37:31.084773+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:31.084773+0000","last_undegraded":"2024-04-25T12:37:31.084773+0000","last_fullsized":"2024-04-25T12:37:31.084773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:34.863027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"75'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403382+0000","last_change":"2024-04-25T12:34:12.896619+0000","last_active":"2024-04-25T12:37:21.403382+0000","last_peered":"2024-04-25T12:37:21.403382+0000","last_clean":"2024-04-25T12:37:21.403382+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:21.403382+0000","last_undegraded":"2024-04-25T12:37:21.403382+0000","last_fullsized":"2024-04-25T12:37:21.403382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:10.480571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.405271+0000","last_change":"2024-04-25T12:34:18.008733+0000","last_active":"2024-04-25T12:37:21.405271+0000","last_peered":"2024-04-25T12:37:21.405271+0000","last_clean":"2024-04-25T12:37:21.405271+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:21.405271+0000","last_undegraded":"2024-04-25T12:37:21.405271+0000","last_fullsized":"2024-04-25T12:37:21.405271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":122,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.048910+0000","last_change":"2024-04-25T12:37:01.212753+0000","last_active":"2024-04-25T12:37:31.048910+0000","last_peered":"2024-04-25T12:37:31.048910+0000","last_clean":"2024-04-25T12:37:31.048910+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:31.048910+0000","last_undegraded":"2024-04-25T12:37:31.048910+0000","last_fullsized":"2024-04-25T12:37:31.048910+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:07.000299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.911689+0000","last_change":"2024-04-25T12:34:12.928104+0000","last_active":"2024-04-25T12:37:21.911689+0000","last_peered":"2024-04-25T12:37:21.911689+0000","last_clean":"2024-04-25T12:37:21.911689+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:21.911689+0000","last_undegraded":"2024-04-25T12:37:21.911689+0000","last_fullsized":"2024-04-25T12:37:21.911689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.402408+0000","last_change":"2024-04-25T12:34:18.007824+0000","last_active":"2024-04-25T12:37:21.402408+0000","last_peered":"2024-04-25T12:37:21.402408+0000","last_clean":"2024-04-25T12:37:21.402408+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:21.402408+0000","last_undegraded":"2024-04-25T12:37:21.402408+0000","last_fullsized":"2024-04-25T12:37:21.402408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:50.328051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.087322+0000","last_change":"2024-04-25T12:34:13.956077+0000","last_active":"2024-04-25T12:37:31.087322+0000","last_peered":"2024-04-25T12:37:31.087322+0000","last_clean":"2024-04-25T12:37:31.087322+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:31.087322+0000","last_undegraded":"2024-04-25T12:37:31.087322+0000","last_fullsized":"2024-04-25T12:37:31.087322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:56.555946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"70'4","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.913556+0000","last_change":"2024-04-25T12:34:12.925938+0000","last_active":"2024-04-25T12:37:21.913556+0000","last_peered":"2024-04-25T12:37:21.913556+0000","last_clean":"2024-04-25T12:37:21.913556+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:21.913556+0000","last_undegraded":"2024-04-25T12:37:21.913556+0000","last_fullsized":"2024-04-25T12:37:21.913556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568895+0000","last_change":"2024-04-25T12:34:18.006652+0000","last_active":"2024-04-25T12:37:21.568895+0000","last_peered":"2024-04-25T12:37:21.568895+0000","last_clean":"2024-04-25T12:37:21.568895+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:21.568895+0000","last_undegraded":"2024-04-25T12:37:21.568895+0000","last_fullsized":"2024-04-25T12:37:21.568895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:22.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.568918+0000","last_change":"2024-04-25T12:36:00.642290+0000","last_active":"2024-04-25T12:37:21.568918+0000","last_peered":"2024-04-25T12:37:21.568918+0000","last_clean":"2024-04-25T12:37:21.568918+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:21.568918+0000","last_undegraded":"2024-04-25T12:37:21.568918+0000","last_fullsized":"2024-04-25T12:37:21.568918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:04.462186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"93'10","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.044218+0000","last_change":"2024-04-25T12:34:13.952442+0000","last_active":"2024-04-25T12:37:31.044218+0000","last_peered":"2024-04-25T12:37:31.044218+0000","last_clean":"2024-04-25T12:37:31.044218+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:31.044218+0000","last_undegraded":"2024-04-25T12:37:31.044218+0000","last_fullsized":"2024-04-25T12:37:31.044218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:32.302931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423322+0000","last_change":"2024-04-25T12:34:12.923588+0000","last_active":"2024-04-25T12:37:21.423322+0000","last_peered":"2024-04-25T12:37:21.423322+0000","last_clean":"2024-04-25T12:37:21.423322+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:21.423322+0000","last_undegraded":"2024-04-25T12:37:21.423322+0000","last_fullsized":"2024-04-25T12:37:21.423322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:13.373996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424396+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:37:21.424396+0000","last_peered":"2024-04-25T12:37:21.424396+0000","last_clean":"2024-04-25T12:37:21.424396+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:21.424396+0000","last_undegraded":"2024-04-25T12:37:21.424396+0000","last_fullsized":"2024-04-25T12:37:21.424396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:58.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052074+0000","last_change":"2024-04-25T12:37:01.204788+0000","last_active":"2024-04-25T12:37:31.052074+0000","last_peered":"2024-04-25T12:37:31.052074+0000","last_clean":"2024-04-25T12:37:31.052074+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:31.052074+0000","last_undegraded":"2024-04-25T12:37:31.052074+0000","last_fullsized":"2024-04-25T12:37:31.052074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:14.535103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908141+0000","last_change":"2024-04-25T12:37:01.217177+0000","last_active":"2024-04-25T12:37:21.908141+0000","last_peered":"2024-04-25T12:37:21.908141+0000","last_clean":"2024-04-25T12:37:21.908141+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:21.908141+0000","last_undegraded":"2024-04-25T12:37:21.908141+0000","last_fullsized":"2024-04-25T12:37:21.908141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:00.819525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.423908+0000","last_change":"2024-04-25T12:34:12.919551+0000","last_active":"2024-04-25T12:37:21.423908+0000","last_peered":"2024-04-25T12:37:21.423908+0000","last_clean":"2024-04-25T12:37:21.423908+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:21.423908+0000","last_undegraded":"2024-04-25T12:37:21.423908+0000","last_fullsized":"2024-04-25T12:37:21.423908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:54.471716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.045592+0000","last_change":"2024-04-25T12:34:13.966795+0000","last_active":"2024-04-25T12:37:31.045592+0000","last_peered":"2024-04-25T12:37:31.045592+0000","last_clean":"2024-04-25T12:37:31.045592+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:31.045592+0000","last_undegraded":"2024-04-25T12:37:31.045592+0000","last_fullsized":"2024-04-25T12:37:31.045592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.456321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.569215+0000","last_change":"2024-04-25T12:37:01.209143+0000","last_active":"2024-04-25T12:37:21.569215+0000","last_peered":"2024-04-25T12:37:21.569215+0000","last_clean":"2024-04-25T12:37:21.569215+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:21.569215+0000","last_undegraded":"2024-04-25T12:37:21.569215+0000","last_fullsized":"2024-04-25T12:37:21.569215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:52.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.907606+0000","last_change":"2024-04-25T12:37:01.223997+0000","last_active":"2024-04-25T12:37:21.907606+0000","last_peered":"2024-04-25T12:37:21.907606+0000","last_clean":"2024-04-25T12:37:21.907606+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:21.907606+0000","last_undegraded":"2024-04-25T12:37:21.907606+0000","last_fullsized":"2024-04-25T12:37:21.907606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:48.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.064161+0000","last_change":"2024-04-25T12:36:00.638667+0000","last_active":"2024-04-25T12:37:31.064161+0000","last_peered":"2024-04-25T12:37:31.064161+0000","last_clean":"2024-04-25T12:37:31.064161+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:31.064161+0000","last_undegraded":"2024-04-25T12:37:31.064161+0000","last_fullsized":"2024-04-25T12:37:31.064161+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.403285+0000","last_change":"2024-04-25T12:36:00.642554+0000","last_active":"2024-04-25T12:37:21.403285+0000","last_peered":"2024-04-25T12:37:21.403285+0000","last_clean":"2024-04-25T12:37:21.403285+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:21.403285+0000","last_undegraded":"2024-04-25T12:37:21.403285+0000","last_fullsized":"2024-04-25T12:37:21.403285+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:01.237040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.057004+0000","last_change":"2024-04-25T12:37:01.224898+0000","last_active":"2024-04-25T12:37:31.057004+0000","last_peered":"2024-04-25T12:37:31.057004+0000","last_clean":"2024-04-25T12:37:31.057004+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:31.057004+0000","last_undegraded":"2024-04-25T12:37:31.057004+0000","last_fullsized":"2024-04-25T12:37:31.057004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:18.991070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.063908+0000","last_change":"2024-04-25T12:34:13.954642+0000","last_active":"2024-04-25T12:37:31.063908+0000","last_peered":"2024-04-25T12:37:31.063908+0000","last_clean":"2024-04-25T12:37:31.063908+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:31.063908+0000","last_undegraded":"2024-04-25T12:37:31.063908+0000","last_fullsized":"2024-04-25T12:37:31.063908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:14.394136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.079695+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:37:31.079695+0000","last_peered":"2024-04-25T12:37:31.079695+0000","last_clean":"2024-04-25T12:37:31.079695+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:31.079695+0000","last_undegraded":"2024-04-25T12:37:31.079695+0000","last_fullsized":"2024-04-25T12:37:31.079695+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:07.501710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":115,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.049065+0000","last_change":"2024-04-25T12:36:00.627066+0000","last_active":"2024-04-25T12:37:31.049065+0000","last_peered":"2024-04-25T12:37:31.049065+0000","last_clean":"2024-04-25T12:37:31.049065+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:31.049065+0000","last_undegraded":"2024-04-25T12:37:31.049065+0000","last_fullsized":"2024-04-25T12:37:31.049065+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:06.694451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.049799+0000","last_change":"2024-04-25T12:37:01.228534+0000","last_active":"2024-04-25T12:37:31.049799+0000","last_peered":"2024-04-25T12:37:31.049799+0000","last_clean":"2024-04-25T12:37:31.049799+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:31.049799+0000","last_undegraded":"2024-04-25T12:37:31.049799+0000","last_fullsized":"2024-04-25T12:37:31.049799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:39.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.041140+0000","last_change":"2024-04-25T12:37:01.210033+0000","last_active":"2024-04-25T12:37:31.041140+0000","last_peered":"2024-04-25T12:37:31.041140+0000","last_clean":"2024-04-25T12:37:31.041140+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:31.041140+0000","last_undegraded":"2024-04-25T12:37:31.041140+0000","last_fullsized":"2024-04-25T12:37:31.041140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:50.639901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.081340+0000","last_change":"2024-04-25T12:37:01.232027+0000","last_active":"2024-04-25T12:37:31.081340+0000","last_peered":"2024-04-25T12:37:31.081340+0000","last_clean":"2024-04-25T12:37:31.081340+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:31.081340+0000","last_undegraded":"2024-04-25T12:37:31.081340+0000","last_fullsized":"2024-04-25T12:37:31.081340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.606007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":110,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.063300+0000","last_change":"2024-04-25T12:36:00.630055+0000","last_active":"2024-04-25T12:37:31.063300+0000","last_peered":"2024-04-25T12:37:31.063300+0000","last_clean":"2024-04-25T12:37:31.063300+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:31.063300+0000","last_undegraded":"2024-04-25T12:37:31.063300+0000","last_fullsized":"2024-04-25T12:37:31.063300+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.113929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.086477+0000","last_change":"2024-04-25T12:36:00.635376+0000","last_active":"2024-04-25T12:37:31.086477+0000","last_peered":"2024-04-25T12:37:31.086477+0000","last_clean":"2024-04-25T12:37:31.086477+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:31.086477+0000","last_undegraded":"2024-04-25T12:37:31.086477+0000","last_fullsized":"2024-04-25T12:37:31.086477+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.024723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":116,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.057288+0000","last_change":"2024-04-25T12:36:00.620838+0000","last_active":"2024-04-25T12:37:31.057288+0000","last_peered":"2024-04-25T12:37:31.057288+0000","last_clean":"2024-04-25T12:37:31.057288+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:31.057288+0000","last_undegraded":"2024-04-25T12:37:31.057288+0000","last_fullsized":"2024-04-25T12:37:31.057288+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:52.954633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.077138+0000","last_change":"2024-04-25T12:34:13.966432+0000","last_active":"2024-04-25T12:37:31.077138+0000","last_peered":"2024-04-25T12:37:31.077138+0000","last_clean":"2024-04-25T12:37:31.077138+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:31.077138+0000","last_undegraded":"2024-04-25T12:37:31.077138+0000","last_fullsized":"2024-04-25T12:37:31.077138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.055206+0000","last_change":"2024-04-25T12:34:13.944765+0000","last_active":"2024-04-25T12:37:31.055206+0000","last_peered":"2024-04-25T12:37:31.055206+0000","last_clean":"2024-04-25T12:37:31.055206+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:31.055206+0000","last_undegraded":"2024-04-25T12:37:31.055206+0000","last_fullsized":"2024-04-25T12:37:31.055206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:14.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.076648+0000","last_change":"2024-04-25T12:34:13.975576+0000","last_active":"2024-04-25T12:37:31.076648+0000","last_peered":"2024-04-25T12:37:31.076648+0000","last_clean":"2024-04-25T12:37:31.076648+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:31.076648+0000","last_undegraded":"2024-04-25T12:37:31.076648+0000","last_fullsized":"2024-04-25T12:37:31.076648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.086103+0000","last_change":"2024-04-25T12:34:13.961245+0000","last_active":"2024-04-25T12:37:31.086103+0000","last_peered":"2024-04-25T12:37:31.086103+0000","last_clean":"2024-04-25T12:37:31.086103+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:31.086103+0000","last_undegraded":"2024-04-25T12:37:31.086103+0000","last_fullsized":"2024-04-25T12:37:31.086103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:24.329031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.060716+0000","last_change":"2024-04-25T12:34:13.942405+0000","last_active":"2024-04-25T12:37:31.060716+0000","last_peered":"2024-04-25T12:37:31.060716+0000","last_clean":"2024-04-25T12:37:31.060716+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:31.060716+0000","last_undegraded":"2024-04-25T12:37:31.060716+0000","last_fullsized":"2024-04-25T12:37:31.060716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:13.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.084587+0000","last_change":"2024-04-25T12:34:13.973112+0000","last_active":"2024-04-25T12:37:31.084587+0000","last_peered":"2024-04-25T12:37:31.084587+0000","last_clean":"2024-04-25T12:37:31.084587+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:31.084587+0000","last_undegraded":"2024-04-25T12:37:31.084587+0000","last_fullsized":"2024-04-25T12:37:31.084587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:10.861612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.086387+0000","last_change":"2024-04-25T12:34:13.950393+0000","last_active":"2024-04-25T12:37:31.086387+0000","last_peered":"2024-04-25T12:37:31.086387+0000","last_clean":"2024-04-25T12:37:31.086387+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:31.086387+0000","last_undegraded":"2024-04-25T12:37:31.086387+0000","last_fullsized":"2024-04-25T12:37:31.086387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.285408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":115,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.047825+0000","last_change":"2024-04-25T12:36:00.628357+0000","last_active":"2024-04-25T12:37:31.047825+0000","last_peered":"2024-04-25T12:37:31.047825+0000","last_clean":"2024-04-25T12:37:31.047825+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:31.047825+0000","last_undegraded":"2024-04-25T12:37:31.047825+0000","last_fullsized":"2024-04-25T12:37:31.047825+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:36:11.480853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"93'8","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.041792+0000","last_change":"2024-04-25T12:37:01.212249+0000","last_active":"2024-04-25T12:37:31.041792+0000","last_peered":"2024-04-25T12:37:31.041792+0000","last_clean":"2024-04-25T12:37:31.041792+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:31.041792+0000","last_undegraded":"2024-04-25T12:37:31.041792+0000","last_fullsized":"2024-04-25T12:37:31.041792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:47.658434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.046222+0000","last_change":"2024-04-25T12:37:01.216745+0000","last_active":"2024-04-25T12:37:31.046222+0000","last_peered":"2024-04-25T12:37:31.046222+0000","last_clean":"2024-04-25T12:37:31.046222+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:31.046222+0000","last_undegraded":"2024-04-25T12:37:31.046222+0000","last_fullsized":"2024-04-25T12:37:31.046222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:51.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.076542+0000","last_change":"2024-04-25T12:34:13.975914+0000","last_active":"2024-04-25T12:37:31.076542+0000","last_peered":"2024-04-25T12:37:31.076542+0000","last_clean":"2024-04-25T12:37:31.076542+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:31.076542+0000","last_undegraded":"2024-04-25T12:37:31.076542+0000","last_fullsized":"2024-04-25T12:37:31.076542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.069413+0000","last_change":"2024-04-25T12:34:13.976422+0000","last_active":"2024-04-25T12:37:31.069413+0000","last_peered":"2024-04-25T12:37:31.069413+0000","last_clean":"2024-04-25T12:37:31.069413+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:31.069413+0000","last_undegraded":"2024-04-25T12:37:31.069413+0000","last_fullsized":"2024-04-25T12:37:31.069413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.064410+0000","last_change":"2024-04-25T12:34:13.974640+0000","last_active":"2024-04-25T12:37:31.064410+0000","last_peered":"2024-04-25T12:37:31.064410+0000","last_clean":"2024-04-25T12:37:31.064410+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:31.064410+0000","last_undegraded":"2024-04-25T12:37:31.064410+0000","last_fullsized":"2024-04-25T12:37:31.064410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.068236+0000","last_change":"2024-04-25T12:34:13.955293+0000","last_active":"2024-04-25T12:37:31.068236+0000","last_peered":"2024-04-25T12:37:31.068236+0000","last_clean":"2024-04-25T12:37:31.068236+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:31.068236+0000","last_undegraded":"2024-04-25T12:37:31.068236+0000","last_fullsized":"2024-04-25T12:37:31.068236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":117,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.076227+0000","last_change":"2024-04-25T12:36:00.628618+0000","last_active":"2024-04-25T12:37:31.076227+0000","last_peered":"2024-04-25T12:37:31.076227+0000","last_clean":"2024-04-25T12:37:31.076227+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:31.076227+0000","last_undegraded":"2024-04-25T12:37:31.076227+0000","last_fullsized":"2024-04-25T12:37:31.076227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.866140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.044954+0000","last_change":"2024-04-25T12:34:13.958208+0000","last_active":"2024-04-25T12:37:31.044954+0000","last_peered":"2024-04-25T12:37:31.044954+0000","last_clean":"2024-04-25T12:37:31.044954+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:31.044954+0000","last_undegraded":"2024-04-25T12:37:31.044954+0000","last_fullsized":"2024-04-25T12:37:31.044954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.052307+0000","last_change":"2024-04-25T12:34:13.944156+0000","last_active":"2024-04-25T12:37:31.052307+0000","last_peered":"2024-04-25T12:37:31.052307+0000","last_clean":"2024-04-25T12:37:31.052307+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:31.052307+0000","last_undegraded":"2024-04-25T12:37:31.052307+0000","last_fullsized":"2024-04-25T12:37:31.052307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:13.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.050476+0000","last_change":"2024-04-25T12:36:00.634254+0000","last_active":"2024-04-25T12:37:31.050476+0000","last_peered":"2024-04-25T12:37:31.050476+0000","last_clean":"2024-04-25T12:37:31.050476+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:31.050476+0000","last_undegraded":"2024-04-25T12:37:31.050476+0000","last_fullsized":"2024-04-25T12:37:31.050476+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.054794+0000","last_change":"2024-04-25T12:34:13.936862+0000","last_active":"2024-04-25T12:37:31.054794+0000","last_peered":"2024-04-25T12:37:31.054794+0000","last_clean":"2024-04-25T12:37:31.054794+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:31.054794+0000","last_undegraded":"2024-04-25T12:37:31.054794+0000","last_fullsized":"2024-04-25T12:37:31.054794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:08.968133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":115,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.051429+0000","last_change":"2024-04-25T12:36:00.624054+0000","last_active":"2024-04-25T12:37:31.051429+0000","last_peered":"2024-04-25T12:37:31.051429+0000","last_clean":"2024-04-25T12:37:31.051429+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:31.051429+0000","last_undegraded":"2024-04-25T12:37:31.051429+0000","last_fullsized":"2024-04-25T12:37:31.051429+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:47.977026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":122,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.051656+0000","last_change":"2024-04-25T12:37:01.215943+0000","last_active":"2024-04-25T12:37:31.051656+0000","last_peered":"2024-04-25T12:37:31.051656+0000","last_clean":"2024-04-25T12:37:31.051656+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:31.051656+0000","last_undegraded":"2024-04-25T12:37:31.051656+0000","last_fullsized":"2024-04-25T12:37:31.051656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.518673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.054524+0000","last_change":"2024-04-25T12:34:13.980752+0000","last_active":"2024-04-25T12:37:31.054524+0000","last_peered":"2024-04-25T12:37:31.054524+0000","last_clean":"2024-04-25T12:37:31.054524+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:31.054524+0000","last_undegraded":"2024-04-25T12:37:31.054524+0000","last_fullsized":"2024-04-25T12:37:31.054524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":501,"ondisk_log_size":501,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41301,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":171,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82602,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462934,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29324,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342516,"statfs":{"total":96636764160,"available":96606736384,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1216512,"data_stored":1207576,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1142784,"data_stored":616256,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37895,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38847,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:37:42.009 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.009 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.009 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.009 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.010 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.011 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.012 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.013 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T12:34:07.013466+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=7, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.014 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.015 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.016 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.017 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.018 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.019 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.020 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.021 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.022 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:37:42.023 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T12:38:02.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:38:02.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.308+0000 7f00d3e00640 1 Processor -- start 2024-04-25T12:38:02.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.308+0000 7f00d3e00640 1 -- start start 2024-04-25T12:38:02.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d3e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f00cc0c4d50 0x7f00cc154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f00cc074870 0x7f00cc15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d3e00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00cc073ce0 con 0x7f00cc0c8c60 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d3e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f00cc0c61f0 con 0x7f00cc074870 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d3e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f00cc0c87d0 con 0x7f00cc0c4d50 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d0a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f00cc074870 0x7f00cc15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d1400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f00cc0c4d50 0x7f00cc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d0a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f00cc074870 0x7f00cc15afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38298/0 (socket says 172.21.15.123:38298) 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d0a00640 1 -- 172.21.15.123:0/1001460816 learned_addr learned my addr 172.21.15.123:0/1001460816 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.311+0000 7f00d1e00640 1 --1- 172.21.15.123:0/1001460816 >> v1:172.21.15.153:6789/0 conn(0x7f00cc0c8c60 0x7f00cc0cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:47264/0 (socket says 172.21.15.123:47264) 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.312+0000 7f00cbe00640 1 -- 172.21.15.123:0/1001460816 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888941345 0 0) 0x7f00cc073ce0 con 0x7f00cc0c8c60 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.312+0000 7f00cbe00640 1 -- 172.21.15.123:0/1001460816 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00b0003520 con 0x7f00cc0c8c60 2024-04-25T12:38:02.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.312+0000 7f00d0a00640 1 -- 172.21.15.123:0/1001460816 >> v2:172.21.15.153:3300/0 conn(0x7f00cc0c4d50 msgr2=0x7f00cc154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:02.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.312+0000 7f00d0a00640 1 --2- 172.21.15.123:0/1001460816 >> v2:172.21.15.153:3300/0 conn(0x7f00cc0c4d50 0x7f00cc154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.312+0000 7f00d0a00640 1 -- 172.21.15.123:0/1001460816 >> v1:172.21.15.153:6789/0 conn(0x7f00cc0c8c60 legacy=0x7f00cc0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:02.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.312+0000 7f00d0a00640 1 -- 172.21.15.123:0/1001460816 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc073fc0 con 0x7f00cc074870 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.313+0000 7f00d0a00640 1 --2- 172.21.15.123:0/1001460816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc074870 0x7f00cc15afa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0ce6d0 tx=0x7f00b802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc08c80f58f64fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.313+0000 7f00cbe00640 1 -- 172.21.15.123:0/1001460816 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8036070 con 0x7f00cc074870 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.313+0000 7f00cbe00640 1 -- 172.21.15.123:0/1001460816 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b8031600 con 0x7f00cc074870 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.313+0000 7f00cbe00640 1 -- 172.21.15.123:0/1001460816 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b80319b0 con 0x7f00cc074870 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 -- 172.21.15.123:0/1001460816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc074870 msgr2=0x7f00cc15afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 --2- 172.21.15.123:0/1001460816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc074870 0x7f00cc15afa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0ce6d0 tx=0x7f00b802a7c0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 -- 172.21.15.123:0/1001460816 shutdown_connections 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 --2- 172.21.15.123:0/1001460816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc074870 0x7f00cc15afa0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 --2- 172.21.15.123:0/1001460816 >> v2:172.21.15.153:3300/0 conn(0x7f00cc0c4d50 0x7f00cc154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 -- 172.21.15.123:0/1001460816 >> 172.21.15.123:0/1001460816 conn(0x7f00cc06c750 msgr2=0x7f00cc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:02.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 -- 172.21.15.123:0/1001460816 shutdown_connections 2024-04-25T12:38:02.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.314+0000 7f00d3e00640 1 -- 172.21.15.123:0/1001460816 wait complete. 2024-04-25T12:38:02.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.316+0000 7f00d3e00640 1 Processor -- start 2024-04-25T12:38:02.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.317+0000 7f00d3e00640 1 -- start start 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.318+0000 7f00d3e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f00cc074870 0x7f00cc1e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.318+0000 7f00d3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc0c4d50 0x7f00cc152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.318+0000 7f00d3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc0c8c60 0x7f00cc0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.318+0000 7f00d3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc0ce1a0 con 0x7f00cc0c8c60 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.318+0000 7f00d3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc0c8460 con 0x7f00cc0c4d50 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.318+0000 7f00d3e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f00cc0ce320 con 0x7f00cc074870 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.318+0000 7f00d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc0c8c60 0x7f00cc0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc0c8c60 0x7f00cc0ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38304/0 (socket says 172.21.15.123:38304) 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 -- 172.21.15.123:0/2159829437 learned_addr learned my addr 172.21.15.123:0/2159829437 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1400640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f00cc074870 0x7f00cc1e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d0a00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc0c4d50 0x7f00cc152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 -- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f00cc074870 msgr2=0x7f00cc1e48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:02.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f00cc074870 0x7f00cc1e48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 -- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc0c4d50 msgr2=0x7f00cc152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:02.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc0c4d50 0x7f00cc152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 -- 172.21.15.123:0/2159829437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc152a90 con 0x7f00cc0c8c60 2024-04-25T12:38:02.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00d1e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc0c8c60 0x7f00cc0ccfd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f00c4001eb0 tx=0x7f00c400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.319+0000 7f00c9600640 1 -- 172.21.15.123:0/2159829437 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c40074e0 con 0x7f00cc0c8c60 2024-04-25T12:38:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.320+0000 7f00c9600640 1 -- 172.21.15.123:0/2159829437 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c4007680 con 0x7f00cc0c8c60 2024-04-25T12:38:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.320+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc1e5260 con 0x7f00cc0c8c60 2024-04-25T12:38:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.320+0000 7f00c9600640 1 -- 172.21.15.123:0/2159829437 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4004a20 con 0x7f00cc0c8c60 2024-04-25T12:38:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.320+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1e5820 con 0x7f00cc0c8c60 2024-04-25T12:38:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.321+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc076700 con 0x7f00cc0c8c60 2024-04-25T12:38:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.322+0000 7f00c9600640 1 -- 172.21.15.123:0/2159829437 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00c401a020 con 0x7f00cc0c8c60 2024-04-25T12:38:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.324+0000 7f00c9600640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f00ac0744b0 0x7f00ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.324+0000 7f00c9600640 1 -- 172.21.15.123:0/2159829437 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f00c4094d70 con 0x7f00cc0c8c60 2024-04-25T12:38:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.324+0000 7f00d1400640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f00ac0744b0 0x7f00ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.325+0000 7f00d1400640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f00ac0744b0 0x7f00ac0769c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009aa0 tx=0x7f00bc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.329+0000 7f00c9600640 1 -- 172.21.15.123:0/2159829437 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00c405ea00 con 0x7f00cc0c8c60 2024-04-25T12:38:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.526+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00cc1e4ff0 con 0x7f00ac0744b0 2024-04-25T12:38:02.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.538+0000 7f00c9600640 1 -- 172.21.15.123:0/2159829437 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333317 (secure 0 0 0) 0x7f00cc1e4ff0 con 0x7f00ac0744b0 2024-04-25T12:38:02.542 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:38:02.544 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:38:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f00ac0744b0 msgr2=0x7f00ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f00ac0744b0 0x7f00ac0769c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009aa0 tx=0x7f00bc009190 comp rx=0 tx=0).stop 2024-04-25T12:38:02.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc0c8c60 msgr2=0x7f00cc0ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc0c8c60 0x7f00cc0ccfd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f00c4001eb0 tx=0x7f00c400c820 comp rx=0 tx=0).stop 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 shutdown_connections 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f00ac0744b0 0x7f00ac0769c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00cc0c8c60 0x7f00cc0ccfd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc0c4d50 0x7f00cc152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 --2- 172.21.15.123:0/2159829437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f00cc074870 0x7f00cc1e48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 >> 172.21.15.123:0/2159829437 conn(0x7f00cc06c750 msgr2=0x7f00cc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 shutdown_connections 2024-04-25T12:38:02.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:02.545+0000 7f00d3e00640 1 -- 172.21.15.123:0/2159829437 wait complete. 2024-04-25T12:38:02.574 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-04-25T12:38:00.851729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631669,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":274,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247024,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":238059,"kb_avail":754727696,"statfs":{"total":773094113280,"available":772841160704,"internally_reserved":0,"allocated":9093120,"data_stored":5890226,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":243773395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"2.1f","version":"93'13","reported_seq":179,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.945303+0000","last_change":"2024-04-25T12:37:41.945368+0000","last_active":"2024-04-25T12:37:41.945303+0000","last_peered":"2024-04-25T12:37:41.945303+0000","last_clean":"2024-04-25T12:37:41.945303+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:41.945303+0000","last_undegraded":"2024-04-25T12:37:41.945303+0000","last_fullsized":"2024-04-25T12:37:41.945303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'13","last_scrub_stamp":"2024-04-25T12:37:41.945200+0000","last_deep_scrub":"93'13","last_deep_scrub_stamp":"2024-04-25T12:37:41.945200+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.945200+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:39.067515+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":123,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.780946+0000","last_change":"2024-04-25T12:37:40.780946+0000","last_active":"2024-04-25T12:37:40.780946+0000","last_peered":"2024-04-25T12:37:40.780946+0000","last_clean":"2024-04-25T12:37:40.780946+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:40.780946+0000","last_undegraded":"2024-04-25T12:37:40.780946+0000","last_fullsized":"2024-04-25T12:37:40.780946+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:40.780817+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:40.780817+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.780817+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:41.153300+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.020448+0000","last_change":"2024-04-25T12:37:52.020503+0000","last_active":"2024-04-25T12:37:52.020448+0000","last_peered":"2024-04-25T12:37:52.020448+0000","last_clean":"2024-04-25T12:37:52.020448+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:52.020448+0000","last_undegraded":"2024-04-25T12:37:52.020448+0000","last_fullsized":"2024-04-25T12:37:52.020448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:52.020327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:52.020327+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.020327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:42.313726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":134,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.776120+0000","last_change":"2024-04-25T12:37:49.776120+0000","last_active":"2024-04-25T12:37:49.776120+0000","last_peered":"2024-04-25T12:37:49.776120+0000","last_clean":"2024-04-25T12:37:49.776120+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:49.776120+0000","last_undegraded":"2024-04-25T12:37:49.776120+0000","last_fullsized":"2024-04-25T12:37:49.776120+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-25T12:37:49.775941+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-25T12:37:49.775941+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.775941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:07.128554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":123,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.758609+0000","last_change":"2024-04-25T12:37:50.758680+0000","last_active":"2024-04-25T12:37:50.758609+0000","last_peered":"2024-04-25T12:37:50.758609+0000","last_clean":"2024-04-25T12:37:50.758609+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:50.758609+0000","last_undegraded":"2024-04-25T12:37:50.758609+0000","last_fullsized":"2024-04-25T12:37:50.758609+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:50.758503+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:50.758503+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.758503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:27.428308+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.596749+0000","last_change":"2024-04-25T12:37:45.596820+0000","last_active":"2024-04-25T12:37:45.596749+0000","last_peered":"2024-04-25T12:37:45.596749+0000","last_clean":"2024-04-25T12:37:45.596749+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:45.596749+0000","last_undegraded":"2024-04-25T12:37:45.596749+0000","last_fullsized":"2024-04-25T12:37:45.596749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.596535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.596535+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.596535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:29.976249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"93'7","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.125881+0000","last_change":"2024-04-25T12:37:39.125955+0000","last_active":"2024-04-25T12:37:39.125881+0000","last_peered":"2024-04-25T12:37:39.125881+0000","last_clean":"2024-04-25T12:37:39.125881+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:39.125881+0000","last_undegraded":"2024-04-25T12:37:39.125881+0000","last_fullsized":"2024-04-25T12:37:39.125881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:39.125696+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:39.125696+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.125696+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:45.706768+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.141232+0000","last_change":"2024-04-25T12:37:40.141286+0000","last_active":"2024-04-25T12:37:40.141232+0000","last_peered":"2024-04-25T12:37:40.141232+0000","last_clean":"2024-04-25T12:37:40.141232+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:40.141232+0000","last_undegraded":"2024-04-25T12:37:40.141232+0000","last_fullsized":"2024-04-25T12:37:40.141232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:40.141071+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:40.141071+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.141071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:18.869979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.591648+0000","last_change":"2024-04-25T12:37:50.591768+0000","last_active":"2024-04-25T12:37:50.591648+0000","last_peered":"2024-04-25T12:37:50.591648+0000","last_clean":"2024-04-25T12:37:50.591648+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:50.591648+0000","last_undegraded":"2024-04-25T12:37:50.591648+0000","last_fullsized":"2024-04-25T12:37:50.591648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:50.591325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:50.591325+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.591325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:51.707296+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"93'3","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:37.196340+0000","last_change":"2024-04-25T12:37:37.196393+0000","last_active":"2024-04-25T12:37:37.196340+0000","last_peered":"2024-04-25T12:37:37.196340+0000","last_clean":"2024-04-25T12:37:37.196340+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:37.196340+0000","last_undegraded":"2024-04-25T12:37:37.196340+0000","last_fullsized":"2024-04-25T12:37:37.196340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:37.196244+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:37.196244+0000","last_clean_scrub_stamp":"2024-04-25T12:37:37.196244+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:49.970014+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.341184+0000","last_change":"2024-04-25T12:37:43.341250+0000","last_active":"2024-04-25T12:37:43.341184+0000","last_peered":"2024-04-25T12:37:43.341184+0000","last_clean":"2024-04-25T12:37:43.341184+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:43.341184+0000","last_undegraded":"2024-04-25T12:37:43.341184+0000","last_fullsized":"2024-04-25T12:37:43.341184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'15","last_scrub_stamp":"2024-04-25T12:37:43.340987+0000","last_deep_scrub":"93'15","last_deep_scrub_stamp":"2024-04-25T12:37:43.340987+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.340987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.966156+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.636763+0000","last_change":"2024-04-25T12:37:46.636834+0000","last_active":"2024-04-25T12:37:46.636763+0000","last_peered":"2024-04-25T12:37:46.636763+0000","last_clean":"2024-04-25T12:37:46.636763+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:46.636763+0000","last_undegraded":"2024-04-25T12:37:46.636763+0000","last_fullsized":"2024-04-25T12:37:46.636763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:46.636538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:46.636538+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.636538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:40.507596+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.301675+0000","last_change":"2024-04-25T12:37:39.301744+0000","last_active":"2024-04-25T12:37:39.301675+0000","last_peered":"2024-04-25T12:37:39.301675+0000","last_clean":"2024-04-25T12:37:39.301675+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:39.301675+0000","last_undegraded":"2024-04-25T12:37:39.301675+0000","last_fullsized":"2024-04-25T12:37:39.301675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:39.301505+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:39.301505+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.301505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:59.601828+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"93'7","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.308766+0000","last_change":"2024-04-25T12:37:53.308898+0000","last_active":"2024-04-25T12:37:53.308766+0000","last_peered":"2024-04-25T12:37:53.308766+0000","last_clean":"2024-04-25T12:37:53.308766+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:53.308766+0000","last_undegraded":"2024-04-25T12:37:53.308766+0000","last_fullsized":"2024-04-25T12:37:53.308766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:53.308530+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:53.308530+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.308530+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:06.361349+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.157193+0000","last_change":"2024-04-25T12:37:49.157260+0000","last_active":"2024-04-25T12:37:49.157193+0000","last_peered":"2024-04-25T12:37:49.157193+0000","last_clean":"2024-04-25T12:37:49.157193+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:49.157193+0000","last_undegraded":"2024-04-25T12:37:49.157193+0000","last_fullsized":"2024-04-25T12:37:49.157193+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.156932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.156932+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.156932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:34.008352+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.034372+0000","last_change":"2024-04-25T12:37:51.034402+0000","last_active":"2024-04-25T12:37:51.034372+0000","last_peered":"2024-04-25T12:37:51.034372+0000","last_clean":"2024-04-25T12:37:51.034372+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:51.034372+0000","last_undegraded":"2024-04-25T12:37:51.034372+0000","last_fullsized":"2024-04-25T12:37:51.034372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:51.034270+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:51.034270+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.034270+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:36.004724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.670764+0000","last_change":"2024-04-25T12:37:53.670806+0000","last_active":"2024-04-25T12:37:53.670764+0000","last_peered":"2024-04-25T12:37:53.670764+0000","last_clean":"2024-04-25T12:37:53.670764+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:53.670764+0000","last_undegraded":"2024-04-25T12:37:53.670764+0000","last_fullsized":"2024-04-25T12:37:53.670764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:53.670672+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:53.670672+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.670672+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:38.153329+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.177897+0000","last_change":"2024-04-25T12:37:50.178033+0000","last_active":"2024-04-25T12:37:50.177897+0000","last_peered":"2024-04-25T12:37:50.177897+0000","last_clean":"2024-04-25T12:37:50.177897+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:50.177897+0000","last_undegraded":"2024-04-25T12:37:50.177897+0000","last_fullsized":"2024-04-25T12:37:50.177897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:50.177608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:50.177608+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.177608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:08.619291+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.973607+0000","last_change":"2024-04-25T12:37:44.973638+0000","last_active":"2024-04-25T12:37:44.973607+0000","last_peered":"2024-04-25T12:37:44.973607+0000","last_clean":"2024-04-25T12:37:44.973607+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:44.973607+0000","last_undegraded":"2024-04-25T12:37:44.973607+0000","last_fullsized":"2024-04-25T12:37:44.973607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:44.973505+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:44.973505+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.973505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:41.866275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"93'5","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.935860+0000","last_change":"2024-04-25T12:37:43.935898+0000","last_active":"2024-04-25T12:37:43.935860+0000","last_peered":"2024-04-25T12:37:43.935860+0000","last_clean":"2024-04-25T12:37:43.935860+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:43.935860+0000","last_undegraded":"2024-04-25T12:37:43.935860+0000","last_fullsized":"2024-04-25T12:37:43.935860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:43.935726+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:43.935726+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.935726+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:28.428844+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.977659+0000","last_change":"2024-04-25T12:37:46.977689+0000","last_active":"2024-04-25T12:37:46.977659+0000","last_peered":"2024-04-25T12:37:46.977659+0000","last_clean":"2024-04-25T12:37:46.977659+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:46.977659+0000","last_undegraded":"2024-04-25T12:37:46.977659+0000","last_fullsized":"2024-04-25T12:37:46.977659+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:46.977559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:46.977559+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.977559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.016158+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.318964+0000","last_change":"2024-04-25T12:37:40.319013+0000","last_active":"2024-04-25T12:37:40.318964+0000","last_peered":"2024-04-25T12:37:40.318964+0000","last_clean":"2024-04-25T12:37:40.318964+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:40.318964+0000","last_undegraded":"2024-04-25T12:37:40.318964+0000","last_fullsized":"2024-04-25T12:37:40.318964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:40.318837+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:40.318837+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.318837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:30.379834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":93,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.422495+0000","last_change":"2024-04-25T12:36:13.820183+0000","last_active":"2024-04-25T12:37:21.422495+0000","last_peered":"2024-04-25T12:37:21.422495+0000","last_clean":"2024-04-25T12:37:21.422495+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:37:21.422495+0000","last_undegraded":"2024-04-25T12:37:21.422495+0000","last_fullsized":"2024-04-25T12:37:21.422495+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.534658+0000","last_change":"2024-04-25T12:37:42.534730+0000","last_active":"2024-04-25T12:37:42.534658+0000","last_peered":"2024-04-25T12:37:42.534658+0000","last_clean":"2024-04-25T12:37:42.534658+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:42.534658+0000","last_undegraded":"2024-04-25T12:37:42.534658+0000","last_fullsized":"2024-04-25T12:37:42.534658+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:42.534404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:42.534404+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.534404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:27.027522+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.985108+0000","last_change":"2024-04-25T12:37:45.985136+0000","last_active":"2024-04-25T12:37:45.985108+0000","last_peered":"2024-04-25T12:37:45.985108+0000","last_clean":"2024-04-25T12:37:45.985108+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:45.985108+0000","last_undegraded":"2024-04-25T12:37:45.985108+0000","last_fullsized":"2024-04-25T12:37:45.985108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:45.985017+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:45.985017+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.985017+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:50.743875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.300758+0000","last_change":"2024-04-25T12:37:50.300835+0000","last_active":"2024-04-25T12:37:50.300758+0000","last_peered":"2024-04-25T12:37:50.300758+0000","last_clean":"2024-04-25T12:37:50.300758+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:50.300758+0000","last_undegraded":"2024-04-25T12:37:50.300758+0000","last_fullsized":"2024-04-25T12:37:50.300758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:50.300483+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:50.300483+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.300483+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:48.164030+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.616920+0000","last_change":"2024-04-25T12:37:49.616986+0000","last_active":"2024-04-25T12:37:49.616920+0000","last_peered":"2024-04-25T12:37:49.616920+0000","last_clean":"2024-04-25T12:37:49.616920+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:49.616920+0000","last_undegraded":"2024-04-25T12:37:49.616920+0000","last_fullsized":"2024-04-25T12:37:49.616920+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.616716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.616716+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.616716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:12.195154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":179,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.005935+0000","last_change":"2024-04-25T12:37:54.005963+0000","last_active":"2024-04-25T12:37:54.005935+0000","last_peered":"2024-04-25T12:37:54.005935+0000","last_clean":"2024-04-25T12:37:54.005935+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:54.005935+0000","last_undegraded":"2024-04-25T12:37:54.005935+0000","last_fullsized":"2024-04-25T12:37:54.005935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:54.005836+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:54.005836+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.005836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:26.054516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"93'14","reported_seq":180,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.919619+0000","last_change":"2024-04-25T12:37:42.919705+0000","last_active":"2024-04-25T12:37:42.919619+0000","last_peered":"2024-04-25T12:37:42.919619+0000","last_clean":"2024-04-25T12:37:42.919619+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:42.919619+0000","last_undegraded":"2024-04-25T12:37:42.919619+0000","last_fullsized":"2024-04-25T12:37:42.919619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'14","last_scrub_stamp":"2024-04-25T12:37:42.919493+0000","last_deep_scrub":"93'14","last_deep_scrub_stamp":"2024-04-25T12:37:42.919493+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.919493+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:21.298661+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:56.007256+0000","last_change":"2024-04-25T12:37:56.007299+0000","last_active":"2024-04-25T12:37:56.007256+0000","last_peered":"2024-04-25T12:37:56.007256+0000","last_clean":"2024-04-25T12:37:56.007256+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:56.007256+0000","last_undegraded":"2024-04-25T12:37:56.007256+0000","last_fullsized":"2024-04-25T12:37:56.007256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:56.007156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:56.007156+0000","last_clean_scrub_stamp":"2024-04-25T12:37:56.007156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:32.960994+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.842873+0000","last_change":"2024-04-25T12:37:40.842901+0000","last_active":"2024-04-25T12:37:40.842873+0000","last_peered":"2024-04-25T12:37:40.842873+0000","last_clean":"2024-04-25T12:37:40.842873+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:40.842873+0000","last_undegraded":"2024-04-25T12:37:40.842873+0000","last_fullsized":"2024-04-25T12:37:40.842873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:40.842764+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:40.842764+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.842764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:58.351731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'2","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.327279+0000","last_change":"2024-04-25T12:37:47.327349+0000","last_active":"2024-04-25T12:37:47.327279+0000","last_peered":"2024-04-25T12:37:47.327279+0000","last_clean":"2024-04-25T12:37:47.327279+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:47.327279+0000","last_undegraded":"2024-04-25T12:37:47.327279+0000","last_fullsized":"2024-04-25T12:37:47.327279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:47.327082+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:47.327082+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.327082+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:42.666016+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.791015+0000","last_change":"2024-04-25T12:37:47.791046+0000","last_active":"2024-04-25T12:37:47.791015+0000","last_peered":"2024-04-25T12:37:47.791015+0000","last_clean":"2024-04-25T12:37:47.791015+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:47.791015+0000","last_undegraded":"2024-04-25T12:37:47.791015+0000","last_fullsized":"2024-04-25T12:37:47.791015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:47.790902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:47.790902+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.790902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.170616+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":177,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.629887+0000","last_change":"2024-04-25T12:37:47.630014+0000","last_active":"2024-04-25T12:37:47.629887+0000","last_peered":"2024-04-25T12:37:47.629887+0000","last_clean":"2024-04-25T12:37:47.629887+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:47.629887+0000","last_undegraded":"2024-04-25T12:37:47.629887+0000","last_fullsized":"2024-04-25T12:37:47.629887+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:47.629654+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:47.629654+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.629654+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:42.007123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":127,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.749393+0000","last_change":"2024-04-25T12:37:52.749427+0000","last_active":"2024-04-25T12:37:52.749393+0000","last_peered":"2024-04-25T12:37:52.749393+0000","last_clean":"2024-04-25T12:37:52.749393+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:52.749393+0000","last_undegraded":"2024-04-25T12:37:52.749393+0000","last_fullsized":"2024-04-25T12:37:52.749393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:52.749288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:52.749288+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.749288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:25.097466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.717046+0000","last_change":"2024-04-25T12:37:52.717078+0000","last_active":"2024-04-25T12:37:52.717046+0000","last_peered":"2024-04-25T12:37:52.717046+0000","last_clean":"2024-04-25T12:37:52.717046+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:52.717046+0000","last_undegraded":"2024-04-25T12:37:52.717046+0000","last_fullsized":"2024-04-25T12:37:52.717046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:52.716935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:52.716935+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.716935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:20.052004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.974312+0000","last_change":"2024-04-25T12:37:52.974344+0000","last_active":"2024-04-25T12:37:52.974312+0000","last_peered":"2024-04-25T12:37:52.974312+0000","last_clean":"2024-04-25T12:37:52.974312+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:52.974312+0000","last_undegraded":"2024-04-25T12:37:52.974312+0000","last_fullsized":"2024-04-25T12:37:52.974312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:52.974188+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:52.974188+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.974188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:51.443245+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424294+0000","last_change":"2024-04-25T12:37:01.227566+0000","last_active":"2024-04-25T12:37:21.424294+0000","last_peered":"2024-04-25T12:37:21.424294+0000","last_clean":"2024-04-25T12:37:21.424294+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:37:21.424294+0000","last_undegraded":"2024-04-25T12:37:21.424294+0000","last_fullsized":"2024-04-25T12:37:21.424294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:11.863765+0000","last_clean_scrub_stamp":"2024-04-25T12:34:11.863765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.733293+0000","last_change":"2024-04-25T12:37:49.733321+0000","last_active":"2024-04-25T12:37:49.733293+0000","last_peered":"2024-04-25T12:37:49.733293+0000","last_clean":"2024-04-25T12:37:49.733293+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:49.733293+0000","last_undegraded":"2024-04-25T12:37:49.733293+0000","last_fullsized":"2024-04-25T12:37:49.733293+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.733195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.733195+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.733195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:48.202252+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.875867+0000","last_change":"2024-04-25T12:37:44.875867+0000","last_active":"2024-04-25T12:37:44.875867+0000","last_peered":"2024-04-25T12:37:44.875867+0000","last_clean":"2024-04-25T12:37:44.875867+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:44.875867+0000","last_undegraded":"2024-04-25T12:37:44.875867+0000","last_fullsized":"2024-04-25T12:37:44.875867+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:44.875712+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:44.875712+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.875712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:13.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'10","reported_seq":185,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.010739+0000","last_change":"2024-04-25T12:37:48.010778+0000","last_active":"2024-04-25T12:37:48.010739+0000","last_peered":"2024-04-25T12:37:48.010739+0000","last_clean":"2024-04-25T12:37:48.010739+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:48.010739+0000","last_undegraded":"2024-04-25T12:37:48.010739+0000","last_fullsized":"2024-04-25T12:37:48.010739+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:48.010646+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:48.010646+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.010646+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:50.803361+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.823843+0000","last_change":"2024-04-25T12:37:45.823893+0000","last_active":"2024-04-25T12:37:45.823843+0000","last_peered":"2024-04-25T12:37:45.823843+0000","last_clean":"2024-04-25T12:37:45.823843+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:45.823843+0000","last_undegraded":"2024-04-25T12:37:45.823843+0000","last_fullsized":"2024-04-25T12:37:45.823843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.823677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.823677+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.823677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:15.139977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.810197+0000","last_change":"2024-04-25T12:37:46.810244+0000","last_active":"2024-04-25T12:37:46.810197+0000","last_peered":"2024-04-25T12:37:46.810197+0000","last_clean":"2024-04-25T12:37:46.810197+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:46.810197+0000","last_undegraded":"2024-04-25T12:37:46.810197+0000","last_fullsized":"2024-04-25T12:37:46.810197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:46.810016+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:46.810016+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.810016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:10.032062+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'4","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:57.287323+0000","last_change":"2024-04-25T12:37:57.287396+0000","last_active":"2024-04-25T12:37:57.287323+0000","last_peered":"2024-04-25T12:37:57.287323+0000","last_clean":"2024-04-25T12:37:57.287323+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:57.287323+0000","last_undegraded":"2024-04-25T12:37:57.287323+0000","last_fullsized":"2024-04-25T12:37:57.287323+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:57.287131+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:57.287131+0000","last_clean_scrub_stamp":"2024-04-25T12:37:57.287131+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:32.753341+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:57.816771+0000","last_change":"2024-04-25T12:37:57.816832+0000","last_active":"2024-04-25T12:37:57.816771+0000","last_peered":"2024-04-25T12:37:57.816771+0000","last_clean":"2024-04-25T12:37:57.816771+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:57.816771+0000","last_undegraded":"2024-04-25T12:37:57.816771+0000","last_fullsized":"2024-04-25T12:37:57.816771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:57.816623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:57.816623+0000","last_clean_scrub_stamp":"2024-04-25T12:37:57.816623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:43.057242+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.897876+0000","last_change":"2024-04-25T12:37:39.897933+0000","last_active":"2024-04-25T12:37:39.897876+0000","last_peered":"2024-04-25T12:37:39.897876+0000","last_clean":"2024-04-25T12:37:39.897876+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:39.897876+0000","last_undegraded":"2024-04-25T12:37:39.897876+0000","last_fullsized":"2024-04-25T12:37:39.897876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:39.897724+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:39.897724+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.897724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:10.301458+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.277747+0000","last_change":"2024-04-25T12:37:51.277832+0000","last_active":"2024-04-25T12:37:51.277747+0000","last_peered":"2024-04-25T12:37:51.277747+0000","last_clean":"2024-04-25T12:37:51.277747+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:51.277747+0000","last_undegraded":"2024-04-25T12:37:51.277747+0000","last_fullsized":"2024-04-25T12:37:51.277747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:51.277541+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:51.277541+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.277541+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:34.828234+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"93'74","reported_seq":273,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.161278+0000","last_change":"2024-04-25T12:37:41.161333+0000","last_active":"2024-04-25T12:37:41.161278+0000","last_peered":"2024-04-25T12:37:41.161278+0000","last_clean":"2024-04-25T12:37:41.161278+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:41.161278+0000","last_undegraded":"2024-04-25T12:37:41.161278+0000","last_fullsized":"2024-04-25T12:37:41.161278+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'74","last_scrub_stamp":"2024-04-25T12:37:41.161161+0000","last_deep_scrub":"93'74","last_deep_scrub_stamp":"2024-04-25T12:37:41.161161+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.161161+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:29.238865+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:55.297668+0000","last_change":"2024-04-25T12:37:55.297736+0000","last_active":"2024-04-25T12:37:55.297668+0000","last_peered":"2024-04-25T12:37:55.297668+0000","last_clean":"2024-04-25T12:37:55.297668+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:55.297668+0000","last_undegraded":"2024-04-25T12:37:55.297668+0000","last_fullsized":"2024-04-25T12:37:55.297668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:55.297420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:55.297420+0000","last_clean_scrub_stamp":"2024-04-25T12:37:55.297420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:54.025381+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.277780+0000","last_change":"2024-04-25T12:37:38.277843+0000","last_active":"2024-04-25T12:37:38.277780+0000","last_peered":"2024-04-25T12:37:38.277780+0000","last_clean":"2024-04-25T12:37:38.277780+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:38.277780+0000","last_undegraded":"2024-04-25T12:37:38.277780+0000","last_fullsized":"2024-04-25T12:37:38.277780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:38.277623+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:38.277623+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.277623+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:43.367276+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"93'5","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.716658+0000","last_change":"2024-04-25T12:37:54.716713+0000","last_active":"2024-04-25T12:37:54.716658+0000","last_peered":"2024-04-25T12:37:54.716658+0000","last_clean":"2024-04-25T12:37:54.716658+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:54.716658+0000","last_undegraded":"2024-04-25T12:37:54.716658+0000","last_fullsized":"2024-04-25T12:37:54.716658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:54.716491+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:54.716491+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.716491+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:13.341408+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.561279+0000","last_change":"2024-04-25T12:37:44.561407+0000","last_active":"2024-04-25T12:37:44.561279+0000","last_peered":"2024-04-25T12:37:44.561279+0000","last_clean":"2024-04-25T12:37:44.561279+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:44.561279+0000","last_undegraded":"2024-04-25T12:37:44.561279+0000","last_fullsized":"2024-04-25T12:37:44.561279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:44.561040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:44.561040+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.561040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:49.319898+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.030775+0000","last_change":"2024-04-25T12:37:46.030848+0000","last_active":"2024-04-25T12:37:46.030775+0000","last_peered":"2024-04-25T12:37:46.030775+0000","last_clean":"2024-04-25T12:37:46.030775+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:46.030775+0000","last_undegraded":"2024-04-25T12:37:46.030775+0000","last_fullsized":"2024-04-25T12:37:46.030775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:46.030579+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:46.030579+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.030579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:07.934799+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"93'2","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.788022+0000","last_change":"2024-04-25T12:37:39.788056+0000","last_active":"2024-04-25T12:37:39.788022+0000","last_peered":"2024-04-25T12:37:39.788022+0000","last_clean":"2024-04-25T12:37:39.788022+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:39.788022+0000","last_undegraded":"2024-04-25T12:37:39.788022+0000","last_fullsized":"2024-04-25T12:37:39.788022+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:39.787933+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:39.787933+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.787933+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:15.167038+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.819523+0000","last_change":"2024-04-25T12:37:41.819553+0000","last_active":"2024-04-25T12:37:41.819523+0000","last_peered":"2024-04-25T12:37:41.819523+0000","last_clean":"2024-04-25T12:37:41.819523+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:41.819523+0000","last_undegraded":"2024-04-25T12:37:41.819523+0000","last_fullsized":"2024-04-25T12:37:41.819523+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:41.819413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:41.819413+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.819413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:25.906731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.109699+0000","last_change":"2024-04-25T12:37:41.109768+0000","last_active":"2024-04-25T12:37:41.109699+0000","last_peered":"2024-04-25T12:37:41.109699+0000","last_clean":"2024-04-25T12:37:41.109699+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:41.109699+0000","last_undegraded":"2024-04-25T12:37:41.109699+0000","last_fullsized":"2024-04-25T12:37:41.109699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:41.109493+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:41.109493+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.109493+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:28.193763+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"93'3","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.036750+0000","last_change":"2024-04-25T12:37:51.036791+0000","last_active":"2024-04-25T12:37:51.036750+0000","last_peered":"2024-04-25T12:37:51.036750+0000","last_clean":"2024-04-25T12:37:51.036750+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:51.036750+0000","last_undegraded":"2024-04-25T12:37:51.036750+0000","last_fullsized":"2024-04-25T12:37:51.036750+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:51.036648+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:51.036648+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.036648+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:13.615087+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.312595+0000","last_change":"2024-04-25T12:37:45.312666+0000","last_active":"2024-04-25T12:37:45.312595+0000","last_peered":"2024-04-25T12:37:45.312595+0000","last_clean":"2024-04-25T12:37:45.312595+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:45.312595+0000","last_undegraded":"2024-04-25T12:37:45.312595+0000","last_fullsized":"2024-04-25T12:37:45.312595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.312351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.312351+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.312351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:30.464269+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.528921+0000","last_change":"2024-04-25T12:37:41.528954+0000","last_active":"2024-04-25T12:37:41.528921+0000","last_peered":"2024-04-25T12:37:41.528921+0000","last_clean":"2024-04-25T12:37:41.528921+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:41.528921+0000","last_undegraded":"2024-04-25T12:37:41.528921+0000","last_fullsized":"2024-04-25T12:37:41.528921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:41.528815+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:41.528815+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.528815+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:19.907983+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"93'4","reported_seq":175,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.189153+0000","last_change":"2024-04-25T12:37:47.189233+0000","last_active":"2024-04-25T12:37:47.189153+0000","last_peered":"2024-04-25T12:37:47.189153+0000","last_clean":"2024-04-25T12:37:47.189153+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:47.189153+0000","last_undegraded":"2024-04-25T12:37:47.189153+0000","last_fullsized":"2024-04-25T12:37:47.189153+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:47.188957+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:47.188957+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.188957+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:37.249954+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.069928+0000","last_change":"2024-04-25T12:37:44.069994+0000","last_active":"2024-04-25T12:37:44.069928+0000","last_peered":"2024-04-25T12:37:44.069928+0000","last_clean":"2024-04-25T12:37:44.069928+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:44.069928+0000","last_undegraded":"2024-04-25T12:37:44.069928+0000","last_fullsized":"2024-04-25T12:37:44.069928+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:44.069727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:44.069727+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.069727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:32.356013+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.319568+0000","last_change":"2024-04-25T12:37:46.319692+0000","last_active":"2024-04-25T12:37:46.319568+0000","last_peered":"2024-04-25T12:37:46.319568+0000","last_clean":"2024-04-25T12:37:46.319568+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:46.319568+0000","last_undegraded":"2024-04-25T12:37:46.319568+0000","last_fullsized":"2024-04-25T12:37:46.319568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:46.319236+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:46.319236+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.319236+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:30.551780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"93'24","reported_seq":190,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.915728+0000","last_change":"2024-04-25T12:37:40.915776+0000","last_active":"2024-04-25T12:37:40.915728+0000","last_peered":"2024-04-25T12:37:40.915728+0000","last_clean":"2024-04-25T12:37:40.915728+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:40.915728+0000","last_undegraded":"2024-04-25T12:37:40.915728+0000","last_fullsized":"2024-04-25T12:37:40.915728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'24","last_scrub_stamp":"2024-04-25T12:37:40.915632+0000","last_deep_scrub":"93'24","last_deep_scrub_stamp":"2024-04-25T12:37:40.915632+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.915632+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:49.082187+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":127,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.908310+0000","last_change":"2024-04-25T12:37:01.226590+0000","last_active":"2024-04-25T12:37:21.908310+0000","last_peered":"2024-04-25T12:37:21.908310+0000","last_clean":"2024-04-25T12:37:21.908310+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:21.908310+0000","last_undegraded":"2024-04-25T12:37:21.908310+0000","last_fullsized":"2024-04-25T12:37:21.908310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":175,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.112291+0000","last_change":"2024-04-25T12:37:42.112357+0000","last_active":"2024-04-25T12:37:42.112291+0000","last_peered":"2024-04-25T12:37:42.112291+0000","last_clean":"2024-04-25T12:37:42.112291+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:42.112291+0000","last_undegraded":"2024-04-25T12:37:42.112291+0000","last_fullsized":"2024-04-25T12:37:42.112291+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:42.112085+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:42.112085+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.112085+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:47.435031+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:36.180578+0000","last_change":"2024-04-25T12:37:36.180654+0000","last_active":"2024-04-25T12:37:36.180578+0000","last_peered":"2024-04-25T12:37:36.180578+0000","last_clean":"2024-04-25T12:37:36.180578+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:36.180578+0000","last_undegraded":"2024-04-25T12:37:36.180578+0000","last_fullsized":"2024-04-25T12:37:36.180578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:36.180353+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:36.180353+0000","last_clean_scrub_stamp":"2024-04-25T12:37:36.180353+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:38.779021+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:55.812440+0000","last_change":"2024-04-25T12:37:55.812491+0000","last_active":"2024-04-25T12:37:55.812440+0000","last_peered":"2024-04-25T12:37:55.812440+0000","last_clean":"2024-04-25T12:37:55.812440+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:55.812440+0000","last_undegraded":"2024-04-25T12:37:55.812440+0000","last_fullsized":"2024-04-25T12:37:55.812440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:55.812291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:55.812291+0000","last_clean_scrub_stamp":"2024-04-25T12:37:55.812291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:56.611591+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.090297+0000","last_change":"2024-04-25T12:37:01.233439+0000","last_active":"2024-04-25T12:37:31.090297+0000","last_peered":"2024-04-25T12:37:31.090297+0000","last_clean":"2024-04-25T12:37:31.090297+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:37:31.090297+0000","last_undegraded":"2024-04-25T12:37:31.090297+0000","last_fullsized":"2024-04-25T12:37:31.090297+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"93'5","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.326435+0000","last_change":"2024-04-25T12:37:48.326543+0000","last_active":"2024-04-25T12:37:48.326435+0000","last_peered":"2024-04-25T12:37:48.326435+0000","last_clean":"2024-04-25T12:37:48.326435+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:48.326435+0000","last_undegraded":"2024-04-25T12:37:48.326435+0000","last_fullsized":"2024-04-25T12:37:48.326435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:48.326251+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:48.326251+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.326251+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:57.118982+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.744240+0000","last_change":"2024-04-25T12:37:48.744269+0000","last_active":"2024-04-25T12:37:48.744240+0000","last_peered":"2024-04-25T12:37:48.744240+0000","last_clean":"2024-04-25T12:37:48.744240+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:48.744240+0000","last_undegraded":"2024-04-25T12:37:48.744240+0000","last_fullsized":"2024-04-25T12:37:48.744240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:48.744135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:48.744135+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.744135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:39.037534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.707884+0000","last_change":"2024-04-25T12:37:51.707916+0000","last_active":"2024-04-25T12:37:51.707884+0000","last_peered":"2024-04-25T12:37:51.707884+0000","last_clean":"2024-04-25T12:37:51.707884+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:51.707884+0000","last_undegraded":"2024-04-25T12:37:51.707884+0000","last_fullsized":"2024-04-25T12:37:51.707884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:51.707748+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:51.707748+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.707748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.273796+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'10","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.909253+0000","last_change":"2024-04-25T12:37:42.909291+0000","last_active":"2024-04-25T12:37:42.909253+0000","last_peered":"2024-04-25T12:37:42.909253+0000","last_clean":"2024-04-25T12:37:42.909253+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:42.909253+0000","last_undegraded":"2024-04-25T12:37:42.909253+0000","last_fullsized":"2024-04-25T12:37:42.909253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:42.909147+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:42.909147+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.909147+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:55.844694+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":82,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.065122+0000","last_change":"2024-04-25T12:37:45.065190+0000","last_active":"2024-04-25T12:37:45.065122+0000","last_peered":"2024-04-25T12:37:45.065122+0000","last_clean":"2024-04-25T12:37:45.065122+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:45.065122+0000","last_undegraded":"2024-04-25T12:37:45.065122+0000","last_fullsized":"2024-04-25T12:37:45.065122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.064846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.064846+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.064846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:58.000393+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.041998+0000","last_change":"2024-04-25T12:37:50.042049+0000","last_active":"2024-04-25T12:37:50.041998+0000","last_peered":"2024-04-25T12:37:50.041998+0000","last_clean":"2024-04-25T12:37:50.041998+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:50.041998+0000","last_undegraded":"2024-04-25T12:37:50.041998+0000","last_fullsized":"2024-04-25T12:37:50.041998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:50.041892+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:50.041892+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.041892+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:13.263938+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"93'32","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:37.480272+0000","last_change":"2024-04-25T12:37:37.480323+0000","last_active":"2024-04-25T12:37:37.480272+0000","last_peered":"2024-04-25T12:37:37.480272+0000","last_clean":"2024-04-25T12:37:37.480272+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:37.480272+0000","last_undegraded":"2024-04-25T12:37:37.480272+0000","last_fullsized":"2024-04-25T12:37:37.480272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'32","last_scrub_stamp":"2024-04-25T12:37:37.480157+0000","last_deep_scrub":"93'32","last_deep_scrub_stamp":"2024-04-25T12:37:37.480157+0000","last_clean_scrub_stamp":"2024-04-25T12:37:37.480157+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:09.030080+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37279,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.011086+0000","last_change":"2024-04-25T12:37:52.011119+0000","last_active":"2024-04-25T12:37:52.011086+0000","last_peered":"2024-04-25T12:37:52.011086+0000","last_clean":"2024-04-25T12:37:52.011086+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:52.011086+0000","last_undegraded":"2024-04-25T12:37:52.011086+0000","last_fullsized":"2024-04-25T12:37:52.011086+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'3","last_scrub_stamp":"2024-04-25T12:37:52.010966+0000","last_deep_scrub":"70'3","last_deep_scrub_stamp":"2024-04-25T12:37:52.010966+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.010966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:53.126937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.297625+0000","last_change":"2024-04-25T12:37:42.297763+0000","last_active":"2024-04-25T12:37:42.297625+0000","last_peered":"2024-04-25T12:37:42.297625+0000","last_clean":"2024-04-25T12:37:42.297625+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:42.297625+0000","last_undegraded":"2024-04-25T12:37:42.297625+0000","last_fullsized":"2024-04-25T12:37:42.297625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'11","last_scrub_stamp":"2024-04-25T12:37:42.297357+0000","last_deep_scrub":"93'11","last_deep_scrub_stamp":"2024-04-25T12:37:42.297357+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.297357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:27.148777+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"93'10","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.954256+0000","last_change":"2024-04-25T12:37:41.954308+0000","last_active":"2024-04-25T12:37:41.954256+0000","last_peered":"2024-04-25T12:37:41.954256+0000","last_clean":"2024-04-25T12:37:41.954256+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:41.954256+0000","last_undegraded":"2024-04-25T12:37:41.954256+0000","last_fullsized":"2024-04-25T12:37:41.954256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:41.954139+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:41.954139+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.954139+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:30.240425+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.565512+0000","last_change":"2024-04-25T12:37:43.565587+0000","last_active":"2024-04-25T12:37:43.565512+0000","last_peered":"2024-04-25T12:37:43.565512+0000","last_clean":"2024-04-25T12:37:43.565512+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:43.565512+0000","last_undegraded":"2024-04-25T12:37:43.565512+0000","last_fullsized":"2024-04-25T12:37:43.565512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:43.565246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:43.565246+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.565246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:40.458016+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.771863+0000","last_change":"2024-04-25T12:37:53.771912+0000","last_active":"2024-04-25T12:37:53.771863+0000","last_peered":"2024-04-25T12:37:53.771863+0000","last_clean":"2024-04-25T12:37:53.771863+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:53.771863+0000","last_undegraded":"2024-04-25T12:37:53.771863+0000","last_fullsized":"2024-04-25T12:37:53.771863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:53.771727+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:53.771727+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.771727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:22.048282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"93'3","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.147359+0000","last_change":"2024-04-25T12:37:38.147400+0000","last_active":"2024-04-25T12:37:38.147359+0000","last_peered":"2024-04-25T12:37:38.147359+0000","last_clean":"2024-04-25T12:37:38.147359+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:38.147359+0000","last_undegraded":"2024-04-25T12:37:38.147359+0000","last_fullsized":"2024-04-25T12:37:38.147359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:38.147245+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:38.147245+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.147245+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:23.765389+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.956066+0000","last_change":"2024-04-25T12:37:45.956095+0000","last_active":"2024-04-25T12:37:45.956066+0000","last_peered":"2024-04-25T12:37:45.956066+0000","last_clean":"2024-04-25T12:37:45.956066+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:45.956066+0000","last_undegraded":"2024-04-25T12:37:45.956066+0000","last_fullsized":"2024-04-25T12:37:45.956066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.955956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.955956+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.955956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:38.246257+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.557493+0000","last_change":"2024-04-25T12:37:40.557550+0000","last_active":"2024-04-25T12:37:40.557493+0000","last_peered":"2024-04-25T12:37:40.557493+0000","last_clean":"2024-04-25T12:37:40.557493+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:40.557493+0000","last_undegraded":"2024-04-25T12:37:40.557493+0000","last_fullsized":"2024-04-25T12:37:40.557493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:40.557352+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:40.557352+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.557352+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:37.679667+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:37.060297+0000","last_change":"2024-04-25T12:37:37.060351+0000","last_active":"2024-04-25T12:37:37.060297+0000","last_peered":"2024-04-25T12:37:37.060297+0000","last_clean":"2024-04-25T12:37:37.060297+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:37.060297+0000","last_undegraded":"2024-04-25T12:37:37.060297+0000","last_fullsized":"2024-04-25T12:37:37.060297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:37.060183+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:37.060183+0000","last_clean_scrub_stamp":"2024-04-25T12:37:37.060183+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:48.920888+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.007482+0000","last_change":"2024-04-25T12:37:50.007511+0000","last_active":"2024-04-25T12:37:50.007482+0000","last_peered":"2024-04-25T12:37:50.007482+0000","last_clean":"2024-04-25T12:37:50.007482+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:50.007482+0000","last_undegraded":"2024-04-25T12:37:50.007482+0000","last_fullsized":"2024-04-25T12:37:50.007482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:50.007382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:50.007382+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.007382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.553152+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.033281+0000","last_change":"2024-04-25T12:37:53.033308+0000","last_active":"2024-04-25T12:37:53.033281+0000","last_peered":"2024-04-25T12:37:53.033281+0000","last_clean":"2024-04-25T12:37:53.033281+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:53.033281+0000","last_undegraded":"2024-04-25T12:37:53.033281+0000","last_fullsized":"2024-04-25T12:37:53.033281+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:53.033179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:53.033179+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.033179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.772002+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"93'10","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.050930+0000","last_change":"2024-04-25T12:37:48.050958+0000","last_active":"2024-04-25T12:37:48.050930+0000","last_peered":"2024-04-25T12:37:48.050930+0000","last_clean":"2024-04-25T12:37:48.050930+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:48.050930+0000","last_undegraded":"2024-04-25T12:37:48.050930+0000","last_fullsized":"2024-04-25T12:37:48.050930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:48.050836+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:48.050836+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.050836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:41.921894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.279884+0000","last_change":"2024-04-25T12:37:52.279949+0000","last_active":"2024-04-25T12:37:52.279884+0000","last_peered":"2024-04-25T12:37:52.279884+0000","last_clean":"2024-04-25T12:37:52.279884+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:52.279884+0000","last_undegraded":"2024-04-25T12:37:52.279884+0000","last_fullsized":"2024-04-25T12:37:52.279884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-04-25T12:37:52.279678+0000","last_deep_scrub":"25'1","last_deep_scrub_stamp":"2024-04-25T12:37:52.279678+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.279678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:39.030279+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:21.424491+0000","last_change":"2024-04-25T12:36:00.647255+0000","last_active":"2024-04-25T12:37:21.424491+0000","last_peered":"2024-04-25T12:37:21.424491+0000","last_clean":"2024-04-25T12:37:21.424491+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:21.424491+0000","last_undegraded":"2024-04-25T12:37:21.424491+0000","last_fullsized":"2024-04-25T12:37:21.424491+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:16.951510+0000","last_clean_scrub_stamp":"2024-04-25T12:34:16.951510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.772271+0000","last_change":"2024-04-25T12:37:51.772300+0000","last_active":"2024-04-25T12:37:51.772271+0000","last_peered":"2024-04-25T12:37:51.772271+0000","last_clean":"2024-04-25T12:37:51.772271+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:51.772271+0000","last_undegraded":"2024-04-25T12:37:51.772271+0000","last_fullsized":"2024-04-25T12:37:51.772271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:51.772172+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:51.772172+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.772172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:12.920832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.782287+0000","last_change":"2024-04-25T12:37:44.782353+0000","last_active":"2024-04-25T12:37:44.782287+0000","last_peered":"2024-04-25T12:37:44.782287+0000","last_clean":"2024-04-25T12:37:44.782287+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:44.782287+0000","last_undegraded":"2024-04-25T12:37:44.782287+0000","last_fullsized":"2024-04-25T12:37:44.782287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:44.782175+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:44.782175+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.782175+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:28.854601+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'4","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.336067+0000","last_change":"2024-04-25T12:37:49.336200+0000","last_active":"2024-04-25T12:37:49.336067+0000","last_peered":"2024-04-25T12:37:49.336067+0000","last_clean":"2024-04-25T12:37:49.336067+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:49.336067+0000","last_undegraded":"2024-04-25T12:37:49.336067+0000","last_fullsized":"2024-04-25T12:37:49.336067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:49.335832+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:49.335832+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.335832+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:40.387712+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.004747+0000","last_change":"2024-04-25T12:37:49.004814+0000","last_active":"2024-04-25T12:37:49.004747+0000","last_peered":"2024-04-25T12:37:49.004747+0000","last_clean":"2024-04-25T12:37:49.004747+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:49.004747+0000","last_undegraded":"2024-04-25T12:37:49.004747+0000","last_fullsized":"2024-04-25T12:37:49.004747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:49.004615+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:49.004615+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.004615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:43.382083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":183,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.988039+0000","last_change":"2024-04-25T12:37:54.988087+0000","last_active":"2024-04-25T12:37:54.988039+0000","last_peered":"2024-04-25T12:37:54.988039+0000","last_clean":"2024-04-25T12:37:54.988039+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:54.988039+0000","last_undegraded":"2024-04-25T12:37:54.988039+0000","last_fullsized":"2024-04-25T12:37:54.988039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:54.987888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:54.987888+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.987888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:05.903127+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:56.787840+0000","last_change":"2024-04-25T12:37:56.787880+0000","last_active":"2024-04-25T12:37:56.787840+0000","last_peered":"2024-04-25T12:37:56.787840+0000","last_clean":"2024-04-25T12:37:56.787840+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:56.787840+0000","last_undegraded":"2024-04-25T12:37:56.787840+0000","last_fullsized":"2024-04-25T12:37:56.787840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:56.787737+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:56.787737+0000","last_clean_scrub_stamp":"2024-04-25T12:37:56.787737+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:28.564345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.066560+0000","last_change":"2024-04-25T12:37:43.066631+0000","last_active":"2024-04-25T12:37:43.066560+0000","last_peered":"2024-04-25T12:37:43.066560+0000","last_clean":"2024-04-25T12:37:43.066560+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:43.066560+0000","last_undegraded":"2024-04-25T12:37:43.066560+0000","last_fullsized":"2024-04-25T12:37:43.066560+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:43.066320+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:43.066320+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.066320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:47.150390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.030619+0000","last_change":"2024-04-25T12:37:49.030670+0000","last_active":"2024-04-25T12:37:49.030619+0000","last_peered":"2024-04-25T12:37:49.030619+0000","last_clean":"2024-04-25T12:37:49.030619+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:49.030619+0000","last_undegraded":"2024-04-25T12:37:49.030619+0000","last_fullsized":"2024-04-25T12:37:49.030619+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.030483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.030483+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.030483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:33.523601+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":181,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.586417+0000","last_change":"2024-04-25T12:37:48.586511+0000","last_active":"2024-04-25T12:37:48.586417+0000","last_peered":"2024-04-25T12:37:48.586417+0000","last_clean":"2024-04-25T12:37:48.586417+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:48.586417+0000","last_undegraded":"2024-04-25T12:37:48.586417+0000","last_fullsized":"2024-04-25T12:37:48.586417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:48.586208+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:48.586208+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.586208+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:39.131979+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.917504+0000","last_change":"2024-04-25T12:37:44.917539+0000","last_active":"2024-04-25T12:37:44.917504+0000","last_peered":"2024-04-25T12:37:44.917504+0000","last_clean":"2024-04-25T12:37:44.917504+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:44.917504+0000","last_undegraded":"2024-04-25T12:37:44.917504+0000","last_fullsized":"2024-04-25T12:37:44.917504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'12","last_scrub_stamp":"2024-04-25T12:37:44.917411+0000","last_deep_scrub":"93'12","last_deep_scrub_stamp":"2024-04-25T12:37:44.917411+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.917411+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:32.483433+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:31.083871+0000","last_change":"2024-04-25T12:36:00.640991+0000","last_active":"2024-04-25T12:37:31.083871+0000","last_peered":"2024-04-25T12:37:31.083871+0000","last_clean":"2024-04-25T12:37:31.083871+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:31.083871+0000","last_undegraded":"2024-04-25T12:37:31.083871+0000","last_fullsized":"2024-04-25T12:37:31.083871+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:34:12.877100+0000","last_clean_scrub_stamp":"2024-04-25T12:34:12.877100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.886746+0000","last_change":"2024-04-25T12:37:46.886776+0000","last_active":"2024-04-25T12:37:46.886746+0000","last_peered":"2024-04-25T12:37:46.886746+0000","last_clean":"2024-04-25T12:37:46.886746+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:46.886746+0000","last_undegraded":"2024-04-25T12:37:46.886746+0000","last_fullsized":"2024-04-25T12:37:46.886746+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:46.886648+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:46.886648+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.886648+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:35.666915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":54,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.687025+0000","last_change":"2024-04-25T12:37:50.687058+0000","last_active":"2024-04-25T12:37:50.687025+0000","last_peered":"2024-04-25T12:37:50.687025+0000","last_clean":"2024-04-25T12:37:50.687025+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:50.687025+0000","last_undegraded":"2024-04-25T12:37:50.687025+0000","last_fullsized":"2024-04-25T12:37:50.687025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:50.686934+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:50.686934+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.686934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:31.966695+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.832855+0000","last_change":"2024-04-25T12:37:41.832903+0000","last_active":"2024-04-25T12:37:41.832855+0000","last_peered":"2024-04-25T12:37:41.832855+0000","last_clean":"2024-04-25T12:37:41.832855+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:41.832855+0000","last_undegraded":"2024-04-25T12:37:41.832855+0000","last_fullsized":"2024-04-25T12:37:41.832855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:41.832740+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:41.832740+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.832740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:34.123041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.542783+0000","last_change":"2024-04-25T12:37:51.542917+0000","last_active":"2024-04-25T12:37:51.542783+0000","last_peered":"2024-04-25T12:37:51.542783+0000","last_clean":"2024-04-25T12:37:51.542783+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:51.542783+0000","last_undegraded":"2024-04-25T12:37:51.542783+0000","last_fullsized":"2024-04-25T12:37:51.542783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'15","last_scrub_stamp":"2024-04-25T12:37:51.542496+0000","last_deep_scrub":"93'15","last_deep_scrub_stamp":"2024-04-25T12:37:51.542496+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.542496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:49.281319+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":119,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.848516+0000","last_change":"2024-04-25T12:37:42.848546+0000","last_active":"2024-04-25T12:37:42.848516+0000","last_peered":"2024-04-25T12:37:42.848516+0000","last_clean":"2024-04-25T12:37:42.848516+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:42.848516+0000","last_undegraded":"2024-04-25T12:37:42.848516+0000","last_fullsized":"2024-04-25T12:37:42.848516+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:42.848421+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:42.848421+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.848421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.888338+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:56.296195+0000","last_change":"2024-04-25T12:37:56.296263+0000","last_active":"2024-04-25T12:37:56.296195+0000","last_peered":"2024-04-25T12:37:56.296195+0000","last_clean":"2024-04-25T12:37:56.296195+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:56.296195+0000","last_undegraded":"2024-04-25T12:37:56.296195+0000","last_fullsized":"2024-04-25T12:37:56.296195+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:56.295994+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:56.295994+0000","last_clean_scrub_stamp":"2024-04-25T12:37:56.295994+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:01.362500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.888842+0000","last_change":"2024-04-25T12:37:43.888875+0000","last_active":"2024-04-25T12:37:43.888842+0000","last_peered":"2024-04-25T12:37:43.888842+0000","last_clean":"2024-04-25T12:37:43.888842+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:43.888842+0000","last_undegraded":"2024-04-25T12:37:43.888842+0000","last_fullsized":"2024-04-25T12:37:43.888842+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:43.888723+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:43.888723+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.888723+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.051839+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.130564+0000","last_change":"2024-04-25T12:37:39.130692+0000","last_active":"2024-04-25T12:37:39.130564+0000","last_peered":"2024-04-25T12:37:39.130564+0000","last_clean":"2024-04-25T12:37:39.130564+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:39.130564+0000","last_undegraded":"2024-04-25T12:37:39.130564+0000","last_fullsized":"2024-04-25T12:37:39.130564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:39.130333+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:39.130333+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.130333+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:05.653918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":175,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.009725+0000","last_change":"2024-04-25T12:37:47.009756+0000","last_active":"2024-04-25T12:37:47.009725+0000","last_peered":"2024-04-25T12:37:47.009725+0000","last_clean":"2024-04-25T12:37:47.009725+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:47.009725+0000","last_undegraded":"2024-04-25T12:37:47.009725+0000","last_fullsized":"2024-04-25T12:37:47.009725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'11","last_scrub_stamp":"2024-04-25T12:37:47.009632+0000","last_deep_scrub":"93'11","last_deep_scrub_stamp":"2024-04-25T12:37:47.009632+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.009632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:31.389346+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.077661+0000","last_change":"2024-04-25T12:37:43.077728+0000","last_active":"2024-04-25T12:37:43.077661+0000","last_peered":"2024-04-25T12:37:43.077661+0000","last_clean":"2024-04-25T12:37:43.077661+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:43.077661+0000","last_undegraded":"2024-04-25T12:37:43.077661+0000","last_fullsized":"2024-04-25T12:37:43.077661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:43.077443+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:43.077443+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.077443+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:23.639432+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.311840+0000","last_change":"2024-04-25T12:37:44.311906+0000","last_active":"2024-04-25T12:37:44.311840+0000","last_peered":"2024-04-25T12:37:44.311840+0000","last_clean":"2024-04-25T12:37:44.311840+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:44.311840+0000","last_undegraded":"2024-04-25T12:37:44.311840+0000","last_fullsized":"2024-04-25T12:37:44.311840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:44.311650+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:44.311650+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.311650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:04.441351+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.869180+0000","last_change":"2024-04-25T12:37:43.869224+0000","last_active":"2024-04-25T12:37:43.869180+0000","last_peered":"2024-04-25T12:37:43.869180+0000","last_clean":"2024-04-25T12:37:43.869180+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:43.869180+0000","last_undegraded":"2024-04-25T12:37:43.869180+0000","last_fullsized":"2024-04-25T12:37:43.869180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:43.869038+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:43.869038+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.869038+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:14.272772+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.300879+0000","last_change":"2024-04-25T12:37:41.300948+0000","last_active":"2024-04-25T12:37:41.300879+0000","last_peered":"2024-04-25T12:37:41.300879+0000","last_clean":"2024-04-25T12:37:41.300879+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:41.300879+0000","last_undegraded":"2024-04-25T12:37:41.300879+0000","last_fullsized":"2024-04-25T12:37:41.300879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:41.300665+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:41.300665+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.300665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:13.476225+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.258248+0000","last_change":"2024-04-25T12:37:54.258320+0000","last_active":"2024-04-25T12:37:54.258248+0000","last_peered":"2024-04-25T12:37:54.258248+0000","last_clean":"2024-04-25T12:37:54.258248+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:54.258248+0000","last_undegraded":"2024-04-25T12:37:54.258248+0000","last_fullsized":"2024-04-25T12:37:54.258248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:54.258048+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:54.258048+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.258048+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:30.534013+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.847728+0000","last_change":"2024-04-25T12:37:47.847759+0000","last_active":"2024-04-25T12:37:47.847728+0000","last_peered":"2024-04-25T12:37:47.847728+0000","last_clean":"2024-04-25T12:37:47.847728+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:47.847728+0000","last_undegraded":"2024-04-25T12:37:47.847728+0000","last_fullsized":"2024-04-25T12:37:47.847728+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:47.847623+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:47.847623+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.847623+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:00.750853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"93'8","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.836184+0000","last_change":"2024-04-25T12:37:42.836230+0000","last_active":"2024-04-25T12:37:42.836184+0000","last_peered":"2024-04-25T12:37:42.836184+0000","last_clean":"2024-04-25T12:37:42.836184+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:42.836184+0000","last_undegraded":"2024-04-25T12:37:42.836184+0000","last_fullsized":"2024-04-25T12:37:42.836184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:42.836028+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:42.836028+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.836028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:39.958343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.796893+0000","last_change":"2024-04-25T12:37:43.796925+0000","last_active":"2024-04-25T12:37:43.796893+0000","last_peered":"2024-04-25T12:37:43.796893+0000","last_clean":"2024-04-25T12:37:43.796893+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:43.796893+0000","last_undegraded":"2024-04-25T12:37:43.796893+0000","last_fullsized":"2024-04-25T12:37:43.796893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:43.796780+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:43.796780+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.796780+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:22.175947+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.094566+0000","last_change":"2024-04-25T12:37:44.094632+0000","last_active":"2024-04-25T12:37:44.094566+0000","last_peered":"2024-04-25T12:37:44.094566+0000","last_clean":"2024-04-25T12:37:44.094566+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:44.094566+0000","last_undegraded":"2024-04-25T12:37:44.094566+0000","last_fullsized":"2024-04-25T12:37:44.094566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:44.094339+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:44.094339+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.094339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:06.409183+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.138466+0000","last_change":"2024-04-25T12:37:45.138545+0000","last_active":"2024-04-25T12:37:45.138466+0000","last_peered":"2024-04-25T12:37:45.138466+0000","last_clean":"2024-04-25T12:37:45.138466+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:45.138466+0000","last_undegraded":"2024-04-25T12:37:45.138466+0000","last_fullsized":"2024-04-25T12:37:45.138466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:45.138176+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:45.138176+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.138176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:48.969915+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":178,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.081626+0000","last_change":"2024-04-25T12:37:42.081690+0000","last_active":"2024-04-25T12:37:42.081626+0000","last_peered":"2024-04-25T12:37:42.081626+0000","last_clean":"2024-04-25T12:37:42.081626+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:42.081626+0000","last_undegraded":"2024-04-25T12:37:42.081626+0000","last_fullsized":"2024-04-25T12:37:42.081626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'16","last_scrub_stamp":"2024-04-25T12:37:42.081396+0000","last_deep_scrub":"93'16","last_deep_scrub_stamp":"2024-04-25T12:37:42.081396+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.081396+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:04.044060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.093321+0000","last_change":"2024-04-25T12:37:38.093385+0000","last_active":"2024-04-25T12:37:38.093321+0000","last_peered":"2024-04-25T12:37:38.093321+0000","last_clean":"2024-04-25T12:37:38.093321+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:38.093321+0000","last_undegraded":"2024-04-25T12:37:38.093321+0000","last_fullsized":"2024-04-25T12:37:38.093321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'11","last_scrub_stamp":"2024-04-25T12:37:38.093157+0000","last_deep_scrub":"93'11","last_deep_scrub_stamp":"2024-04-25T12:37:38.093157+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.093157+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.126985+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.807314+0000","last_change":"2024-04-25T12:37:48.807350+0000","last_active":"2024-04-25T12:37:48.807314+0000","last_peered":"2024-04-25T12:37:48.807314+0000","last_clean":"2024-04-25T12:37:48.807314+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:48.807314+0000","last_undegraded":"2024-04-25T12:37:48.807314+0000","last_fullsized":"2024-04-25T12:37:48.807314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:48.807218+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:48.807218+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.807218+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:24.754964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.117939+0000","last_change":"2024-04-25T12:37:40.117972+0000","last_active":"2024-04-25T12:37:40.117939+0000","last_peered":"2024-04-25T12:37:40.117939+0000","last_clean":"2024-04-25T12:37:40.117939+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:40.117939+0000","last_undegraded":"2024-04-25T12:37:40.117939+0000","last_fullsized":"2024-04-25T12:37:40.117939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:40.117836+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:40.117836+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.117836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:39.198481+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.524361+0000","last_change":"2024-04-25T12:37:39.524387+0000","last_active":"2024-04-25T12:37:39.524361+0000","last_peered":"2024-04-25T12:37:39.524361+0000","last_clean":"2024-04-25T12:37:39.524361+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:39.524361+0000","last_undegraded":"2024-04-25T12:37:39.524361+0000","last_fullsized":"2024-04-25T12:37:39.524361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:39.524277+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:39.524277+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.524277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:47.690832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.194536+0000","last_change":"2024-04-25T12:37:48.194606+0000","last_active":"2024-04-25T12:37:48.194536+0000","last_peered":"2024-04-25T12:37:48.194536+0000","last_clean":"2024-04-25T12:37:48.194536+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:48.194536+0000","last_undegraded":"2024-04-25T12:37:48.194536+0000","last_fullsized":"2024-04-25T12:37:48.194536+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:48.194276+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:48.194276+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.194276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:20.369836+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.506748+0000","last_change":"2024-04-25T12:37:38.506814+0000","last_active":"2024-04-25T12:37:38.506748+0000","last_peered":"2024-04-25T12:37:38.506748+0000","last_clean":"2024-04-25T12:37:38.506748+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:38.506748+0000","last_undegraded":"2024-04-25T12:37:38.506748+0000","last_fullsized":"2024-04-25T12:37:38.506748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:38.506564+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:38.506564+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.506564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.015531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.925077+0000","last_change":"2024-04-25T12:37:45.925127+0000","last_active":"2024-04-25T12:37:45.925077+0000","last_peered":"2024-04-25T12:37:45.925077+0000","last_clean":"2024-04-25T12:37:45.925077+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:45.925077+0000","last_undegraded":"2024-04-25T12:37:45.925077+0000","last_fullsized":"2024-04-25T12:37:45.925077+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:45.924964+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:45.924964+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.924964+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:56.579137+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.768784+0000","last_change":"2024-04-25T12:37:54.768830+0000","last_active":"2024-04-25T12:37:54.768784+0000","last_peered":"2024-04-25T12:37:54.768784+0000","last_clean":"2024-04-25T12:37:54.768784+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:54.768784+0000","last_undegraded":"2024-04-25T12:37:54.768784+0000","last_fullsized":"2024-04-25T12:37:54.768784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:54.768695+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:54.768695+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.768695+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:09.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.160533+0000","last_change":"2024-04-25T12:37:46.160605+0000","last_active":"2024-04-25T12:37:46.160533+0000","last_peered":"2024-04-25T12:37:46.160533+0000","last_clean":"2024-04-25T12:37:46.160533+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:46.160533+0000","last_undegraded":"2024-04-25T12:37:46.160533+0000","last_fullsized":"2024-04-25T12:37:46.160533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:46.160303+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:46.160303+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.160303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:21.482991+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":501,"ondisk_log_size":501,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41301,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":171,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82602,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542194,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462938,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29324,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342516,"statfs":{"total":96636764160,"available":96606736384,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1216512,"data_stored":1207576,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1142784,"data_stored":616256,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37895,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38847,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:38:02.618 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:38:02.619 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T12:34:11.863765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:38:02.619 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:38:02.619 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:38:02.620 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-25T12:34:16.951510+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:38:02.620 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T12:34:12.877100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=34, tm_sec=12, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=12, tm_min=37, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T12:38:02.620 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T12:38:08.208 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:08.216 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:22.621 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:38:22.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.907+0000 7f031a800640 1 Processor -- start 2024-04-25T12:38:22.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.908+0000 7f031a800640 1 -- start start 2024-04-25T12:38:22.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f031a800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0314074870 0x7f0314154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:22.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f031a800640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f031415afa0 0x7f03140cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:22.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f031a800640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f031415b3b0 con 0x7f03140c4d50 2024-04-25T12:38:22.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f031a800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f03140c61f0 con 0x7f0314074870 2024-04-25T12:38:22.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f031a800640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f03140c87d0 con 0x7f031415afa0 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f030aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0314074870 0x7f0314154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f030aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0314074870 0x7f0314154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43530/0 (socket says 172.21.15.123:43530) 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f030aa00640 1 -- 172.21.15.123:0/3078328298 learned_addr learned my addr 172.21.15.123:0/3078328298 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.910+0000 7f0313e00640 1 --2- 172.21.15.123:0/3078328298 >> v2:172.21.15.153:3300/0 conn(0x7f031415afa0 0x7f03140cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.911+0000 7f030aa00640 1 -- 172.21.15.123:0/3078328298 >> v2:172.21.15.153:3300/0 conn(0x7f031415afa0 msgr2=0x7f03140cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.911+0000 7f030aa00640 1 --2- 172.21.15.123:0/3078328298 >> v2:172.21.15.153:3300/0 conn(0x7f031415afa0 0x7f03140cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.911+0000 7f030aa00640 1 -- 172.21.15.123:0/3078328298 >> v1:172.21.15.153:6789/0 conn(0x7f03140c4d50 legacy=0x7f03140c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:22.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.911+0000 7f030aa00640 1 -- 172.21.15.123:0/3078328298 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0314073ce0 con 0x7f0314074870 2024-04-25T12:38:22.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.912+0000 7f030aa00640 1 --2- 172.21.15.123:0/3078328298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0314074870 0x7f0314154890 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f03140ce8b0 tx=0x7f02fc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6367f62a6f065459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:22.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.912+0000 7f0312a00640 1 -- 172.21.15.123:0/3078328298 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc037070 con 0x7f0314074870 2024-04-25T12:38:22.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.912+0000 7f0312a00640 1 -- 172.21.15.123:0/3078328298 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02fc02ae50 con 0x7f0314074870 2024-04-25T12:38:22.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.912+0000 7f0312a00640 1 -- 172.21.15.123:0/3078328298 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc032860 con 0x7f0314074870 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 -- 172.21.15.123:0/3078328298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0314074870 msgr2=0x7f0314154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 --2- 172.21.15.123:0/3078328298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0314074870 0x7f0314154890 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f03140ce8b0 tx=0x7f02fc02a300 comp rx=0 tx=0).stop 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 -- 172.21.15.123:0/3078328298 shutdown_connections 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 --2- 172.21.15.123:0/3078328298 >> v2:172.21.15.153:3300/0 conn(0x7f031415afa0 0x7f03140cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 --2- 172.21.15.123:0/3078328298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0314074870 0x7f0314154890 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 -- 172.21.15.123:0/3078328298 >> 172.21.15.123:0/3078328298 conn(0x7f031406c750 msgr2=0x7f031406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 -- 172.21.15.123:0/3078328298 shutdown_connections 2024-04-25T12:38:22.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.913+0000 7f031a800640 1 -- 172.21.15.123:0/3078328298 wait complete. 2024-04-25T12:38:22.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.916+0000 7f031a800640 1 Processor -- start 2024-04-25T12:38:22.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.916+0000 7f031a800640 1 -- start start 2024-04-25T12:38:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f031a800640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0314074870 0x7f03141f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f031a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03140c4d50 0x7f0314159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f031a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f031415afa0 0x7f03140cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f031a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03140c8460 con 0x7f03140c4d50 2024-04-25T12:38:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f031a800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f03140742a0 con 0x7f031415afa0 2024-04-25T12:38:22.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f031a800640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f03140c9370 con 0x7f0314074870 2024-04-25T12:38:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f030aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03140c4d50 0x7f0314159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f0313400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0314074870 0x7f03141f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f030aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03140c4d50 0x7f0314159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43544/0 (socket says 172.21.15.123:43544) 2024-04-25T12:38:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f030aa00640 1 -- 172.21.15.123:0/1674946963 learned_addr learned my addr 172.21.15.123:0/1674946963 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.918+0000 7f0313e00640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f031415afa0 0x7f03140cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f0313400640 1 -- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f031415afa0 msgr2=0x7f03140cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f0313400640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f031415afa0 0x7f03140cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f0313400640 1 -- 172.21.15.123:0/1674946963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03140c4d50 msgr2=0x7f0314159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f0313400640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03140c4d50 0x7f0314159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f0313400640 1 -- 172.21.15.123:0/1674946963 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03141f4540 con 0x7f0314074870 2024-04-25T12:38:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f030aa00640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03140c4d50 0x7f0314159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:38:22.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f0313400640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0314074870 0x7f03141f4000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f030400c3c0 tx=0x7f030400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f030be00640 1 -- 172.21.15.123:0/1674946963 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304017070 con 0x7f0314074870 2024-04-25T12:38:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f030be00640 1 -- 172.21.15.123:0/1674946963 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03040052d0 con 0x7f0314074870 2024-04-25T12:38:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.919+0000 7f030be00640 1 -- 172.21.15.123:0/1674946963 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304003830 con 0x7f0314074870 2024-04-25T12:38:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.920+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031415d050 con 0x7f0314074870 2024-04-25T12:38:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.920+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0314153960 con 0x7f0314074870 2024-04-25T12:38:22.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.921+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0314076700 con 0x7f0314074870 2024-04-25T12:38:22.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.922+0000 7f030be00640 1 -- 172.21.15.123:0/1674946963 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f030401a070 con 0x7f0314074870 2024-04-25T12:38:22.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.923+0000 7f030be00640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f02f00744e0 0x7f02f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:22.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.924+0000 7f030be00640 1 -- 172.21.15.123:0/1674946963 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f0304094880 con 0x7f0314074870 2024-04-25T12:38:22.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.924+0000 7f030aa00640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f02f00744e0 0x7f02f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:22.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.924+0000 7f030aa00640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f02f00744e0 0x7f02f00769f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02fc035040 tx=0x7f02fc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:22.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:22.927+0000 7f030be00640 1 -- 172.21.15.123:0/1674946963 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f030405e590 con 0x7f0314074870 2024-04-25T12:38:23.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.126+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 --> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f031415be60 con 0x7f02f00744e0 2024-04-25T12:38:23.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.135+0000 7f030be00640 1 -- 172.21.15.123:0/1674946963 <== mgr.4102 v2:172.21.15.153:6800/2246116237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333486 (secure 0 0 0) 0x7f031415be60 con 0x7f02f00744e0 2024-04-25T12:38:23.139 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:38:23.141 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f02f00744e0 msgr2=0x7f02f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f02f00744e0 0x7f02f00769f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02fc035040 tx=0x7f02fc02e000 comp rx=0 tx=0).stop 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0314074870 msgr2=0x7f03141f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0314074870 0x7f03141f4000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f030400c3c0 tx=0x7f030400c880 comp rx=0 tx=0).stop 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 shutdown_connections 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f02f00744e0 0x7f02f00769f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f031415afa0 0x7f03140cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03140c4d50 0x7f0314159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 --2- 172.21.15.123:0/1674946963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0314074870 0x7f03141f4000 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 >> 172.21.15.123:0/1674946963 conn(0x7f031406c750 msgr2=0x7f03140c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 shutdown_connections 2024-04-25T12:38:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.141+0000 7f031a800640 1 -- 172.21.15.123:0/1674946963 wait complete. 2024-04-25T12:38:23.170 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-04-25T12:38:22.858447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631669,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":274,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":592776,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247024,"kb_used_data":8880,"kb_used_omap":20,"kb_used_meta":238059,"kb_avail":754727696,"statfs":{"total":773094113280,"available":772841160704,"internally_reserved":0,"allocated":9093120,"data_stored":5890226,"data_compressed":70132,"data_compressed_allocated":98304,"data_compressed_original":1179648,"omap_allocated":20525,"internal_metadata":243773395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004019"},"pg_stats":[{"pgid":"2.1f","version":"93'13","reported_seq":179,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.945303+0000","last_change":"2024-04-25T12:37:41.945368+0000","last_active":"2024-04-25T12:37:41.945303+0000","last_peered":"2024-04-25T12:37:41.945303+0000","last_clean":"2024-04-25T12:37:41.945303+0000","last_became_active":"2024-04-25T12:34:12.911773+0000","last_became_peered":"2024-04-25T12:34:12.911773+0000","last_unstale":"2024-04-25T12:37:41.945303+0000","last_undegraded":"2024-04-25T12:37:41.945303+0000","last_fullsized":"2024-04-25T12:37:41.945303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'13","last_scrub_stamp":"2024-04-25T12:37:41.945200+0000","last_deep_scrub":"93'13","last_deep_scrub_stamp":"2024-04-25T12:37:41.945200+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.945200+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:39.067515+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":123,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.780946+0000","last_change":"2024-04-25T12:37:40.780946+0000","last_active":"2024-04-25T12:37:40.780946+0000","last_peered":"2024-04-25T12:37:40.780946+0000","last_clean":"2024-04-25T12:37:40.780946+0000","last_became_active":"2024-04-25T12:36:00.629944+0000","last_became_peered":"2024-04-25T12:36:00.629944+0000","last_unstale":"2024-04-25T12:37:40.780946+0000","last_undegraded":"2024-04-25T12:37:40.780946+0000","last_fullsized":"2024-04-25T12:37:40.780946+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:40.780817+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:40.780817+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.780817+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:41.153300+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.020448+0000","last_change":"2024-04-25T12:37:52.020503+0000","last_active":"2024-04-25T12:37:52.020448+0000","last_peered":"2024-04-25T12:37:52.020448+0000","last_clean":"2024-04-25T12:37:52.020448+0000","last_became_active":"2024-04-25T12:37:01.221869+0000","last_became_peered":"2024-04-25T12:37:01.221869+0000","last_unstale":"2024-04-25T12:37:52.020448+0000","last_undegraded":"2024-04-25T12:37:52.020448+0000","last_fullsized":"2024-04-25T12:37:52.020448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:52.020327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:52.020327+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.020327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:42.313726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":134,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.776120+0000","last_change":"2024-04-25T12:37:49.776120+0000","last_active":"2024-04-25T12:37:49.776120+0000","last_peered":"2024-04-25T12:37:49.776120+0000","last_clean":"2024-04-25T12:37:49.776120+0000","last_became_active":"2024-04-25T12:36:00.627913+0000","last_became_peered":"2024-04-25T12:36:00.627913+0000","last_unstale":"2024-04-25T12:37:49.776120+0000","last_undegraded":"2024-04-25T12:37:49.776120+0000","last_fullsized":"2024-04-25T12:37:49.776120+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-25T12:37:49.775941+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-25T12:37:49.775941+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.775941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:07.128554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.758680+0000","last_change":"2024-04-25T12:37:50.758680+0000","last_active":"2024-04-25T12:37:50.758680+0000","last_peered":"2024-04-25T12:37:50.758680+0000","last_clean":"2024-04-25T12:37:50.758680+0000","last_became_active":"2024-04-25T12:36:00.628975+0000","last_became_peered":"2024-04-25T12:36:00.628975+0000","last_unstale":"2024-04-25T12:37:50.758680+0000","last_undegraded":"2024-04-25T12:37:50.758680+0000","last_fullsized":"2024-04-25T12:37:50.758680+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:50.758503+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:50.758503+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.758503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:27.428308+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.596749+0000","last_change":"2024-04-25T12:37:45.596820+0000","last_active":"2024-04-25T12:37:45.596749+0000","last_peered":"2024-04-25T12:37:45.596749+0000","last_clean":"2024-04-25T12:37:45.596749+0000","last_became_active":"2024-04-25T12:37:01.215733+0000","last_became_peered":"2024-04-25T12:37:01.215733+0000","last_unstale":"2024-04-25T12:37:45.596749+0000","last_undegraded":"2024-04-25T12:37:45.596749+0000","last_fullsized":"2024-04-25T12:37:45.596749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.596535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.596535+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.596535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:29.976249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"93'7","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.125881+0000","last_change":"2024-04-25T12:37:39.125955+0000","last_active":"2024-04-25T12:37:39.125881+0000","last_peered":"2024-04-25T12:37:39.125881+0000","last_clean":"2024-04-25T12:37:39.125881+0000","last_became_active":"2024-04-25T12:34:12.928637+0000","last_became_peered":"2024-04-25T12:34:12.928637+0000","last_unstale":"2024-04-25T12:37:39.125881+0000","last_undegraded":"2024-04-25T12:37:39.125881+0000","last_fullsized":"2024-04-25T12:37:39.125881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:39.125696+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:39.125696+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.125696+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:45.706768+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.141232+0000","last_change":"2024-04-25T12:37:40.141286+0000","last_active":"2024-04-25T12:37:40.141232+0000","last_peered":"2024-04-25T12:37:40.141232+0000","last_clean":"2024-04-25T12:37:40.141232+0000","last_became_active":"2024-04-25T12:34:13.976730+0000","last_became_peered":"2024-04-25T12:34:13.976730+0000","last_unstale":"2024-04-25T12:37:40.141232+0000","last_undegraded":"2024-04-25T12:37:40.141232+0000","last_fullsized":"2024-04-25T12:37:40.141232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:40.141071+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:40.141071+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.141071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:18.869979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.591648+0000","last_change":"2024-04-25T12:37:50.591768+0000","last_active":"2024-04-25T12:37:50.591648+0000","last_peered":"2024-04-25T12:37:50.591648+0000","last_clean":"2024-04-25T12:37:50.591648+0000","last_became_active":"2024-04-25T12:37:01.226105+0000","last_became_peered":"2024-04-25T12:37:01.226105+0000","last_unstale":"2024-04-25T12:37:50.591648+0000","last_undegraded":"2024-04-25T12:37:50.591648+0000","last_fullsized":"2024-04-25T12:37:50.591648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:50.591325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:50.591325+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.591325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:51.707296+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"93'3","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:37.196340+0000","last_change":"2024-04-25T12:37:37.196393+0000","last_active":"2024-04-25T12:37:37.196340+0000","last_peered":"2024-04-25T12:37:37.196340+0000","last_clean":"2024-04-25T12:37:37.196340+0000","last_became_active":"2024-04-25T12:34:12.926999+0000","last_became_peered":"2024-04-25T12:34:12.926999+0000","last_unstale":"2024-04-25T12:37:37.196340+0000","last_undegraded":"2024-04-25T12:37:37.196340+0000","last_fullsized":"2024-04-25T12:37:37.196340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:37.196244+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:37.196244+0000","last_clean_scrub_stamp":"2024-04-25T12:37:37.196244+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:49.970014+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.341184+0000","last_change":"2024-04-25T12:37:43.341250+0000","last_active":"2024-04-25T12:37:43.341184+0000","last_peered":"2024-04-25T12:37:43.341184+0000","last_clean":"2024-04-25T12:37:43.341184+0000","last_became_active":"2024-04-25T12:34:13.977435+0000","last_became_peered":"2024-04-25T12:34:13.977435+0000","last_unstale":"2024-04-25T12:37:43.341184+0000","last_undegraded":"2024-04-25T12:37:43.341184+0000","last_fullsized":"2024-04-25T12:37:43.341184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'15","last_scrub_stamp":"2024-04-25T12:37:43.340987+0000","last_deep_scrub":"93'15","last_deep_scrub_stamp":"2024-04-25T12:37:43.340987+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.340987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.966156+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.636763+0000","last_change":"2024-04-25T12:37:46.636834+0000","last_active":"2024-04-25T12:37:46.636763+0000","last_peered":"2024-04-25T12:37:46.636763+0000","last_clean":"2024-04-25T12:37:46.636763+0000","last_became_active":"2024-04-25T12:37:01.220326+0000","last_became_peered":"2024-04-25T12:37:01.220326+0000","last_unstale":"2024-04-25T12:37:46.636763+0000","last_undegraded":"2024-04-25T12:37:46.636763+0000","last_fullsized":"2024-04-25T12:37:46.636763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:46.636538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:46.636538+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.636538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:40.507596+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":166,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.301675+0000","last_change":"2024-04-25T12:37:39.301744+0000","last_active":"2024-04-25T12:37:39.301675+0000","last_peered":"2024-04-25T12:37:39.301675+0000","last_clean":"2024-04-25T12:37:39.301675+0000","last_became_active":"2024-04-25T12:34:13.971210+0000","last_became_peered":"2024-04-25T12:34:13.971210+0000","last_unstale":"2024-04-25T12:37:39.301675+0000","last_undegraded":"2024-04-25T12:37:39.301675+0000","last_fullsized":"2024-04-25T12:37:39.301675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:39.301505+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:39.301505+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.301505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:59.601828+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"93'7","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.308766+0000","last_change":"2024-04-25T12:37:53.308898+0000","last_active":"2024-04-25T12:37:53.308766+0000","last_peered":"2024-04-25T12:37:53.308766+0000","last_clean":"2024-04-25T12:37:53.308766+0000","last_became_active":"2024-04-25T12:34:12.923395+0000","last_became_peered":"2024-04-25T12:34:12.923395+0000","last_unstale":"2024-04-25T12:37:53.308766+0000","last_undegraded":"2024-04-25T12:37:53.308766+0000","last_fullsized":"2024-04-25T12:37:53.308766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:53.308530+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:53.308530+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.308530+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:06.361349+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.157193+0000","last_change":"2024-04-25T12:37:49.157260+0000","last_active":"2024-04-25T12:37:49.157193+0000","last_peered":"2024-04-25T12:37:49.157193+0000","last_clean":"2024-04-25T12:37:49.157193+0000","last_became_active":"2024-04-25T12:36:00.639530+0000","last_became_peered":"2024-04-25T12:36:00.639530+0000","last_unstale":"2024-04-25T12:37:49.157193+0000","last_undegraded":"2024-04-25T12:37:49.157193+0000","last_fullsized":"2024-04-25T12:37:49.157193+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.156932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.156932+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.156932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:34.008352+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7,5],"acting":[0,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.034372+0000","last_change":"2024-04-25T12:37:51.034402+0000","last_active":"2024-04-25T12:37:51.034372+0000","last_peered":"2024-04-25T12:37:51.034372+0000","last_clean":"2024-04-25T12:37:51.034372+0000","last_became_active":"2024-04-25T12:37:01.209152+0000","last_became_peered":"2024-04-25T12:37:01.209152+0000","last_unstale":"2024-04-25T12:37:51.034372+0000","last_undegraded":"2024-04-25T12:37:51.034372+0000","last_fullsized":"2024-04-25T12:37:51.034372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:51.034270+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:51.034270+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.034270+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:36.004724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.670764+0000","last_change":"2024-04-25T12:37:53.670806+0000","last_active":"2024-04-25T12:37:53.670764+0000","last_peered":"2024-04-25T12:37:53.670764+0000","last_clean":"2024-04-25T12:37:53.670764+0000","last_became_active":"2024-04-25T12:37:01.209022+0000","last_became_peered":"2024-04-25T12:37:01.209022+0000","last_unstale":"2024-04-25T12:37:53.670764+0000","last_undegraded":"2024-04-25T12:37:53.670764+0000","last_fullsized":"2024-04-25T12:37:53.670764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:53.670672+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:53.670672+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.670672+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:38.153329+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.177897+0000","last_change":"2024-04-25T12:37:50.178033+0000","last_active":"2024-04-25T12:37:50.177897+0000","last_peered":"2024-04-25T12:37:50.177897+0000","last_clean":"2024-04-25T12:37:50.177897+0000","last_became_active":"2024-04-25T12:37:01.221657+0000","last_became_peered":"2024-04-25T12:37:01.221657+0000","last_unstale":"2024-04-25T12:37:50.177897+0000","last_undegraded":"2024-04-25T12:37:50.177897+0000","last_fullsized":"2024-04-25T12:37:50.177897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:50.177608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:50.177608+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.177608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:08.619291+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.973607+0000","last_change":"2024-04-25T12:37:44.973638+0000","last_active":"2024-04-25T12:37:44.973607+0000","last_peered":"2024-04-25T12:37:44.973607+0000","last_clean":"2024-04-25T12:37:44.973607+0000","last_became_active":"2024-04-25T12:34:13.935012+0000","last_became_peered":"2024-04-25T12:34:13.935012+0000","last_unstale":"2024-04-25T12:37:44.973607+0000","last_undegraded":"2024-04-25T12:37:44.973607+0000","last_fullsized":"2024-04-25T12:37:44.973607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:44.973505+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:44.973505+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.973505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:41.866275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"93'5","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.935860+0000","last_change":"2024-04-25T12:37:43.935898+0000","last_active":"2024-04-25T12:37:43.935860+0000","last_peered":"2024-04-25T12:37:43.935860+0000","last_clean":"2024-04-25T12:37:43.935860+0000","last_became_active":"2024-04-25T12:34:12.918511+0000","last_became_peered":"2024-04-25T12:34:12.918511+0000","last_unstale":"2024-04-25T12:37:43.935860+0000","last_undegraded":"2024-04-25T12:37:43.935860+0000","last_fullsized":"2024-04-25T12:37:43.935860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:43.935726+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:43.935726+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.935726+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:28.428844+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.977659+0000","last_change":"2024-04-25T12:37:46.977689+0000","last_active":"2024-04-25T12:37:46.977659+0000","last_peered":"2024-04-25T12:37:46.977659+0000","last_clean":"2024-04-25T12:37:46.977659+0000","last_became_active":"2024-04-25T12:34:18.015816+0000","last_became_peered":"2024-04-25T12:34:18.015816+0000","last_unstale":"2024-04-25T12:37:46.977659+0000","last_undegraded":"2024-04-25T12:37:46.977659+0000","last_fullsized":"2024-04-25T12:37:46.977659+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:46.977559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:46.977559+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.977559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.016158+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.318964+0000","last_change":"2024-04-25T12:37:40.319013+0000","last_active":"2024-04-25T12:37:40.318964+0000","last_peered":"2024-04-25T12:37:40.318964+0000","last_clean":"2024-04-25T12:37:40.318964+0000","last_became_active":"2024-04-25T12:34:13.963412+0000","last_became_peered":"2024-04-25T12:34:13.963412+0000","last_unstale":"2024-04-25T12:37:40.318964+0000","last_undegraded":"2024-04-25T12:37:40.318964+0000","last_fullsized":"2024-04-25T12:37:40.318964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:40.318837+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:40.318837+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.318837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:30.379834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"93'3","reported_seq":102,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:38:00.299479+0000","last_change":"2024-04-25T12:38:00.299553+0000","last_active":"2024-04-25T12:38:00.299479+0000","last_peered":"2024-04-25T12:38:00.299479+0000","last_clean":"2024-04-25T12:38:00.299479+0000","last_became_active":"2024-04-25T12:36:13.810659+0000","last_became_peered":"2024-04-25T12:36:13.810659+0000","last_unstale":"2024-04-25T12:38:00.299479+0000","last_undegraded":"2024-04-25T12:38:00.299479+0000","last_fullsized":"2024-04-25T12:38:00.299479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:38:00.299243+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:38:00.299243+0000","last_clean_scrub_stamp":"2024-04-25T12:38:00.299243+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:42.645629+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.534658+0000","last_change":"2024-04-25T12:37:42.534730+0000","last_active":"2024-04-25T12:37:42.534658+0000","last_peered":"2024-04-25T12:37:42.534658+0000","last_clean":"2024-04-25T12:37:42.534658+0000","last_became_active":"2024-04-25T12:34:18.007799+0000","last_became_peered":"2024-04-25T12:34:18.007799+0000","last_unstale":"2024-04-25T12:37:42.534658+0000","last_undegraded":"2024-04-25T12:37:42.534658+0000","last_fullsized":"2024-04-25T12:37:42.534658+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:42.534404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:42.534404+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.534404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:27.027522+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.985108+0000","last_change":"2024-04-25T12:37:45.985136+0000","last_active":"2024-04-25T12:37:45.985108+0000","last_peered":"2024-04-25T12:37:45.985108+0000","last_clean":"2024-04-25T12:37:45.985108+0000","last_became_active":"2024-04-25T12:34:13.935643+0000","last_became_peered":"2024-04-25T12:34:13.935643+0000","last_unstale":"2024-04-25T12:37:45.985108+0000","last_undegraded":"2024-04-25T12:37:45.985108+0000","last_fullsized":"2024-04-25T12:37:45.985108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:45.985017+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:45.985017+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.985017+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:50.743875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.300758+0000","last_change":"2024-04-25T12:37:50.300835+0000","last_active":"2024-04-25T12:37:50.300758+0000","last_peered":"2024-04-25T12:37:50.300758+0000","last_clean":"2024-04-25T12:37:50.300758+0000","last_became_active":"2024-04-25T12:34:12.921545+0000","last_became_peered":"2024-04-25T12:34:12.921545+0000","last_unstale":"2024-04-25T12:37:50.300758+0000","last_undegraded":"2024-04-25T12:37:50.300758+0000","last_fullsized":"2024-04-25T12:37:50.300758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:50.300483+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:50.300483+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.300483+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:48.164030+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.616920+0000","last_change":"2024-04-25T12:37:49.616986+0000","last_active":"2024-04-25T12:37:49.616920+0000","last_peered":"2024-04-25T12:37:49.616920+0000","last_clean":"2024-04-25T12:37:49.616920+0000","last_became_active":"2024-04-25T12:37:01.225693+0000","last_became_peered":"2024-04-25T12:37:01.225693+0000","last_unstale":"2024-04-25T12:37:49.616920+0000","last_undegraded":"2024-04-25T12:37:49.616920+0000","last_fullsized":"2024-04-25T12:37:49.616920+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.616716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.616716+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.616716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:12.195154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":179,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.005935+0000","last_change":"2024-04-25T12:37:54.005963+0000","last_active":"2024-04-25T12:37:54.005935+0000","last_peered":"2024-04-25T12:37:54.005935+0000","last_clean":"2024-04-25T12:37:54.005935+0000","last_became_active":"2024-04-25T12:37:01.205935+0000","last_became_peered":"2024-04-25T12:37:01.205935+0000","last_unstale":"2024-04-25T12:37:54.005935+0000","last_undegraded":"2024-04-25T12:37:54.005935+0000","last_fullsized":"2024-04-25T12:37:54.005935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:54.005836+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:54.005836+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.005836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:26.054516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"93'14","reported_seq":180,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.919619+0000","last_change":"2024-04-25T12:37:42.919705+0000","last_active":"2024-04-25T12:37:42.919619+0000","last_peered":"2024-04-25T12:37:42.919619+0000","last_clean":"2024-04-25T12:37:42.919619+0000","last_became_active":"2024-04-25T12:34:12.918286+0000","last_became_peered":"2024-04-25T12:34:12.918286+0000","last_unstale":"2024-04-25T12:37:42.919619+0000","last_undegraded":"2024-04-25T12:37:42.919619+0000","last_fullsized":"2024-04-25T12:37:42.919619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'14","last_scrub_stamp":"2024-04-25T12:37:42.919493+0000","last_deep_scrub":"93'14","last_deep_scrub_stamp":"2024-04-25T12:37:42.919493+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.919493+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:21.298661+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:56.007299+0000","last_change":"2024-04-25T12:37:56.007299+0000","last_active":"2024-04-25T12:37:56.007299+0000","last_peered":"2024-04-25T12:37:56.007299+0000","last_clean":"2024-04-25T12:37:56.007299+0000","last_became_active":"2024-04-25T12:37:01.225536+0000","last_became_peered":"2024-04-25T12:37:01.225536+0000","last_unstale":"2024-04-25T12:37:56.007299+0000","last_undegraded":"2024-04-25T12:37:56.007299+0000","last_fullsized":"2024-04-25T12:37:56.007299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:56.007156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:56.007156+0000","last_clean_scrub_stamp":"2024-04-25T12:37:56.007156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:32.960994+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.842873+0000","last_change":"2024-04-25T12:37:40.842901+0000","last_active":"2024-04-25T12:37:40.842873+0000","last_peered":"2024-04-25T12:37:40.842873+0000","last_clean":"2024-04-25T12:37:40.842873+0000","last_became_active":"2024-04-25T12:37:01.208215+0000","last_became_peered":"2024-04-25T12:37:01.208215+0000","last_unstale":"2024-04-25T12:37:40.842873+0000","last_undegraded":"2024-04-25T12:37:40.842873+0000","last_fullsized":"2024-04-25T12:37:40.842873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:40.842764+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:40.842764+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.842764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:58.351731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'2","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.327279+0000","last_change":"2024-04-25T12:37:47.327349+0000","last_active":"2024-04-25T12:37:47.327279+0000","last_peered":"2024-04-25T12:37:47.327279+0000","last_clean":"2024-04-25T12:37:47.327279+0000","last_became_active":"2024-04-25T12:34:12.900236+0000","last_became_peered":"2024-04-25T12:34:12.900236+0000","last_unstale":"2024-04-25T12:37:47.327279+0000","last_undegraded":"2024-04-25T12:37:47.327279+0000","last_fullsized":"2024-04-25T12:37:47.327279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:47.327082+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:47.327082+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.327082+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:42.666016+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.791015+0000","last_change":"2024-04-25T12:37:47.791046+0000","last_active":"2024-04-25T12:37:47.791015+0000","last_peered":"2024-04-25T12:37:47.791015+0000","last_clean":"2024-04-25T12:37:47.791015+0000","last_became_active":"2024-04-25T12:37:01.224415+0000","last_became_peered":"2024-04-25T12:37:01.224415+0000","last_unstale":"2024-04-25T12:37:47.791015+0000","last_undegraded":"2024-04-25T12:37:47.791015+0000","last_fullsized":"2024-04-25T12:37:47.791015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:47.790902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:47.790902+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.790902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.170616+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":177,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.629887+0000","last_change":"2024-04-25T12:37:47.630014+0000","last_active":"2024-04-25T12:37:47.629887+0000","last_peered":"2024-04-25T12:37:47.629887+0000","last_clean":"2024-04-25T12:37:47.629887+0000","last_became_active":"2024-04-25T12:37:01.221788+0000","last_became_peered":"2024-04-25T12:37:01.221788+0000","last_unstale":"2024-04-25T12:37:47.629887+0000","last_undegraded":"2024-04-25T12:37:47.629887+0000","last_fullsized":"2024-04-25T12:37:47.629887+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:47.629654+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:47.629654+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.629654+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:42.007123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":128,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.749427+0000","last_change":"2024-04-25T12:37:52.749427+0000","last_active":"2024-04-25T12:37:52.749427+0000","last_peered":"2024-04-25T12:37:52.749427+0000","last_clean":"2024-04-25T12:37:52.749427+0000","last_became_active":"2024-04-25T12:37:01.211565+0000","last_became_peered":"2024-04-25T12:37:01.211565+0000","last_unstale":"2024-04-25T12:37:52.749427+0000","last_undegraded":"2024-04-25T12:37:52.749427+0000","last_fullsized":"2024-04-25T12:37:52.749427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:52.749288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:52.749288+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.749288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:25.097466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.717046+0000","last_change":"2024-04-25T12:37:52.717078+0000","last_active":"2024-04-25T12:37:52.717046+0000","last_peered":"2024-04-25T12:37:52.717046+0000","last_clean":"2024-04-25T12:37:52.717046+0000","last_became_active":"2024-04-25T12:37:01.229112+0000","last_became_peered":"2024-04-25T12:37:01.229112+0000","last_unstale":"2024-04-25T12:37:52.717046+0000","last_undegraded":"2024-04-25T12:37:52.717046+0000","last_fullsized":"2024-04-25T12:37:52.717046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:52.716935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:52.716935+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.716935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:20.052004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.974312+0000","last_change":"2024-04-25T12:37:52.974344+0000","last_active":"2024-04-25T12:37:52.974312+0000","last_peered":"2024-04-25T12:37:52.974312+0000","last_clean":"2024-04-25T12:37:52.974312+0000","last_became_active":"2024-04-25T12:37:01.223137+0000","last_became_peered":"2024-04-25T12:37:01.223137+0000","last_unstale":"2024-04-25T12:37:52.974312+0000","last_undegraded":"2024-04-25T12:37:52.974312+0000","last_fullsized":"2024-04-25T12:37:52.974312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:52.974188+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:52.974188+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.974188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:51.443245+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:38:01.325583+0000","last_change":"2024-04-25T12:38:01.325655+0000","last_active":"2024-04-25T12:38:01.325583+0000","last_peered":"2024-04-25T12:38:01.325583+0000","last_clean":"2024-04-25T12:38:01.325583+0000","last_became_active":"2024-04-25T12:37:01.227163+0000","last_became_peered":"2024-04-25T12:37:01.227163+0000","last_unstale":"2024-04-25T12:38:01.325583+0000","last_undegraded":"2024-04-25T12:38:01.325583+0000","last_fullsized":"2024-04-25T12:38:01.325583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-04-25T12:38:01.325285+0000","last_deep_scrub":"25'1","last_deep_scrub_stamp":"2024-04-25T12:38:01.325285+0000","last_clean_scrub_stamp":"2024-04-25T12:38:01.325285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:37.492749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.733293+0000","last_change":"2024-04-25T12:37:49.733321+0000","last_active":"2024-04-25T12:37:49.733293+0000","last_peered":"2024-04-25T12:37:49.733293+0000","last_clean":"2024-04-25T12:37:49.733293+0000","last_became_active":"2024-04-25T12:37:01.226507+0000","last_became_peered":"2024-04-25T12:37:01.226507+0000","last_unstale":"2024-04-25T12:37:49.733293+0000","last_undegraded":"2024-04-25T12:37:49.733293+0000","last_fullsized":"2024-04-25T12:37:49.733293+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.733195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.733195+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.733195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:48.202252+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.875867+0000","last_change":"2024-04-25T12:37:44.875867+0000","last_active":"2024-04-25T12:37:44.875867+0000","last_peered":"2024-04-25T12:37:44.875867+0000","last_clean":"2024-04-25T12:37:44.875867+0000","last_became_active":"2024-04-25T12:36:00.622028+0000","last_became_peered":"2024-04-25T12:36:00.622028+0000","last_unstale":"2024-04-25T12:37:44.875867+0000","last_undegraded":"2024-04-25T12:37:44.875867+0000","last_fullsized":"2024-04-25T12:37:44.875867+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:44.875712+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:44.875712+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.875712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:13.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'10","reported_seq":185,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.010739+0000","last_change":"2024-04-25T12:37:48.010778+0000","last_active":"2024-04-25T12:37:48.010739+0000","last_peered":"2024-04-25T12:37:48.010739+0000","last_clean":"2024-04-25T12:37:48.010739+0000","last_became_active":"2024-04-25T12:36:00.625525+0000","last_became_peered":"2024-04-25T12:36:00.625525+0000","last_unstale":"2024-04-25T12:37:48.010739+0000","last_undegraded":"2024-04-25T12:37:48.010739+0000","last_fullsized":"2024-04-25T12:37:48.010739+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:48.010646+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:48.010646+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.010646+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:50.803361+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.823843+0000","last_change":"2024-04-25T12:37:45.823893+0000","last_active":"2024-04-25T12:37:45.823843+0000","last_peered":"2024-04-25T12:37:45.823843+0000","last_clean":"2024-04-25T12:37:45.823843+0000","last_became_active":"2024-04-25T12:37:01.221785+0000","last_became_peered":"2024-04-25T12:37:01.221785+0000","last_unstale":"2024-04-25T12:37:45.823843+0000","last_undegraded":"2024-04-25T12:37:45.823843+0000","last_fullsized":"2024-04-25T12:37:45.823843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.823677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.823677+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.823677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:15.139977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.810197+0000","last_change":"2024-04-25T12:37:46.810244+0000","last_active":"2024-04-25T12:37:46.810197+0000","last_peered":"2024-04-25T12:37:46.810197+0000","last_clean":"2024-04-25T12:37:46.810197+0000","last_became_active":"2024-04-25T12:37:01.224450+0000","last_became_peered":"2024-04-25T12:37:01.224450+0000","last_unstale":"2024-04-25T12:37:46.810197+0000","last_undegraded":"2024-04-25T12:37:46.810197+0000","last_fullsized":"2024-04-25T12:37:46.810197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:46.810016+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:46.810016+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.810016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:10.032062+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'4","reported_seq":175,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:57.287396+0000","last_change":"2024-04-25T12:37:57.287396+0000","last_active":"2024-04-25T12:37:57.287396+0000","last_peered":"2024-04-25T12:37:57.287396+0000","last_clean":"2024-04-25T12:37:57.287396+0000","last_became_active":"2024-04-25T12:36:00.629809+0000","last_became_peered":"2024-04-25T12:36:00.629809+0000","last_unstale":"2024-04-25T12:37:57.287396+0000","last_undegraded":"2024-04-25T12:37:57.287396+0000","last_fullsized":"2024-04-25T12:37:57.287396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:57.287131+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:57.287131+0000","last_clean_scrub_stamp":"2024-04-25T12:37:57.287131+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:32.753341+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":136,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:57.816832+0000","last_change":"2024-04-25T12:37:57.816832+0000","last_active":"2024-04-25T12:37:57.816832+0000","last_peered":"2024-04-25T12:37:57.816832+0000","last_clean":"2024-04-25T12:37:57.816832+0000","last_became_active":"2024-04-25T12:37:01.225995+0000","last_became_peered":"2024-04-25T12:37:01.225995+0000","last_unstale":"2024-04-25T12:37:57.816832+0000","last_undegraded":"2024-04-25T12:37:57.816832+0000","last_fullsized":"2024-04-25T12:37:57.816832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:57.816623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:57.816623+0000","last_clean_scrub_stamp":"2024-04-25T12:37:57.816623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:43.057242+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.897876+0000","last_change":"2024-04-25T12:37:39.897933+0000","last_active":"2024-04-25T12:37:39.897876+0000","last_peered":"2024-04-25T12:37:39.897876+0000","last_clean":"2024-04-25T12:37:39.897876+0000","last_became_active":"2024-04-25T12:34:13.958498+0000","last_became_peered":"2024-04-25T12:34:13.958498+0000","last_unstale":"2024-04-25T12:37:39.897876+0000","last_undegraded":"2024-04-25T12:37:39.897876+0000","last_fullsized":"2024-04-25T12:37:39.897876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:39.897724+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:39.897724+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.897724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:10.301458+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.277747+0000","last_change":"2024-04-25T12:37:51.277832+0000","last_active":"2024-04-25T12:37:51.277747+0000","last_peered":"2024-04-25T12:37:51.277747+0000","last_clean":"2024-04-25T12:37:51.277747+0000","last_became_active":"2024-04-25T12:34:12.919319+0000","last_became_peered":"2024-04-25T12:34:12.919319+0000","last_unstale":"2024-04-25T12:37:51.277747+0000","last_undegraded":"2024-04-25T12:37:51.277747+0000","last_fullsized":"2024-04-25T12:37:51.277747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:51.277541+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:51.277541+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.277541+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:34.828234+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"93'74","reported_seq":274,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.161333+0000","last_change":"2024-04-25T12:37:41.161333+0000","last_active":"2024-04-25T12:37:41.161333+0000","last_peered":"2024-04-25T12:37:41.161333+0000","last_clean":"2024-04-25T12:37:41.161333+0000","last_became_active":"2024-04-25T12:35:16.901068+0000","last_became_peered":"2024-04-25T12:35:16.901068+0000","last_unstale":"2024-04-25T12:37:41.161333+0000","last_undegraded":"2024-04-25T12:37:41.161333+0000","last_fullsized":"2024-04-25T12:37:41.161333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'74","last_scrub_stamp":"2024-04-25T12:37:41.161161+0000","last_deep_scrub":"93'74","last_deep_scrub_stamp":"2024-04-25T12:37:41.161161+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.161161+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:29.238865+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:55.297668+0000","last_change":"2024-04-25T12:37:55.297736+0000","last_active":"2024-04-25T12:37:55.297668+0000","last_peered":"2024-04-25T12:37:55.297668+0000","last_clean":"2024-04-25T12:37:55.297668+0000","last_became_active":"2024-04-25T12:34:18.011551+0000","last_became_peered":"2024-04-25T12:34:18.011551+0000","last_unstale":"2024-04-25T12:37:55.297668+0000","last_undegraded":"2024-04-25T12:37:55.297668+0000","last_fullsized":"2024-04-25T12:37:55.297668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:55.297420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:55.297420+0000","last_clean_scrub_stamp":"2024-04-25T12:37:55.297420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:54.025381+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.277780+0000","last_change":"2024-04-25T12:37:38.277843+0000","last_active":"2024-04-25T12:37:38.277780+0000","last_peered":"2024-04-25T12:37:38.277780+0000","last_clean":"2024-04-25T12:37:38.277780+0000","last_became_active":"2024-04-25T12:34:13.968297+0000","last_became_peered":"2024-04-25T12:34:13.968297+0000","last_unstale":"2024-04-25T12:37:38.277780+0000","last_undegraded":"2024-04-25T12:37:38.277780+0000","last_fullsized":"2024-04-25T12:37:38.277780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:38.277623+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:38.277623+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.277623+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:43.367276+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"93'5","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.716658+0000","last_change":"2024-04-25T12:37:54.716713+0000","last_active":"2024-04-25T12:37:54.716658+0000","last_peered":"2024-04-25T12:37:54.716658+0000","last_clean":"2024-04-25T12:37:54.716658+0000","last_became_active":"2024-04-25T12:37:01.230500+0000","last_became_peered":"2024-04-25T12:37:01.230500+0000","last_unstale":"2024-04-25T12:37:54.716658+0000","last_undegraded":"2024-04-25T12:37:54.716658+0000","last_fullsized":"2024-04-25T12:37:54.716658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:54.716491+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:54.716491+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.716491+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:13.341408+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.561279+0000","last_change":"2024-04-25T12:37:44.561407+0000","last_active":"2024-04-25T12:37:44.561279+0000","last_peered":"2024-04-25T12:37:44.561279+0000","last_clean":"2024-04-25T12:37:44.561279+0000","last_became_active":"2024-04-25T12:34:18.006498+0000","last_became_peered":"2024-04-25T12:34:18.006498+0000","last_unstale":"2024-04-25T12:37:44.561279+0000","last_undegraded":"2024-04-25T12:37:44.561279+0000","last_fullsized":"2024-04-25T12:37:44.561279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:44.561040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:44.561040+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.561040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:49.319898+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.030775+0000","last_change":"2024-04-25T12:37:46.030848+0000","last_active":"2024-04-25T12:37:46.030775+0000","last_peered":"2024-04-25T12:37:46.030775+0000","last_clean":"2024-04-25T12:37:46.030775+0000","last_became_active":"2024-04-25T12:37:01.219769+0000","last_became_peered":"2024-04-25T12:37:01.219769+0000","last_unstale":"2024-04-25T12:37:46.030775+0000","last_undegraded":"2024-04-25T12:37:46.030775+0000","last_fullsized":"2024-04-25T12:37:46.030775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:46.030579+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:46.030579+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.030579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:07.934799+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"93'2","reported_seq":132,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.788056+0000","last_change":"2024-04-25T12:37:39.788056+0000","last_active":"2024-04-25T12:37:39.788056+0000","last_peered":"2024-04-25T12:37:39.788056+0000","last_clean":"2024-04-25T12:37:39.788056+0000","last_became_active":"2024-04-25T12:36:00.633199+0000","last_became_peered":"2024-04-25T12:36:00.633199+0000","last_unstale":"2024-04-25T12:37:39.788056+0000","last_undegraded":"2024-04-25T12:37:39.788056+0000","last_fullsized":"2024-04-25T12:37:39.788056+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:39.787933+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:39.787933+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.787933+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:15.167038+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":122,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.819553+0000","last_change":"2024-04-25T12:37:41.819553+0000","last_active":"2024-04-25T12:37:41.819553+0000","last_peered":"2024-04-25T12:37:41.819553+0000","last_clean":"2024-04-25T12:37:41.819553+0000","last_became_active":"2024-04-25T12:36:00.642175+0000","last_became_peered":"2024-04-25T12:36:00.642175+0000","last_unstale":"2024-04-25T12:37:41.819553+0000","last_undegraded":"2024-04-25T12:37:41.819553+0000","last_fullsized":"2024-04-25T12:37:41.819553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:41.819413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:41.819413+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.819413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:25.906731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.109699+0000","last_change":"2024-04-25T12:37:41.109768+0000","last_active":"2024-04-25T12:37:41.109699+0000","last_peered":"2024-04-25T12:37:41.109699+0000","last_clean":"2024-04-25T12:37:41.109699+0000","last_became_active":"2024-04-25T12:34:13.974053+0000","last_became_peered":"2024-04-25T12:34:13.974053+0000","last_unstale":"2024-04-25T12:37:41.109699+0000","last_undegraded":"2024-04-25T12:37:41.109699+0000","last_fullsized":"2024-04-25T12:37:41.109699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:41.109493+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:41.109493+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.109493+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:28.193763+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"93'3","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.036750+0000","last_change":"2024-04-25T12:37:51.036791+0000","last_active":"2024-04-25T12:37:51.036750+0000","last_peered":"2024-04-25T12:37:51.036750+0000","last_clean":"2024-04-25T12:37:51.036750+0000","last_became_active":"2024-04-25T12:36:00.626743+0000","last_became_peered":"2024-04-25T12:36:00.626743+0000","last_unstale":"2024-04-25T12:37:51.036750+0000","last_undegraded":"2024-04-25T12:37:51.036750+0000","last_fullsized":"2024-04-25T12:37:51.036750+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:51.036648+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:51.036648+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.036648+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:13.615087+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.312595+0000","last_change":"2024-04-25T12:37:45.312666+0000","last_active":"2024-04-25T12:37:45.312595+0000","last_peered":"2024-04-25T12:37:45.312595+0000","last_clean":"2024-04-25T12:37:45.312595+0000","last_became_active":"2024-04-25T12:34:18.005810+0000","last_became_peered":"2024-04-25T12:34:18.005810+0000","last_unstale":"2024-04-25T12:37:45.312595+0000","last_undegraded":"2024-04-25T12:37:45.312595+0000","last_fullsized":"2024-04-25T12:37:45.312595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.312351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.312351+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.312351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:30.464269+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.528921+0000","last_change":"2024-04-25T12:37:41.528954+0000","last_active":"2024-04-25T12:37:41.528921+0000","last_peered":"2024-04-25T12:37:41.528921+0000","last_clean":"2024-04-25T12:37:41.528921+0000","last_became_active":"2024-04-25T12:34:13.976923+0000","last_became_peered":"2024-04-25T12:34:13.976923+0000","last_unstale":"2024-04-25T12:37:41.528921+0000","last_undegraded":"2024-04-25T12:37:41.528921+0000","last_fullsized":"2024-04-25T12:37:41.528921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:41.528815+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:41.528815+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.528815+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:19.907983+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"93'4","reported_seq":175,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.189153+0000","last_change":"2024-04-25T12:37:47.189233+0000","last_active":"2024-04-25T12:37:47.189153+0000","last_peered":"2024-04-25T12:37:47.189153+0000","last_clean":"2024-04-25T12:37:47.189153+0000","last_became_active":"2024-04-25T12:36:00.630856+0000","last_became_peered":"2024-04-25T12:36:00.630856+0000","last_unstale":"2024-04-25T12:37:47.189153+0000","last_undegraded":"2024-04-25T12:37:47.189153+0000","last_fullsized":"2024-04-25T12:37:47.189153+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:47.188957+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:47.188957+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.188957+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:37.249954+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.069928+0000","last_change":"2024-04-25T12:37:44.069994+0000","last_active":"2024-04-25T12:37:44.069928+0000","last_peered":"2024-04-25T12:37:44.069928+0000","last_clean":"2024-04-25T12:37:44.069928+0000","last_became_active":"2024-04-25T12:36:00.642396+0000","last_became_peered":"2024-04-25T12:36:00.642396+0000","last_unstale":"2024-04-25T12:37:44.069928+0000","last_undegraded":"2024-04-25T12:37:44.069928+0000","last_fullsized":"2024-04-25T12:37:44.069928+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:44.069727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:44.069727+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.069727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:32.356013+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.319568+0000","last_change":"2024-04-25T12:37:46.319692+0000","last_active":"2024-04-25T12:37:46.319568+0000","last_peered":"2024-04-25T12:37:46.319568+0000","last_clean":"2024-04-25T12:37:46.319568+0000","last_became_active":"2024-04-25T12:34:13.960720+0000","last_became_peered":"2024-04-25T12:34:13.960720+0000","last_unstale":"2024-04-25T12:37:46.319568+0000","last_undegraded":"2024-04-25T12:37:46.319568+0000","last_fullsized":"2024-04-25T12:37:46.319568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:46.319236+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:46.319236+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.319236+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:30.551780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"93'24","reported_seq":190,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.915728+0000","last_change":"2024-04-25T12:37:40.915776+0000","last_active":"2024-04-25T12:37:40.915728+0000","last_peered":"2024-04-25T12:37:40.915728+0000","last_clean":"2024-04-25T12:37:40.915728+0000","last_became_active":"2024-04-25T12:34:12.896423+0000","last_became_peered":"2024-04-25T12:34:12.896423+0000","last_unstale":"2024-04-25T12:37:40.915728+0000","last_undegraded":"2024-04-25T12:37:40.915728+0000","last_fullsized":"2024-04-25T12:37:40.915728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'24","last_scrub_stamp":"2024-04-25T12:37:40.915632+0000","last_deep_scrub":"93'24","last_deep_scrub_stamp":"2024-04-25T12:37:40.915632+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.915632+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:49.082187+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:58.839954+0000","last_change":"2024-04-25T12:37:58.839984+0000","last_active":"2024-04-25T12:37:58.839954+0000","last_peered":"2024-04-25T12:37:58.839954+0000","last_clean":"2024-04-25T12:37:58.839954+0000","last_became_active":"2024-04-25T12:37:01.226028+0000","last_became_peered":"2024-04-25T12:37:01.226028+0000","last_unstale":"2024-04-25T12:37:58.839954+0000","last_undegraded":"2024-04-25T12:37:58.839954+0000","last_fullsized":"2024-04-25T12:37:58.839954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:58.839849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:58.839849+0000","last_clean_scrub_stamp":"2024-04-25T12:37:58.839849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:11.680197+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":175,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.112291+0000","last_change":"2024-04-25T12:37:42.112357+0000","last_active":"2024-04-25T12:37:42.112291+0000","last_peered":"2024-04-25T12:37:42.112291+0000","last_clean":"2024-04-25T12:37:42.112291+0000","last_became_active":"2024-04-25T12:36:00.637779+0000","last_became_peered":"2024-04-25T12:36:00.637779+0000","last_unstale":"2024-04-25T12:37:42.112291+0000","last_undegraded":"2024-04-25T12:37:42.112291+0000","last_fullsized":"2024-04-25T12:37:42.112291+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:42.112085+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:42.112085+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.112085+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:47.435031+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:36.180578+0000","last_change":"2024-04-25T12:37:36.180654+0000","last_active":"2024-04-25T12:37:36.180578+0000","last_peered":"2024-04-25T12:37:36.180578+0000","last_clean":"2024-04-25T12:37:36.180578+0000","last_became_active":"2024-04-25T12:34:12.913814+0000","last_became_peered":"2024-04-25T12:34:12.913814+0000","last_unstale":"2024-04-25T12:37:36.180578+0000","last_undegraded":"2024-04-25T12:37:36.180578+0000","last_fullsized":"2024-04-25T12:37:36.180578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:36.180353+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:36.180353+0000","last_clean_scrub_stamp":"2024-04-25T12:37:36.180353+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:38.779021+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:55.812440+0000","last_change":"2024-04-25T12:37:55.812491+0000","last_active":"2024-04-25T12:37:55.812440+0000","last_peered":"2024-04-25T12:37:55.812440+0000","last_clean":"2024-04-25T12:37:55.812440+0000","last_became_active":"2024-04-25T12:37:01.220205+0000","last_became_peered":"2024-04-25T12:37:01.220205+0000","last_unstale":"2024-04-25T12:37:55.812440+0000","last_undegraded":"2024-04-25T12:37:55.812440+0000","last_fullsized":"2024-04-25T12:37:55.812440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:55.812291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:55.812291+0000","last_clean_scrub_stamp":"2024-04-25T12:37:55.812291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:56.611591+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":181,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:38:02.364590+0000","last_change":"2024-04-25T12:38:02.364639+0000","last_active":"2024-04-25T12:38:02.364590+0000","last_peered":"2024-04-25T12:38:02.364590+0000","last_clean":"2024-04-25T12:38:02.364590+0000","last_became_active":"2024-04-25T12:37:01.231301+0000","last_became_peered":"2024-04-25T12:37:01.231301+0000","last_unstale":"2024-04-25T12:38:02.364590+0000","last_undegraded":"2024-04-25T12:38:02.364590+0000","last_fullsized":"2024-04-25T12:38:02.364590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'15","last_scrub_stamp":"2024-04-25T12:38:02.364443+0000","last_deep_scrub":"93'15","last_deep_scrub_stamp":"2024-04-25T12:38:02.364443+0000","last_clean_scrub_stamp":"2024-04-25T12:38:02.364443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:09.246081+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"93'5","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.326435+0000","last_change":"2024-04-25T12:37:48.326543+0000","last_active":"2024-04-25T12:37:48.326435+0000","last_peered":"2024-04-25T12:37:48.326435+0000","last_clean":"2024-04-25T12:37:48.326435+0000","last_became_active":"2024-04-25T12:34:12.910717+0000","last_became_peered":"2024-04-25T12:34:12.910717+0000","last_unstale":"2024-04-25T12:37:48.326435+0000","last_undegraded":"2024-04-25T12:37:48.326435+0000","last_fullsized":"2024-04-25T12:37:48.326435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:48.326251+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:48.326251+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.326251+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:57.118982+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.744240+0000","last_change":"2024-04-25T12:37:48.744269+0000","last_active":"2024-04-25T12:37:48.744240+0000","last_peered":"2024-04-25T12:37:48.744240+0000","last_clean":"2024-04-25T12:37:48.744240+0000","last_became_active":"2024-04-25T12:37:01.225477+0000","last_became_peered":"2024-04-25T12:37:01.225477+0000","last_unstale":"2024-04-25T12:37:48.744240+0000","last_undegraded":"2024-04-25T12:37:48.744240+0000","last_fullsized":"2024-04-25T12:37:48.744240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:48.744135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:48.744135+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.744135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:39.037534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.707884+0000","last_change":"2024-04-25T12:37:51.707916+0000","last_active":"2024-04-25T12:37:51.707884+0000","last_peered":"2024-04-25T12:37:51.707884+0000","last_clean":"2024-04-25T12:37:51.707884+0000","last_became_active":"2024-04-25T12:37:01.229013+0000","last_became_peered":"2024-04-25T12:37:01.229013+0000","last_unstale":"2024-04-25T12:37:51.707884+0000","last_undegraded":"2024-04-25T12:37:51.707884+0000","last_fullsized":"2024-04-25T12:37:51.707884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:51.707748+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:51.707748+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.707748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.273796+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'10","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.909253+0000","last_change":"2024-04-25T12:37:42.909291+0000","last_active":"2024-04-25T12:37:42.909253+0000","last_peered":"2024-04-25T12:37:42.909253+0000","last_clean":"2024-04-25T12:37:42.909253+0000","last_became_active":"2024-04-25T12:34:12.915896+0000","last_became_peered":"2024-04-25T12:34:12.915896+0000","last_unstale":"2024-04-25T12:37:42.909253+0000","last_undegraded":"2024-04-25T12:37:42.909253+0000","last_fullsized":"2024-04-25T12:37:42.909253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:42.909147+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:42.909147+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.909147+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:55.844694+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":82,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.065122+0000","last_change":"2024-04-25T12:37:45.065190+0000","last_active":"2024-04-25T12:37:45.065122+0000","last_peered":"2024-04-25T12:37:45.065122+0000","last_clean":"2024-04-25T12:37:45.065122+0000","last_became_active":"2024-04-25T12:36:32.380921+0000","last_became_peered":"2024-04-25T12:36:32.380921+0000","last_unstale":"2024-04-25T12:37:45.065122+0000","last_undegraded":"2024-04-25T12:37:45.065122+0000","last_fullsized":"2024-04-25T12:37:45.065122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.064846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.064846+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.064846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:58.000393+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7,3],"acting":[1,0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.041998+0000","last_change":"2024-04-25T12:37:50.042049+0000","last_active":"2024-04-25T12:37:50.041998+0000","last_peered":"2024-04-25T12:37:50.041998+0000","last_clean":"2024-04-25T12:37:50.041998+0000","last_became_active":"2024-04-25T12:37:01.207260+0000","last_became_peered":"2024-04-25T12:37:01.207260+0000","last_unstale":"2024-04-25T12:37:50.041998+0000","last_undegraded":"2024-04-25T12:37:50.041998+0000","last_fullsized":"2024-04-25T12:37:50.041998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:50.041892+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:50.041892+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.041892+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:13.263938+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"93'32","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:37.480272+0000","last_change":"2024-04-25T12:37:37.480323+0000","last_active":"2024-04-25T12:37:37.480272+0000","last_peered":"2024-04-25T12:37:37.480272+0000","last_clean":"2024-04-25T12:37:37.480272+0000","last_became_active":"2024-04-25T12:34:12.914246+0000","last_became_peered":"2024-04-25T12:34:12.914246+0000","last_unstale":"2024-04-25T12:37:37.480272+0000","last_undegraded":"2024-04-25T12:37:37.480272+0000","last_fullsized":"2024-04-25T12:37:37.480272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'32","last_scrub_stamp":"2024-04-25T12:37:37.480157+0000","last_deep_scrub":"93'32","last_deep_scrub_stamp":"2024-04-25T12:37:37.480157+0000","last_clean_scrub_stamp":"2024-04-25T12:37:37.480157+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:09.030080+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37279,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"70'3","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.011086+0000","last_change":"2024-04-25T12:37:52.011119+0000","last_active":"2024-04-25T12:37:52.011086+0000","last_peered":"2024-04-25T12:37:52.011086+0000","last_clean":"2024-04-25T12:37:52.011086+0000","last_became_active":"2024-04-25T12:36:00.635516+0000","last_became_peered":"2024-04-25T12:36:00.635516+0000","last_unstale":"2024-04-25T12:37:52.011086+0000","last_undegraded":"2024-04-25T12:37:52.011086+0000","last_fullsized":"2024-04-25T12:37:52.011086+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'3","last_scrub_stamp":"2024-04-25T12:37:52.010966+0000","last_deep_scrub":"70'3","last_deep_scrub_stamp":"2024-04-25T12:37:52.010966+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.010966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:53.126937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.297625+0000","last_change":"2024-04-25T12:37:42.297763+0000","last_active":"2024-04-25T12:37:42.297625+0000","last_peered":"2024-04-25T12:37:42.297625+0000","last_clean":"2024-04-25T12:37:42.297625+0000","last_became_active":"2024-04-25T12:34:13.960761+0000","last_became_peered":"2024-04-25T12:34:13.960761+0000","last_unstale":"2024-04-25T12:37:42.297625+0000","last_undegraded":"2024-04-25T12:37:42.297625+0000","last_fullsized":"2024-04-25T12:37:42.297625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'11","last_scrub_stamp":"2024-04-25T12:37:42.297357+0000","last_deep_scrub":"93'11","last_deep_scrub_stamp":"2024-04-25T12:37:42.297357+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.297357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:27.148777+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"93'10","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.954256+0000","last_change":"2024-04-25T12:37:41.954308+0000","last_active":"2024-04-25T12:37:41.954256+0000","last_peered":"2024-04-25T12:37:41.954256+0000","last_clean":"2024-04-25T12:37:41.954256+0000","last_became_active":"2024-04-25T12:34:12.896237+0000","last_became_peered":"2024-04-25T12:34:12.896237+0000","last_unstale":"2024-04-25T12:37:41.954256+0000","last_undegraded":"2024-04-25T12:37:41.954256+0000","last_fullsized":"2024-04-25T12:37:41.954256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:41.954139+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:41.954139+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.954139+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:30.240425+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.565512+0000","last_change":"2024-04-25T12:37:43.565587+0000","last_active":"2024-04-25T12:37:43.565512+0000","last_peered":"2024-04-25T12:37:43.565512+0000","last_clean":"2024-04-25T12:37:43.565512+0000","last_became_active":"2024-04-25T12:34:18.008263+0000","last_became_peered":"2024-04-25T12:34:18.008263+0000","last_unstale":"2024-04-25T12:37:43.565512+0000","last_undegraded":"2024-04-25T12:37:43.565512+0000","last_fullsized":"2024-04-25T12:37:43.565512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:43.565246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:43.565246+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.565246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:40.458016+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.771863+0000","last_change":"2024-04-25T12:37:53.771912+0000","last_active":"2024-04-25T12:37:53.771863+0000","last_peered":"2024-04-25T12:37:53.771863+0000","last_clean":"2024-04-25T12:37:53.771863+0000","last_became_active":"2024-04-25T12:37:01.211503+0000","last_became_peered":"2024-04-25T12:37:01.211503+0000","last_unstale":"2024-04-25T12:37:53.771863+0000","last_undegraded":"2024-04-25T12:37:53.771863+0000","last_fullsized":"2024-04-25T12:37:53.771863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:53.771727+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:53.771727+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.771727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:22.048282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"93'3","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.147359+0000","last_change":"2024-04-25T12:37:38.147400+0000","last_active":"2024-04-25T12:37:38.147359+0000","last_peered":"2024-04-25T12:37:38.147359+0000","last_clean":"2024-04-25T12:37:38.147359+0000","last_became_active":"2024-04-25T12:34:12.927770+0000","last_became_peered":"2024-04-25T12:34:12.927770+0000","last_unstale":"2024-04-25T12:37:38.147359+0000","last_undegraded":"2024-04-25T12:37:38.147359+0000","last_fullsized":"2024-04-25T12:37:38.147359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:38.147245+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:38.147245+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.147245+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:23.765389+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.956066+0000","last_change":"2024-04-25T12:37:45.956095+0000","last_active":"2024-04-25T12:37:45.956066+0000","last_peered":"2024-04-25T12:37:45.956066+0000","last_clean":"2024-04-25T12:37:45.956066+0000","last_became_active":"2024-04-25T12:34:18.007358+0000","last_became_peered":"2024-04-25T12:34:18.007358+0000","last_unstale":"2024-04-25T12:37:45.956066+0000","last_undegraded":"2024-04-25T12:37:45.956066+0000","last_fullsized":"2024-04-25T12:37:45.956066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:45.955956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:45.955956+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.955956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:38.246257+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.557493+0000","last_change":"2024-04-25T12:37:40.557550+0000","last_active":"2024-04-25T12:37:40.557493+0000","last_peered":"2024-04-25T12:37:40.557493+0000","last_clean":"2024-04-25T12:37:40.557493+0000","last_became_active":"2024-04-25T12:34:13.955809+0000","last_became_peered":"2024-04-25T12:34:13.955809+0000","last_unstale":"2024-04-25T12:37:40.557493+0000","last_undegraded":"2024-04-25T12:37:40.557493+0000","last_fullsized":"2024-04-25T12:37:40.557493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:40.557352+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:40.557352+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.557352+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:37.679667+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:37.060297+0000","last_change":"2024-04-25T12:37:37.060351+0000","last_active":"2024-04-25T12:37:37.060297+0000","last_peered":"2024-04-25T12:37:37.060297+0000","last_clean":"2024-04-25T12:37:37.060297+0000","last_became_active":"2024-04-25T12:34:12.922315+0000","last_became_peered":"2024-04-25T12:34:12.922315+0000","last_unstale":"2024-04-25T12:37:37.060297+0000","last_undegraded":"2024-04-25T12:37:37.060297+0000","last_fullsized":"2024-04-25T12:37:37.060297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:37.060183+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:37.060183+0000","last_clean_scrub_stamp":"2024-04-25T12:37:37.060183+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:48.920888+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.007482+0000","last_change":"2024-04-25T12:37:50.007511+0000","last_active":"2024-04-25T12:37:50.007482+0000","last_peered":"2024-04-25T12:37:50.007482+0000","last_clean":"2024-04-25T12:37:50.007482+0000","last_became_active":"2024-04-25T12:34:18.006156+0000","last_became_peered":"2024-04-25T12:34:18.006156+0000","last_unstale":"2024-04-25T12:37:50.007482+0000","last_undegraded":"2024-04-25T12:37:50.007482+0000","last_fullsized":"2024-04-25T12:37:50.007482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:50.007382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:50.007382+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.007382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.553152+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:53.033308+0000","last_change":"2024-04-25T12:37:53.033308+0000","last_active":"2024-04-25T12:37:53.033308+0000","last_peered":"2024-04-25T12:37:53.033308+0000","last_clean":"2024-04-25T12:37:53.033308+0000","last_became_active":"2024-04-25T12:36:00.641863+0000","last_became_peered":"2024-04-25T12:36:00.641863+0000","last_unstale":"2024-04-25T12:37:53.033308+0000","last_undegraded":"2024-04-25T12:37:53.033308+0000","last_fullsized":"2024-04-25T12:37:53.033308+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:53.033179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:53.033179+0000","last_clean_scrub_stamp":"2024-04-25T12:37:53.033179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.772002+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"93'10","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.050930+0000","last_change":"2024-04-25T12:37:48.050958+0000","last_active":"2024-04-25T12:37:48.050930+0000","last_peered":"2024-04-25T12:37:48.050930+0000","last_clean":"2024-04-25T12:37:48.050930+0000","last_became_active":"2024-04-25T12:34:13.952127+0000","last_became_peered":"2024-04-25T12:34:13.952127+0000","last_unstale":"2024-04-25T12:37:48.050930+0000","last_undegraded":"2024-04-25T12:37:48.050930+0000","last_fullsized":"2024-04-25T12:37:48.050930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:48.050836+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:48.050836+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.050836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:41.921894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:52.279884+0000","last_change":"2024-04-25T12:37:52.279949+0000","last_active":"2024-04-25T12:37:52.279884+0000","last_peered":"2024-04-25T12:37:52.279884+0000","last_clean":"2024-04-25T12:37:52.279884+0000","last_became_active":"2024-04-25T12:34:12.922476+0000","last_became_peered":"2024-04-25T12:34:12.922476+0000","last_unstale":"2024-04-25T12:37:52.279884+0000","last_undegraded":"2024-04-25T12:37:52.279884+0000","last_fullsized":"2024-04-25T12:37:52.279884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-04-25T12:37:52.279678+0000","last_deep_scrub":"25'1","last_deep_scrub_stamp":"2024-04-25T12:37:52.279678+0000","last_clean_scrub_stamp":"2024-04-25T12:37:52.279678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:39.030279+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:59.326879+0000","last_change":"2024-04-25T12:37:59.327007+0000","last_active":"2024-04-25T12:37:59.326879+0000","last_peered":"2024-04-25T12:37:59.326879+0000","last_clean":"2024-04-25T12:37:59.326879+0000","last_became_active":"2024-04-25T12:36:00.646765+0000","last_became_peered":"2024-04-25T12:36:00.646765+0000","last_unstale":"2024-04-25T12:37:59.326879+0000","last_undegraded":"2024-04-25T12:37:59.326879+0000","last_fullsized":"2024-04-25T12:37:59.326879+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:59.326628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:59.326628+0000","last_clean_scrub_stamp":"2024-04-25T12:37:59.326628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:51.233568+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.772271+0000","last_change":"2024-04-25T12:37:51.772300+0000","last_active":"2024-04-25T12:37:51.772271+0000","last_peered":"2024-04-25T12:37:51.772271+0000","last_clean":"2024-04-25T12:37:51.772271+0000","last_became_active":"2024-04-25T12:37:01.204428+0000","last_became_peered":"2024-04-25T12:37:01.204428+0000","last_unstale":"2024-04-25T12:37:51.772271+0000","last_undegraded":"2024-04-25T12:37:51.772271+0000","last_fullsized":"2024-04-25T12:37:51.772271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:51.772172+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:51.772172+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.772172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:12.920832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.782287+0000","last_change":"2024-04-25T12:37:44.782353+0000","last_active":"2024-04-25T12:37:44.782287+0000","last_peered":"2024-04-25T12:37:44.782287+0000","last_clean":"2024-04-25T12:37:44.782287+0000","last_became_active":"2024-04-25T12:37:01.205546+0000","last_became_peered":"2024-04-25T12:37:01.205546+0000","last_unstale":"2024-04-25T12:37:44.782287+0000","last_undegraded":"2024-04-25T12:37:44.782287+0000","last_fullsized":"2024-04-25T12:37:44.782287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'3","last_scrub_stamp":"2024-04-25T12:37:44.782175+0000","last_deep_scrub":"93'3","last_deep_scrub_stamp":"2024-04-25T12:37:44.782175+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.782175+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:28.854601+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'4","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.336067+0000","last_change":"2024-04-25T12:37:49.336200+0000","last_active":"2024-04-25T12:37:49.336067+0000","last_peered":"2024-04-25T12:37:49.336067+0000","last_clean":"2024-04-25T12:37:49.336067+0000","last_became_active":"2024-04-25T12:34:12.919183+0000","last_became_peered":"2024-04-25T12:34:12.919183+0000","last_unstale":"2024-04-25T12:37:49.336067+0000","last_undegraded":"2024-04-25T12:37:49.336067+0000","last_fullsized":"2024-04-25T12:37:49.336067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:49.335832+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:49.335832+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.335832+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:40.387712+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.004747+0000","last_change":"2024-04-25T12:37:49.004814+0000","last_active":"2024-04-25T12:37:49.004747+0000","last_peered":"2024-04-25T12:37:49.004747+0000","last_clean":"2024-04-25T12:37:49.004747+0000","last_became_active":"2024-04-25T12:34:13.966569+0000","last_became_peered":"2024-04-25T12:34:13.966569+0000","last_unstale":"2024-04-25T12:37:49.004747+0000","last_undegraded":"2024-04-25T12:37:49.004747+0000","last_fullsized":"2024-04-25T12:37:49.004747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:49.004615+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:49.004615+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.004615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:43.382083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":183,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.988039+0000","last_change":"2024-04-25T12:37:54.988087+0000","last_active":"2024-04-25T12:37:54.988039+0000","last_peered":"2024-04-25T12:37:54.988039+0000","last_clean":"2024-04-25T12:37:54.988039+0000","last_became_active":"2024-04-25T12:37:01.208634+0000","last_became_peered":"2024-04-25T12:37:01.208634+0000","last_unstale":"2024-04-25T12:37:54.988039+0000","last_undegraded":"2024-04-25T12:37:54.988039+0000","last_fullsized":"2024-04-25T12:37:54.988039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:54.987888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:54.987888+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.987888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:05.903127+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:56.787840+0000","last_change":"2024-04-25T12:37:56.787880+0000","last_active":"2024-04-25T12:37:56.787840+0000","last_peered":"2024-04-25T12:37:56.787840+0000","last_clean":"2024-04-25T12:37:56.787840+0000","last_became_active":"2024-04-25T12:37:01.211956+0000","last_became_peered":"2024-04-25T12:37:01.211956+0000","last_unstale":"2024-04-25T12:37:56.787840+0000","last_undegraded":"2024-04-25T12:37:56.787840+0000","last_fullsized":"2024-04-25T12:37:56.787840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:56.787737+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:56.787737+0000","last_clean_scrub_stamp":"2024-04-25T12:37:56.787737+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:28.564345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.066560+0000","last_change":"2024-04-25T12:37:43.066631+0000","last_active":"2024-04-25T12:37:43.066560+0000","last_peered":"2024-04-25T12:37:43.066560+0000","last_clean":"2024-04-25T12:37:43.066560+0000","last_became_active":"2024-04-25T12:36:00.638354+0000","last_became_peered":"2024-04-25T12:36:00.638354+0000","last_unstale":"2024-04-25T12:37:43.066560+0000","last_undegraded":"2024-04-25T12:37:43.066560+0000","last_fullsized":"2024-04-25T12:37:43.066560+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:43.066320+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:43.066320+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.066320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:47.150390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:49.030619+0000","last_change":"2024-04-25T12:37:49.030670+0000","last_active":"2024-04-25T12:37:49.030619+0000","last_peered":"2024-04-25T12:37:49.030619+0000","last_clean":"2024-04-25T12:37:49.030619+0000","last_became_active":"2024-04-25T12:36:00.642108+0000","last_became_peered":"2024-04-25T12:36:00.642108+0000","last_unstale":"2024-04-25T12:37:49.030619+0000","last_undegraded":"2024-04-25T12:37:49.030619+0000","last_fullsized":"2024-04-25T12:37:49.030619+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:37:49.030483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:37:49.030483+0000","last_clean_scrub_stamp":"2024-04-25T12:37:49.030483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:33.523601+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":181,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.586417+0000","last_change":"2024-04-25T12:37:48.586511+0000","last_active":"2024-04-25T12:37:48.586417+0000","last_peered":"2024-04-25T12:37:48.586417+0000","last_clean":"2024-04-25T12:37:48.586417+0000","last_became_active":"2024-04-25T12:37:01.223541+0000","last_became_peered":"2024-04-25T12:37:01.223541+0000","last_unstale":"2024-04-25T12:37:48.586417+0000","last_undegraded":"2024-04-25T12:37:48.586417+0000","last_fullsized":"2024-04-25T12:37:48.586417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:48.586208+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:48.586208+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.586208+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:39.131979+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.917504+0000","last_change":"2024-04-25T12:37:44.917539+0000","last_active":"2024-04-25T12:37:44.917504+0000","last_peered":"2024-04-25T12:37:44.917504+0000","last_clean":"2024-04-25T12:37:44.917504+0000","last_became_active":"2024-04-25T12:34:13.954385+0000","last_became_peered":"2024-04-25T12:34:13.954385+0000","last_unstale":"2024-04-25T12:37:44.917504+0000","last_undegraded":"2024-04-25T12:37:44.917504+0000","last_fullsized":"2024-04-25T12:37:44.917504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'12","last_scrub_stamp":"2024-04-25T12:37:44.917411+0000","last_deep_scrub":"93'12","last_deep_scrub_stamp":"2024-04-25T12:37:44.917411+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.917411+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:32.483433+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:58.334758+0000","last_change":"2024-04-25T12:37:58.334868+0000","last_active":"2024-04-25T12:37:58.334758+0000","last_peered":"2024-04-25T12:37:58.334758+0000","last_clean":"2024-04-25T12:37:58.334758+0000","last_became_active":"2024-04-25T12:36:00.640669+0000","last_became_peered":"2024-04-25T12:36:00.640669+0000","last_unstale":"2024-04-25T12:37:58.334758+0000","last_undegraded":"2024-04-25T12:37:58.334758+0000","last_fullsized":"2024-04-25T12:37:58.334758+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:58.334560+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:58.334560+0000","last_clean_scrub_stamp":"2024-04-25T12:37:58.334560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:37.930964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.886776+0000","last_change":"2024-04-25T12:37:46.886776+0000","last_active":"2024-04-25T12:37:46.886776+0000","last_peered":"2024-04-25T12:37:46.886776+0000","last_clean":"2024-04-25T12:37:46.886776+0000","last_became_active":"2024-04-25T12:36:00.626618+0000","last_became_peered":"2024-04-25T12:36:00.626618+0000","last_unstale":"2024-04-25T12:37:46.886776+0000","last_undegraded":"2024-04-25T12:37:46.886776+0000","last_fullsized":"2024-04-25T12:37:46.886776+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:46.886648+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:46.886648+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.886648+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:35.666915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":54,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:50.687025+0000","last_change":"2024-04-25T12:37:50.687058+0000","last_active":"2024-04-25T12:37:50.687025+0000","last_peered":"2024-04-25T12:37:50.687025+0000","last_clean":"2024-04-25T12:37:50.687025+0000","last_became_active":"2024-04-25T12:37:01.228041+0000","last_became_peered":"2024-04-25T12:37:01.228041+0000","last_unstale":"2024-04-25T12:37:50.687025+0000","last_undegraded":"2024-04-25T12:37:50.687025+0000","last_fullsized":"2024-04-25T12:37:50.687025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:50.686934+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:50.686934+0000","last_clean_scrub_stamp":"2024-04-25T12:37:50.686934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:31.966695+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.832855+0000","last_change":"2024-04-25T12:37:41.832903+0000","last_active":"2024-04-25T12:37:41.832855+0000","last_peered":"2024-04-25T12:37:41.832855+0000","last_clean":"2024-04-25T12:37:41.832855+0000","last_became_active":"2024-04-25T12:37:01.209714+0000","last_became_peered":"2024-04-25T12:37:01.209714+0000","last_unstale":"2024-04-25T12:37:41.832855+0000","last_undegraded":"2024-04-25T12:37:41.832855+0000","last_fullsized":"2024-04-25T12:37:41.832855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:41.832740+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:41.832740+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.832740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:34.123041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:51.542783+0000","last_change":"2024-04-25T12:37:51.542917+0000","last_active":"2024-04-25T12:37:51.542783+0000","last_peered":"2024-04-25T12:37:51.542783+0000","last_clean":"2024-04-25T12:37:51.542783+0000","last_became_active":"2024-04-25T12:37:01.231631+0000","last_became_peered":"2024-04-25T12:37:01.231631+0000","last_unstale":"2024-04-25T12:37:51.542783+0000","last_undegraded":"2024-04-25T12:37:51.542783+0000","last_fullsized":"2024-04-25T12:37:51.542783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'15","last_scrub_stamp":"2024-04-25T12:37:51.542496+0000","last_deep_scrub":"93'15","last_deep_scrub_stamp":"2024-04-25T12:37:51.542496+0000","last_clean_scrub_stamp":"2024-04-25T12:37:51.542496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:49.281319+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":120,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.848546+0000","last_change":"2024-04-25T12:37:42.848546+0000","last_active":"2024-04-25T12:37:42.848546+0000","last_peered":"2024-04-25T12:37:42.848546+0000","last_clean":"2024-04-25T12:37:42.848546+0000","last_became_active":"2024-04-25T12:36:00.629676+0000","last_became_peered":"2024-04-25T12:36:00.629676+0000","last_unstale":"2024-04-25T12:37:42.848546+0000","last_undegraded":"2024-04-25T12:37:42.848546+0000","last_fullsized":"2024-04-25T12:37:42.848546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'2","last_scrub_stamp":"2024-04-25T12:37:42.848421+0000","last_deep_scrub":"93'2","last_deep_scrub_stamp":"2024-04-25T12:37:42.848421+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.848421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.888338+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:56.296195+0000","last_change":"2024-04-25T12:37:56.296263+0000","last_active":"2024-04-25T12:37:56.296195+0000","last_peered":"2024-04-25T12:37:56.296195+0000","last_clean":"2024-04-25T12:37:56.296195+0000","last_became_active":"2024-04-25T12:36:00.635052+0000","last_became_peered":"2024-04-25T12:36:00.635052+0000","last_unstale":"2024-04-25T12:37:56.296195+0000","last_undegraded":"2024-04-25T12:37:56.296195+0000","last_fullsized":"2024-04-25T12:37:56.296195+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:56.295994+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:56.295994+0000","last_clean_scrub_stamp":"2024-04-25T12:37:56.295994+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:01.362500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.888875+0000","last_change":"2024-04-25T12:37:43.888875+0000","last_active":"2024-04-25T12:37:43.888875+0000","last_peered":"2024-04-25T12:37:43.888875+0000","last_clean":"2024-04-25T12:37:43.888875+0000","last_became_active":"2024-04-25T12:36:00.620490+0000","last_became_peered":"2024-04-25T12:36:00.620490+0000","last_unstale":"2024-04-25T12:37:43.888875+0000","last_undegraded":"2024-04-25T12:37:43.888875+0000","last_fullsized":"2024-04-25T12:37:43.888875+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:43.888723+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:43.888723+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.888723+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.051839+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.130564+0000","last_change":"2024-04-25T12:37:39.130692+0000","last_active":"2024-04-25T12:37:39.130564+0000","last_peered":"2024-04-25T12:37:39.130564+0000","last_clean":"2024-04-25T12:37:39.130564+0000","last_became_active":"2024-04-25T12:34:13.966001+0000","last_became_peered":"2024-04-25T12:34:13.966001+0000","last_unstale":"2024-04-25T12:37:39.130564+0000","last_undegraded":"2024-04-25T12:37:39.130564+0000","last_fullsized":"2024-04-25T12:37:39.130564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:39.130333+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:39.130333+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.130333+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:05.653918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":175,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.009725+0000","last_change":"2024-04-25T12:37:47.009756+0000","last_active":"2024-04-25T12:37:47.009725+0000","last_peered":"2024-04-25T12:37:47.009725+0000","last_clean":"2024-04-25T12:37:47.009725+0000","last_became_active":"2024-04-25T12:34:13.944409+0000","last_became_peered":"2024-04-25T12:34:13.944409+0000","last_unstale":"2024-04-25T12:37:47.009725+0000","last_undegraded":"2024-04-25T12:37:47.009725+0000","last_fullsized":"2024-04-25T12:37:47.009725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'11","last_scrub_stamp":"2024-04-25T12:37:47.009632+0000","last_deep_scrub":"93'11","last_deep_scrub_stamp":"2024-04-25T12:37:47.009632+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.009632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:31.389346+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.077661+0000","last_change":"2024-04-25T12:37:43.077728+0000","last_active":"2024-04-25T12:37:43.077661+0000","last_peered":"2024-04-25T12:37:43.077661+0000","last_clean":"2024-04-25T12:37:43.077661+0000","last_became_active":"2024-04-25T12:34:13.975170+0000","last_became_peered":"2024-04-25T12:34:13.975170+0000","last_unstale":"2024-04-25T12:37:43.077661+0000","last_undegraded":"2024-04-25T12:37:43.077661+0000","last_fullsized":"2024-04-25T12:37:43.077661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:43.077443+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:43.077443+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.077443+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:23.639432+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.311840+0000","last_change":"2024-04-25T12:37:44.311906+0000","last_active":"2024-04-25T12:37:44.311840+0000","last_peered":"2024-04-25T12:37:44.311840+0000","last_clean":"2024-04-25T12:37:44.311840+0000","last_became_active":"2024-04-25T12:34:13.960840+0000","last_became_peered":"2024-04-25T12:34:13.960840+0000","last_unstale":"2024-04-25T12:37:44.311840+0000","last_undegraded":"2024-04-25T12:37:44.311840+0000","last_fullsized":"2024-04-25T12:37:44.311840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:44.311650+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:44.311650+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.311650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:04.441351+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":174,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.869180+0000","last_change":"2024-04-25T12:37:43.869224+0000","last_active":"2024-04-25T12:37:43.869180+0000","last_peered":"2024-04-25T12:37:43.869180+0000","last_clean":"2024-04-25T12:37:43.869180+0000","last_became_active":"2024-04-25T12:34:13.942062+0000","last_became_peered":"2024-04-25T12:34:13.942062+0000","last_unstale":"2024-04-25T12:37:43.869180+0000","last_undegraded":"2024-04-25T12:37:43.869180+0000","last_fullsized":"2024-04-25T12:37:43.869180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:43.869038+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:43.869038+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.869038+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:14.272772+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:41.300879+0000","last_change":"2024-04-25T12:37:41.300948+0000","last_active":"2024-04-25T12:37:41.300879+0000","last_peered":"2024-04-25T12:37:41.300879+0000","last_clean":"2024-04-25T12:37:41.300879+0000","last_became_active":"2024-04-25T12:34:13.968255+0000","last_became_peered":"2024-04-25T12:34:13.968255+0000","last_unstale":"2024-04-25T12:37:41.300879+0000","last_undegraded":"2024-04-25T12:37:41.300879+0000","last_fullsized":"2024-04-25T12:37:41.300879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:41.300665+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:41.300665+0000","last_clean_scrub_stamp":"2024-04-25T12:37:41.300665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:13.476225+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.258248+0000","last_change":"2024-04-25T12:37:54.258320+0000","last_active":"2024-04-25T12:37:54.258248+0000","last_peered":"2024-04-25T12:37:54.258248+0000","last_clean":"2024-04-25T12:37:54.258248+0000","last_became_active":"2024-04-25T12:34:13.950079+0000","last_became_peered":"2024-04-25T12:34:13.950079+0000","last_unstale":"2024-04-25T12:37:54.258248+0000","last_undegraded":"2024-04-25T12:37:54.258248+0000","last_fullsized":"2024-04-25T12:37:54.258248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:54.258048+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:54.258048+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.258048+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:30.534013+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:47.847759+0000","last_change":"2024-04-25T12:37:47.847759+0000","last_active":"2024-04-25T12:37:47.847759+0000","last_peered":"2024-04-25T12:37:47.847759+0000","last_clean":"2024-04-25T12:37:47.847759+0000","last_became_active":"2024-04-25T12:36:00.627985+0000","last_became_peered":"2024-04-25T12:36:00.627985+0000","last_unstale":"2024-04-25T12:37:47.847759+0000","last_undegraded":"2024-04-25T12:37:47.847759+0000","last_fullsized":"2024-04-25T12:37:47.847759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:47.847623+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:47.847623+0000","last_clean_scrub_stamp":"2024-04-25T12:37:47.847623+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:00.750853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"93'8","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.836184+0000","last_change":"2024-04-25T12:37:42.836230+0000","last_active":"2024-04-25T12:37:42.836184+0000","last_peered":"2024-04-25T12:37:42.836184+0000","last_clean":"2024-04-25T12:37:42.836184+0000","last_became_active":"2024-04-25T12:37:01.210867+0000","last_became_peered":"2024-04-25T12:37:01.210867+0000","last_unstale":"2024-04-25T12:37:42.836184+0000","last_undegraded":"2024-04-25T12:37:42.836184+0000","last_fullsized":"2024-04-25T12:37:42.836184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:42.836028+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:42.836028+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.836028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:39.958343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:43.796893+0000","last_change":"2024-04-25T12:37:43.796925+0000","last_active":"2024-04-25T12:37:43.796893+0000","last_peered":"2024-04-25T12:37:43.796893+0000","last_clean":"2024-04-25T12:37:43.796893+0000","last_became_active":"2024-04-25T12:37:01.216348+0000","last_became_peered":"2024-04-25T12:37:01.216348+0000","last_unstale":"2024-04-25T12:37:43.796893+0000","last_undegraded":"2024-04-25T12:37:43.796893+0000","last_fullsized":"2024-04-25T12:37:43.796893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'10","last_scrub_stamp":"2024-04-25T12:37:43.796780+0000","last_deep_scrub":"93'10","last_deep_scrub_stamp":"2024-04-25T12:37:43.796780+0000","last_clean_scrub_stamp":"2024-04-25T12:37:43.796780+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:22.175947+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":169,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:44.094566+0000","last_change":"2024-04-25T12:37:44.094632+0000","last_active":"2024-04-25T12:37:44.094566+0000","last_peered":"2024-04-25T12:37:44.094566+0000","last_clean":"2024-04-25T12:37:44.094566+0000","last_became_active":"2024-04-25T12:34:13.975616+0000","last_became_peered":"2024-04-25T12:34:13.975616+0000","last_unstale":"2024-04-25T12:37:44.094566+0000","last_undegraded":"2024-04-25T12:37:44.094566+0000","last_fullsized":"2024-04-25T12:37:44.094566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:44.094339+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:44.094339+0000","last_clean_scrub_stamp":"2024-04-25T12:37:44.094339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:06.409183+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.138466+0000","last_change":"2024-04-25T12:37:45.138545+0000","last_active":"2024-04-25T12:37:45.138466+0000","last_peered":"2024-04-25T12:37:45.138466+0000","last_clean":"2024-04-25T12:37:45.138466+0000","last_became_active":"2024-04-25T12:34:13.976111+0000","last_became_peered":"2024-04-25T12:34:13.976111+0000","last_unstale":"2024-04-25T12:37:45.138466+0000","last_undegraded":"2024-04-25T12:37:45.138466+0000","last_fullsized":"2024-04-25T12:37:45.138466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:45.138176+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:45.138176+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.138176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:48.969915+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":178,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:42.081626+0000","last_change":"2024-04-25T12:37:42.081690+0000","last_active":"2024-04-25T12:37:42.081626+0000","last_peered":"2024-04-25T12:37:42.081626+0000","last_clean":"2024-04-25T12:37:42.081626+0000","last_became_active":"2024-04-25T12:34:13.974283+0000","last_became_peered":"2024-04-25T12:34:13.974283+0000","last_unstale":"2024-04-25T12:37:42.081626+0000","last_undegraded":"2024-04-25T12:37:42.081626+0000","last_fullsized":"2024-04-25T12:37:42.081626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'16","last_scrub_stamp":"2024-04-25T12:37:42.081396+0000","last_deep_scrub":"93'16","last_deep_scrub_stamp":"2024-04-25T12:37:42.081396+0000","last_clean_scrub_stamp":"2024-04-25T12:37:42.081396+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:04.044060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.093321+0000","last_change":"2024-04-25T12:37:38.093385+0000","last_active":"2024-04-25T12:37:38.093321+0000","last_peered":"2024-04-25T12:37:38.093321+0000","last_clean":"2024-04-25T12:37:38.093321+0000","last_became_active":"2024-04-25T12:34:13.952101+0000","last_became_peered":"2024-04-25T12:34:13.952101+0000","last_unstale":"2024-04-25T12:37:38.093321+0000","last_undegraded":"2024-04-25T12:37:38.093321+0000","last_fullsized":"2024-04-25T12:37:38.093321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'11","last_scrub_stamp":"2024-04-25T12:37:38.093157+0000","last_deep_scrub":"93'11","last_deep_scrub_stamp":"2024-04-25T12:37:38.093157+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.093157+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.126985+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.807314+0000","last_change":"2024-04-25T12:37:48.807350+0000","last_active":"2024-04-25T12:37:48.807314+0000","last_peered":"2024-04-25T12:37:48.807314+0000","last_clean":"2024-04-25T12:37:48.807314+0000","last_became_active":"2024-04-25T12:36:00.628221+0000","last_became_peered":"2024-04-25T12:36:00.628221+0000","last_unstale":"2024-04-25T12:37:48.807314+0000","last_undegraded":"2024-04-25T12:37:48.807314+0000","last_fullsized":"2024-04-25T12:37:48.807314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-04-25T12:37:48.807218+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-04-25T12:37:48.807218+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.807218+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:24.754964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:40.117939+0000","last_change":"2024-04-25T12:37:40.117972+0000","last_active":"2024-04-25T12:37:40.117939+0000","last_peered":"2024-04-25T12:37:40.117939+0000","last_clean":"2024-04-25T12:37:40.117939+0000","last_became_active":"2024-04-25T12:34:13.957804+0000","last_became_peered":"2024-04-25T12:34:13.957804+0000","last_unstale":"2024-04-25T12:37:40.117939+0000","last_undegraded":"2024-04-25T12:37:40.117939+0000","last_fullsized":"2024-04-25T12:37:40.117939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'5","last_scrub_stamp":"2024-04-25T12:37:40.117836+0000","last_deep_scrub":"93'5","last_deep_scrub_stamp":"2024-04-25T12:37:40.117836+0000","last_clean_scrub_stamp":"2024-04-25T12:37:40.117836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:39.198481+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":168,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:39.524361+0000","last_change":"2024-04-25T12:37:39.524387+0000","last_active":"2024-04-25T12:37:39.524361+0000","last_peered":"2024-04-25T12:37:39.524361+0000","last_clean":"2024-04-25T12:37:39.524361+0000","last_became_active":"2024-04-25T12:34:13.934860+0000","last_became_peered":"2024-04-25T12:34:13.934860+0000","last_unstale":"2024-04-25T12:37:39.524361+0000","last_undegraded":"2024-04-25T12:37:39.524361+0000","last_fullsized":"2024-04-25T12:37:39.524361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'6","last_scrub_stamp":"2024-04-25T12:37:39.524277+0000","last_deep_scrub":"93'6","last_deep_scrub_stamp":"2024-04-25T12:37:39.524277+0000","last_clean_scrub_stamp":"2024-04-25T12:37:39.524277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:47.690832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":173,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:48.194536+0000","last_change":"2024-04-25T12:37:48.194606+0000","last_active":"2024-04-25T12:37:48.194536+0000","last_peered":"2024-04-25T12:37:48.194536+0000","last_clean":"2024-04-25T12:37:48.194536+0000","last_became_active":"2024-04-25T12:36:00.631327+0000","last_became_peered":"2024-04-25T12:36:00.631327+0000","last_unstale":"2024-04-25T12:37:48.194536+0000","last_undegraded":"2024-04-25T12:37:48.194536+0000","last_fullsized":"2024-04-25T12:37:48.194536+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:48.194276+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:48.194276+0000","last_clean_scrub_stamp":"2024-04-25T12:37:48.194276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:20.369836+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":171,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:38.506814+0000","last_change":"2024-04-25T12:37:38.506814+0000","last_active":"2024-04-25T12:37:38.506814+0000","last_peered":"2024-04-25T12:37:38.506814+0000","last_clean":"2024-04-25T12:37:38.506814+0000","last_became_active":"2024-04-25T12:34:13.931811+0000","last_became_peered":"2024-04-25T12:34:13.931811+0000","last_unstale":"2024-04-25T12:37:38.506814+0000","last_undegraded":"2024-04-25T12:37:38.506814+0000","last_fullsized":"2024-04-25T12:37:38.506814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:38.506564+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:38.506564+0000","last_clean_scrub_stamp":"2024-04-25T12:37:38.506564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.015531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:45.925127+0000","last_change":"2024-04-25T12:37:45.925127+0000","last_active":"2024-04-25T12:37:45.925127+0000","last_peered":"2024-04-25T12:37:45.925127+0000","last_clean":"2024-04-25T12:37:45.925127+0000","last_became_active":"2024-04-25T12:36:00.623584+0000","last_became_peered":"2024-04-25T12:36:00.623584+0000","last_unstale":"2024-04-25T12:37:45.925127+0000","last_undegraded":"2024-04-25T12:37:45.925127+0000","last_fullsized":"2024-04-25T12:37:45.925127+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'7","last_scrub_stamp":"2024-04-25T12:37:45.924964+0000","last_deep_scrub":"93'7","last_deep_scrub_stamp":"2024-04-25T12:37:45.924964+0000","last_clean_scrub_stamp":"2024-04-25T12:37:45.924964+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:56.579137+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":132,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:54.768830+0000","last_change":"2024-04-25T12:37:54.768830+0000","last_active":"2024-04-25T12:37:54.768830+0000","last_peered":"2024-04-25T12:37:54.768830+0000","last_clean":"2024-04-25T12:37:54.768830+0000","last_became_active":"2024-04-25T12:37:01.215569+0000","last_became_peered":"2024-04-25T12:37:01.215569+0000","last_unstale":"2024-04-25T12:37:54.768830+0000","last_undegraded":"2024-04-25T12:37:54.768830+0000","last_fullsized":"2024-04-25T12:37:54.768830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'4","last_scrub_stamp":"2024-04-25T12:37:54.768695+0000","last_deep_scrub":"93'4","last_deep_scrub_stamp":"2024-04-25T12:37:54.768695+0000","last_clean_scrub_stamp":"2024-04-25T12:37:54.768695+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:09.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":170,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T12:37:46.160533+0000","last_change":"2024-04-25T12:37:46.160605+0000","last_active":"2024-04-25T12:37:46.160533+0000","last_peered":"2024-04-25T12:37:46.160533+0000","last_clean":"2024-04-25T12:37:46.160533+0000","last_became_active":"2024-04-25T12:34:13.980351+0000","last_became_peered":"2024-04-25T12:34:13.980351+0000","last_unstale":"2024-04-25T12:37:46.160533+0000","last_undegraded":"2024-04-25T12:37:46.160533+0000","last_fullsized":"2024-04-25T12:37:46.160533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'8","last_scrub_stamp":"2024-04-25T12:37:46.160303+0000","last_deep_scrub":"93'8","last_deep_scrub_stamp":"2024-04-25T12:37:46.160303+0000","last_clean_scrub_stamp":"2024-04-25T12:37:46.160303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:21.482991+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":501,"ondisk_log_size":501,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41301,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":171,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2408,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":82602,"data_compressed":3398,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":6615,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":99288,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1110016,"data_stored":579587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1110016,"data_stored":579095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1122304,"data_stored":579681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542199,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1105920,"data_stored":579565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":47,"seq":201863462942,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29324,"kb_used_data":1092,"kb_used_omap":9,"kb_used_meta":28214,"kb_avail":94342516,"statfs":{"total":96636764160,"available":96606736384,"internally_reserved":0,"allocated":1118208,"data_stored":579577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9369,"internal_metadata":28892007},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1216512,"data_stored":1207576,"data_compressed":35879,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542199,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1142784,"data_stored":616256,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1167360,"data_stored":1168889,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32554,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":37895,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":38847,"data_compressed":1699,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6615,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:38:23.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T12:38:23.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.479+0000 7f46a1200640 1 Processor -- start 2024-04-25T12:38:23.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.480+0000 7f46a1200640 1 -- start start 2024-04-25T12:38:23.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f46a1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f46a1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f469c0c9f00 0x7f469c0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f46a1200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469c15b380 con 0x7f469c0730e0 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f46a1200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f469c074b20 con 0x7f469c0c9f00 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f46a1200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f469c0ca780 con 0x7f469c154650 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f4699400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f469c0c9f00 0x7f469c0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f4699e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f4699400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f469c0c9f00 0x7f469c0c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43560/0 (socket says 172.21.15.123:43560) 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f4699400640 1 -- 172.21.15.123:0/2597362357 learned_addr learned my addr 172.21.15.123:0/2597362357 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.482+0000 7f469a800640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f469c0730e0 0x7f469c0708f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.123:44400/0 (socket says 172.21.15.123:44400) 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.483+0000 7f4699400640 1 -- 172.21.15.123:0/2597362357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 msgr2=0x7f469c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.483+0000 7f4699400640 1 --2- 172.21.15.123:0/2597362357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c15ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.483+0000 7f4699400640 1 -- 172.21.15.123:0/2597362357 >> v1:172.21.15.153:6789/0 conn(0x7f469c0730e0 legacy=0x7f469c0708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.483+0000 7f4698a00640 1 -- 172.21.15.123:0/2597362357 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3627099461 0 0) 0x7f4694001110 con 0x7f469c0730e0 2024-04-25T12:38:23.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.483+0000 7f4699400640 1 -- 172.21.15.123:0/2597362357 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c0caa60 con 0x7f469c0c9f00 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.484+0000 7f4699400640 1 --2- 172.21.15.123:0/2597362357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0c9f00 0x7f469c0c8170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4688009e50 tx=0x7f468802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a6d4b79f1e528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.484+0000 7f4698a00640 1 -- 172.21.15.123:0/2597362357 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688036070 con 0x7f469c0c9f00 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.484+0000 7f4698a00640 1 -- 172.21.15.123:0/2597362357 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4688031600 con 0x7f469c0c9f00 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.484+0000 7f4698a00640 1 -- 172.21.15.123:0/2597362357 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46880319d0 con 0x7f469c0c9f00 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 -- 172.21.15.123:0/2597362357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0c9f00 msgr2=0x7f469c0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 --2- 172.21.15.123:0/2597362357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0c9f00 0x7f469c0c8170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4688009e50 tx=0x7f468802a7a0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 -- 172.21.15.123:0/2597362357 shutdown_connections 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 --2- 172.21.15.123:0/2597362357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0c9f00 0x7f469c0c8170 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 --2- 172.21.15.123:0/2597362357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 -- 172.21.15.123:0/2597362357 >> 172.21.15.123:0/2597362357 conn(0x7f469c06f440 msgr2=0x7f469c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:23.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 -- 172.21.15.123:0/2597362357 shutdown_connections 2024-04-25T12:38:23.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.485+0000 7f46a1200640 1 -- 172.21.15.123:0/2597362357 wait complete. 2024-04-25T12:38:23.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.487+0000 7f46a1200640 1 Processor -- start 2024-04-25T12:38:23.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.487+0000 7f46a1200640 1 -- start start 2024-04-25T12:38:23.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f46a1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0730e0 0x7f469c14c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:23.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f46a1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c0c9f00 0x7f469c15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:23.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f46a1200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:23.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f46a1200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f469c15e010 con 0x7f469c0730e0 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f46a1200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f469c15dd30 con 0x7f469c0c9f00 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f46a1200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f469c15e2f0 con 0x7f469c154650 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f4699e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0730e0 0x7f469c14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f4699e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0730e0 0x7f469c14c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43568/0 (socket says 172.21.15.123:43568) 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f4699e00640 1 -- 172.21.15.123:0/2232870627 learned_addr learned my addr 172.21.15.123:0/2232870627 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f469a800640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.489+0000 7f4699400640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c0c9f00 0x7f469c15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4699e00640 1 -- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 msgr2=0x7f469c14d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4699e00640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c14d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4699e00640 1 -- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c0c9f00 msgr2=0x7f469c15b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4699e00640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c0c9f00 0x7f469c15b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4699e00640 1 -- 172.21.15.123:0/2232870627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c0c4620 con 0x7f469c0730e0 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4699e00640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0730e0 0x7f469c14c800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e8a0 tx=0x7f468c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:23.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c017070 con 0x7f469c0730e0 2024-04-25T12:38:23.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f468c00c940 con 0x7f469c0730e0 2024-04-25T12:38:23.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c1f63d0 con 0x7f469c0730e0 2024-04-25T12:38:23.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.490+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c00cd30 con 0x7f469c0730e0 2024-04-25T12:38:23.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.491+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c1f3020 con 0x7f469c0730e0 2024-04-25T12:38:23.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.492+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c076700 con 0x7f469c0730e0 2024-04-25T12:38:23.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.493+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f468c01a070 con 0x7f469c0730e0 2024-04-25T12:38:23.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.494+0000 7f4692000640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f467c0744b0 0x7f467c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:23.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.495+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f468c095950 con 0x7f469c0730e0 2024-04-25T12:38:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.495+0000 7f4699400640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f467c0744b0 0x7f467c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:23.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.496+0000 7f4699400640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f467c0744b0 0x7f467c0769c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4688033000 tx=0x7f4688009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.500+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f468c05f550 con 0x7f469c0730e0 2024-04-25T12:38:23.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.700+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f469c0c6770 con 0x7f469c0730e0 2024-04-25T12:38:23.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.704+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f468c063850 con 0x7f469c0730e0 2024-04-25T12:38:23.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.705+0000 7f4692000640 1 -- 172.21.15.123:0/2232870627 <== mon.0 v2:172.21.15.123:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f468c09b080 con 0x7f469c0730e0 2024-04-25T12:38:23.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.708+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f467c0744b0 msgr2=0x7f467c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.708+0000 7f46a1200640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f467c0744b0 0x7f467c0769c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4688033000 tx=0x7f4688009000 comp rx=0 tx=0).stop 2024-04-25T12:38:23.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.708+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0730e0 msgr2=0x7f469c14c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:23.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.708+0000 7f46a1200640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0730e0 0x7f469c14c800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e8a0 tx=0x7f468c00ed60 comp rx=0 tx=0).stop 2024-04-25T12:38:23.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 shutdown_connections 2024-04-25T12:38:23.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f467c0744b0 0x7f467c0769c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c154650 0x7f469c14d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c0c9f00 0x7f469c15b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 --2- 172.21.15.123:0/2232870627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f469c0730e0 0x7f469c14c800 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:23.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 >> 172.21.15.123:0/2232870627 conn(0x7f469c06f440 msgr2=0x7f469c0c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:23.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 shutdown_connections 2024-04-25T12:38:23.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:23.710+0000 7f46a1200640 1 -- 172.21.15.123:0/2232870627 wait complete. 2024-04-25T12:38:23.734 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T12:38:23.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:38:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.000+0000 7f7a40200640 1 Processor -- start 2024-04-25T12:38:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.000+0000 7f7a40200640 1 -- start start 2024-04-25T12:38:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.002+0000 7f7a40200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a38074870 0x7f7a38154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.002+0000 7f7a40200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a3815b3b0 con 0x7f7a380c4d50 2024-04-25T12:38:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.002+0000 7f7a40200640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a38073ce0 con 0x7f7a3815afa0 2024-04-25T12:38:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.002+0000 7f7a40200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7a380c61f0 con 0x7f7a38074870 2024-04-25T12:38:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.002+0000 7f7a3d800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7a380c4d50 0x7f7a380c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58718/0 (socket says 172.21.15.123:58718) 2024-04-25T12:38:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.002+0000 7f7a3d800640 1 -- 172.21.15.123:0/2969417184 learned_addr learned my addr 172.21.15.123:0/2969417184 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:38:24.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.002+0000 7f7a3ce00640 1 --2- 172.21.15.123:0/2969417184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a38074870 0x7f7a38154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.003+0000 7f7a37e00640 1 -- 172.21.15.123:0/2969417184 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3436432536 0 0) 0x7f7a3815b3b0 con 0x7f7a380c4d50 2024-04-25T12:38:24.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.003+0000 7f7a37e00640 1 -- 172.21.15.123:0/2969417184 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a1c003520 con 0x7f7a380c4d50 2024-04-25T12:38:24.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.003+0000 7f7a37e00640 1 -- 172.21.15.123:0/2969417184 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405653104 0 0) 0x7f7a38073ce0 con 0x7f7a3815afa0 2024-04-25T12:38:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.003+0000 7f7a37e00640 1 -- 172.21.15.123:0/2969417184 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a3815b3b0 con 0x7f7a3815afa0 2024-04-25T12:38:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.003+0000 7f7a3ce00640 1 -- 172.21.15.123:0/2969417184 >> v1:172.21.15.153:6789/0 conn(0x7f7a3815afa0 legacy=0x7f7a380cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.003+0000 7f7a3ce00640 1 -- 172.21.15.123:0/2969417184 >> v1:172.21.15.123:6789/0 conn(0x7f7a380c4d50 legacy=0x7f7a380c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.003+0000 7f7a3ce00640 1 -- 172.21.15.123:0/2969417184 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a38073fc0 con 0x7f7a38074870 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.004+0000 7f7a3ce00640 1 --2- 172.21.15.123:0/2969417184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a38074870 0x7f7a38154890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a240097d0 tx=0x7f7a2402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0b40867d473173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.004+0000 7f7a37e00640 1 -- 172.21.15.123:0/2969417184 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24037070 con 0x7f7a38074870 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.004+0000 7f7a37e00640 1 -- 172.21.15.123:0/2969417184 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7a24032440 con 0x7f7a38074870 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.004+0000 7f7a37e00640 1 -- 172.21.15.123:0/2969417184 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24032810 con 0x7f7a38074870 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.005+0000 7f7a40200640 1 -- 172.21.15.123:0/2969417184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a38074870 msgr2=0x7f7a38154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.005+0000 7f7a40200640 1 --2- 172.21.15.123:0/2969417184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a38074870 0x7f7a38154890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a240097d0 tx=0x7f7a2402a290 comp rx=0 tx=0).stop 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.005+0000 7f7a40200640 1 -- 172.21.15.123:0/2969417184 shutdown_connections 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.005+0000 7f7a40200640 1 --2- 172.21.15.123:0/2969417184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a38074870 0x7f7a38154890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.005+0000 7f7a40200640 1 -- 172.21.15.123:0/2969417184 >> 172.21.15.123:0/2969417184 conn(0x7f7a3806c750 msgr2=0x7f7a3806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.005+0000 7f7a40200640 1 -- 172.21.15.123:0/2969417184 shutdown_connections 2024-04-25T12:38:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.005+0000 7f7a40200640 1 -- 172.21.15.123:0/2969417184 wait complete. 2024-04-25T12:38:24.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.007+0000 7f7a40200640 1 Processor -- start 2024-04-25T12:38:24.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.008+0000 7f7a40200640 1 -- start start 2024-04-25T12:38:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a40200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a38074870 0x7f7a38146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a40200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a380c4d50 0x7f7a38147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a40200640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a3815afa0 0x7f7a381f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a40200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380c8460 con 0x7f7a38074870 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a40200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380c87d0 con 0x7f7a380c4d50 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a40200640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7a380c9370 con 0x7f7a3815afa0 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a3d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a38074870 0x7f7a38146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a3d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a38074870 0x7f7a38146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43582/0 (socket says 172.21.15.123:43582) 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a3d800640 1 -- 172.21.15.123:0/3776459545 learned_addr learned my addr 172.21.15.123:0/3776459545 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a3e200640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a3815afa0 0x7f7a381f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.009+0000 7f7a3ce00640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a380c4d50 0x7f7a38147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a3d800640 1 -- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a3815afa0 msgr2=0x7f7a381f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a3d800640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a3815afa0 0x7f7a381f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a3d800640 1 -- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a380c4d50 msgr2=0x7f7a38147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a3d800640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a380c4d50 0x7f7a38147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a3d800640 1 -- 172.21.15.123:0/3776459545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a380c3320 con 0x7f7a38074870 2024-04-25T12:38:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a3d800640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a38074870 0x7f7a38146f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c410 tx=0x7f7a2800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a35600640 1 -- 172.21.15.123:0/3776459545 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28017070 con 0x7f7a38074870 2024-04-25T12:38:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a35600640 1 -- 172.21.15.123:0/3776459545 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7a28005ce0 con 0x7f7a38074870 2024-04-25T12:38:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a380c35c0 con 0x7f7a38074870 2024-04-25T12:38:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.010+0000 7f7a35600640 1 -- 172.21.15.123:0/3776459545 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28007310 con 0x7f7a38074870 2024-04-25T12:38:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.011+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a380c3b80 con 0x7f7a38074870 2024-04-25T12:38:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.012+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a38073560 con 0x7f7a38074870 2024-04-25T12:38:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.013+0000 7f7a35600640 1 -- 172.21.15.123:0/3776459545 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a2801a070 con 0x7f7a38074870 2024-04-25T12:38:24.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.014+0000 7f7a35600640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a180744b0 0x7f7a180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.014+0000 7f7a35600640 1 -- 172.21.15.123:0/3776459545 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f7a28094c20 con 0x7f7a38074870 2024-04-25T12:38:24.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.014+0000 7f7a3ce00640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a180744b0 0x7f7a180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.015+0000 7f7a3ce00640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a180744b0 0x7f7a180769c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a24034000 tx=0x7f7a2402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:24.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.019+0000 7f7a35600640 1 -- 172.21.15.123:0/3776459545 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a38073560 con 0x7f7a38074870 2024-04-25T12:38:24.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.259+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a380708c0 con 0x7f7a38074870 2024-04-25T12:38:24.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.260+0000 7f7a35600640 1 -- 172.21.15.123:0/3776459545 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6881 (secure 0 0 0) 0x7f7a380708c0 con 0x7f7a38074870 2024-04-25T12:38:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:38:24.261 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:55.199669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4270,"mds_1":4264,"mds_2":4258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4258":{"gid":4258,"name":"d","rank":2,"incarnation":13,"state":"up:active","state_seq":13,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4264":{"gid":4264,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":12,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4258,4264,4270]},"id":1}]} 2024-04-25T12:38:24.261 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 14 2024-04-25T12:38:24.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a180744b0 msgr2=0x7f7a180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a180744b0 0x7f7a180769c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a24034000 tx=0x7f7a2402e000 comp rx=0 tx=0).stop 2024-04-25T12:38:24.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a38074870 msgr2=0x7f7a38146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a38074870 0x7f7a38146f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c410 tx=0x7f7a2800c8d0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 shutdown_connections 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f7a180744b0 0x7f7a180769c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7a3815afa0 0x7f7a381f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a380c4d50 0x7f7a38147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 --2- 172.21.15.123:0/3776459545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a38074870 0x7f7a38146f20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.263+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 >> 172.21.15.123:0/3776459545 conn(0x7f7a3806c750 msgr2=0x7f7a3814c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.264+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 shutdown_connections 2024-04-25T12:38:24.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.264+0000 7f7a40200640 1 -- 172.21.15.123:0/3776459545 wait complete. 2024-04-25T12:38:24.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:38:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.551+0000 7f313ca00640 1 Processor -- start 2024-04-25T12:38:24.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.552+0000 7f313ca00640 1 -- start start 2024-04-25T12:38:24.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f313ca00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31380c9080 0x7f31381546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f313ca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313815a330 con 0x7f313806c570 2024-04-25T12:38:24.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f313ca00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3138073b10 con 0x7f31380c82b0 2024-04-25T12:38:24.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f313ca00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3138073df0 con 0x7f31380c9080 2024-04-25T12:38:24.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f3137400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31380c9080 0x7f31381546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f3137400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31380c9080 0x7f31381546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43594/0 (socket says 172.21.15.123:43594) 2024-04-25T12:38:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f3137e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f313806c570 0x7f3138164cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58734/0 (socket says 172.21.15.123:58734) 2024-04-25T12:38:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.554+0000 7f3137400640 1 -- 172.21.15.123:0/3327036511 learned_addr learned my addr 172.21.15.123:0/3327036511 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.555+0000 7f3137400640 1 -- 172.21.15.123:0/3327036511 >> v1:172.21.15.153:6789/0 conn(0x7f31380c82b0 legacy=0x7f313815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.555+0000 7f3137400640 1 -- 172.21.15.123:0/3327036511 >> v1:172.21.15.123:6789/0 conn(0x7f313806c570 legacy=0x7f3138164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.555+0000 7f3137400640 1 -- 172.21.15.123:0/3327036511 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31380740d0 con 0x7f31380c9080 2024-04-25T12:38:24.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.555+0000 7f3136a00640 1 -- 172.21.15.123:0/3327036511 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584276632 0 0) 0x7f313815a330 con 0x7f313806c570 2024-04-25T12:38:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.556+0000 7f3137400640 1 --2- 172.21.15.123:0/3327036511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31380c9080 0x7f31381546d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3128009870 tx=0x7f312802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebabfbf021a66740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.556+0000 7f3136a00640 1 -- 172.21.15.123:0/3327036511 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128037070 con 0x7f31380c9080 2024-04-25T12:38:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.556+0000 7f3136a00640 1 -- 172.21.15.123:0/3327036511 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3128032440 con 0x7f31380c9080 2024-04-25T12:38:24.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.556+0000 7f3136a00640 1 -- 172.21.15.123:0/3327036511 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128032830 con 0x7f31380c9080 2024-04-25T12:38:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.557+0000 7f313ca00640 1 -- 172.21.15.123:0/3327036511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31380c9080 msgr2=0x7f31381546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.557+0000 7f313ca00640 1 --2- 172.21.15.123:0/3327036511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31380c9080 0x7f31381546d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3128009870 tx=0x7f312802a270 comp rx=0 tx=0).stop 2024-04-25T12:38:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.557+0000 7f313ca00640 1 -- 172.21.15.123:0/3327036511 shutdown_connections 2024-04-25T12:38:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.557+0000 7f313ca00640 1 --2- 172.21.15.123:0/3327036511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31380c9080 0x7f31381546d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.557+0000 7f313ca00640 1 -- 172.21.15.123:0/3327036511 >> 172.21.15.123:0/3327036511 conn(0x7f31380cd600 msgr2=0x7f31380cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:24.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.557+0000 7f313ca00640 1 -- 172.21.15.123:0/3327036511 shutdown_connections 2024-04-25T12:38:24.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.557+0000 7f313ca00640 1 -- 172.21.15.123:0/3327036511 wait complete. 2024-04-25T12:38:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.559+0000 7f313ca00640 1 Processor -- start 2024-04-25T12:38:24.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.559+0000 7f313ca00640 1 -- start start 2024-04-25T12:38:24.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f313ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806c570 0x7f31380cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f313ca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31380c82b0 0x7f31381e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f313ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31380c9080 0x7f3138149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f313ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f313815e760 con 0x7f313806c570 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f313ca00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f313815db30 con 0x7f31380c9080 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f313ca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3138073760 con 0x7f31380c82b0 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806c570 0x7f31380cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806c570 0x7f31380cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43608/0 (socket says 172.21.15.123:43608) 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 -- 172.21.15.123:0/1155966250 learned_addr learned my addr 172.21.15.123:0/1155966250 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f312fe00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31380c82b0 0x7f31381e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137e00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31380c9080 0x7f3138149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 -- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31380c82b0 msgr2=0x7f31381e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31380c82b0 0x7f31381e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 -- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31380c9080 msgr2=0x7f3138149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31380c9080 0x7f3138149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.561+0000 7f3137400640 1 -- 172.21.15.123:0/1155966250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31380cc140 con 0x7f313806c570 2024-04-25T12:38:24.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.562+0000 7f3137400640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806c570 0x7f31380cbc00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3128009870 tx=0x7f31280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:24.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.562+0000 7f312f400640 1 -- 172.21.15.123:0/1155966250 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128037040 con 0x7f313806c570 2024-04-25T12:38:24.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.562+0000 7f312f400640 1 -- 172.21.15.123:0/1155966250 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3128042070 con 0x7f313806c570 2024-04-25T12:38:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.562+0000 7f312f400640 1 -- 172.21.15.123:0/1155966250 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128030bd0 con 0x7f313806c570 2024-04-25T12:38:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.562+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31381f3c00 con 0x7f313806c570 2024-04-25T12:38:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.562+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31381f4440 con 0x7f313806c570 2024-04-25T12:38:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.563+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31380cc310 con 0x7f313806c570 2024-04-25T12:38:24.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.565+0000 7f312f400640 1 -- 172.21.15.123:0/1155966250 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3128047460 con 0x7f313806c570 2024-04-25T12:38:24.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.566+0000 7f312f400640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31140744b0 0x7f31140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:24.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.566+0000 7f312f400640 1 -- 172.21.15.123:0/1155966250 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f31280b6360 con 0x7f313806c570 2024-04-25T12:38:24.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.566+0000 7f312fe00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31140744b0 0x7f31140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:24.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.566+0000 7f312fe00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31140744b0 0x7f31140769c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f31381eafd0 tx=0x7f3120009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:24.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.570+0000 7f312f400640 1 -- 172.21.15.123:0/1155966250 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31380cc310 con 0x7f313806c570 2024-04-25T12:38:24.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.810+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3138076da0 con 0x7f313806c570 2024-04-25T12:38:24.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.812+0000 7f312f400640 1 -- 172.21.15.123:0/1155966250 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6881 (secure 0 0 0) 0x7f312807ffe0 con 0x7f313806c570 2024-04-25T12:38:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:38:24.812 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3443728249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3443728249},{"type":"v1","addr":"172.21.15.153:6837","nonce":3443728249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/588238712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":588238712},{"type":"v1","addr":"172.21.15.123:6835","nonce":588238712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/964947243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":964947243},{"type":"v1","addr":"172.21.15.123:6837","nonce":964947243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T12:34:13.424937+0000","modified":"2024-04-25T12:34:55.199669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4270,"mds_1":4264,"mds_2":4258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4258":{"gid":4258,"name":"d","rank":2,"incarnation":13,"state":"up:active","state_seq":13,"addr":"172.21.15.153:6835/3198384231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3198384231},{"type":"v1","addr":"172.21.15.153:6835","nonce":3198384231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4264":{"gid":4264,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":12,"addr":"172.21.15.153:6839/4167745001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":4167745001},{"type":"v1","addr":"172.21.15.153:6839","nonce":4167745001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4270":{"gid":4270,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6834/1720190095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1720190095},{"type":"v1","addr":"172.21.15.123:6834","nonce":1720190095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4258,4264,4270]},"id":1}]} 2024-04-25T12:38:24.812 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 14 2024-04-25T12:38:24.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31140744b0 msgr2=0x7f31140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31140744b0 0x7f31140769c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f31381eafd0 tx=0x7f3120009040 comp rx=0 tx=0).stop 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806c570 msgr2=0x7f31380cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806c570 0x7f31380cbc00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3128009870 tx=0x7f31280046c0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 shutdown_connections 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f31140744b0 0x7f31140769c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31380c9080 0x7f3138149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f31380c82b0 0x7f31381e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 --2- 172.21.15.123:0/1155966250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f313806c570 0x7f31380cbc00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 >> 172.21.15.123:0/1155966250 conn(0x7f31380cd600 msgr2=0x7f313806c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.814+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 shutdown_connections 2024-04-25T12:38:24.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:24.815+0000 7f313ca00640 1 -- 172.21.15.123:0/1155966250 wait complete. 2024-04-25T12:38:24.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:38:25.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.107+0000 7f12a5400640 1 Processor -- start 2024-04-25T12:38:25.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.107+0000 7f12a5400640 1 -- start start 2024-04-25T12:38:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.109+0000 7f12a5400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12a00c9080 0x7f12a01546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f12a5400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f12a5400640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a015a330 con 0x7f12a015e590 2024-04-25T12:38:25.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f12a5400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f12a0073b10 con 0x7f12a00c9080 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f12a5400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f12a0073df0 con 0x7f12a00c82b0 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12a00c9080 0x7f12a01546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12a00c9080 0x7f12a01546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43618/0 (socket says 172.21.15.123:43618) 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129fe00640 1 -- 172.21.15.123:0/95429494 learned_addr learned my addr 172.21.15.123:0/95429494 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f1297400640 1 --2- 172.21.15.123:0/95429494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129f400640 1 -- 172.21.15.123:0/95429494 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 881747509 0 0) 0x7f12a015a330 con 0x7f12a015e590 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129fe00640 1 -- 172.21.15.123:0/95429494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 msgr2=0x7f12a015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129fe00640 1 --2- 172.21.15.123:0/95429494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129fe00640 1 -- 172.21.15.123:0/95429494 >> v1:172.21.15.153:6789/0 conn(0x7f12a015e590 legacy=0x7f12a006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.110+0000 7f129fe00640 1 -- 172.21.15.123:0/95429494 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a00740d0 con 0x7f12a00c9080 2024-04-25T12:38:25.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.112+0000 7f129fe00640 1 --2- 172.21.15.123:0/95429494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a00c9080 0x7f12a01546d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f12a015a500 tx=0x7f129002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bca16545205e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.112+0000 7f129f400640 1 -- 172.21.15.123:0/95429494 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290037070 con 0x7f12a00c9080 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.112+0000 7f129f400640 1 -- 172.21.15.123:0/95429494 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1290032440 con 0x7f12a00c9080 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.112+0000 7f129f400640 1 -- 172.21.15.123:0/95429494 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290032830 con 0x7f12a00c9080 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 -- 172.21.15.123:0/95429494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a00c9080 msgr2=0x7f12a01546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 --2- 172.21.15.123:0/95429494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a00c9080 0x7f12a01546d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f12a015a500 tx=0x7f129002a250 comp rx=0 tx=0).stop 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 -- 172.21.15.123:0/95429494 shutdown_connections 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 --2- 172.21.15.123:0/95429494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 --2- 172.21.15.123:0/95429494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a00c9080 0x7f12a01546d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 -- 172.21.15.123:0/95429494 >> 172.21.15.123:0/95429494 conn(0x7f12a00cd600 msgr2=0x7f12a00cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 -- 172.21.15.123:0/95429494 shutdown_connections 2024-04-25T12:38:25.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.113+0000 7f12a5400640 1 -- 172.21.15.123:0/95429494 wait complete. 2024-04-25T12:38:25.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.115+0000 7f12a5400640 1 Processor -- start 2024-04-25T12:38:25.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.115+0000 7f12a5400640 1 -- start start 2024-04-25T12:38:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a5400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a00cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a00c9080 0x7f12a00c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a015e590 0x7f12a006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a5400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0073760 con 0x7f12a015e590 2024-04-25T12:38:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a5400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12a015db30 con 0x7f12a00c9080 2024-04-25T12:38:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a5400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f12a014b8c0 con 0x7f12a00c82b0 2024-04-25T12:38:25.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a015e590 0x7f12a006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f129fe00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a00cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a015e590 0x7f12a006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43622/0 (socket says 172.21.15.123:43622) 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 -- 172.21.15.123:0/10148524 learned_addr learned my addr 172.21.15.123:0/10148524 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f1297400640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a00c9080 0x7f12a00c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 -- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 msgr2=0x7f12a00cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a00cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 -- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a00c9080 msgr2=0x7f12a00c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a00c9080 0x7f12a00c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.117+0000 7f12a4a00640 1 -- 172.21.15.123:0/10148524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a00cc5f0 con 0x7f12a015e590 2024-04-25T12:38:25.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.118+0000 7f12a4a00640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a015e590 0x7f12a006c570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f129800c7b0 tx=0x7f129800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:25.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.118+0000 7f129cc00640 1 -- 172.21.15.123:0/10148524 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298003b70 con 0x7f12a015e590 2024-04-25T12:38:25.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.118+0000 7f129cc00640 1 -- 172.21.15.123:0/10148524 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1298003d30 con 0x7f12a015e590 2024-04-25T12:38:25.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.118+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a015d0f0 con 0x7f12a015e590 2024-04-25T12:38:25.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.118+0000 7f129cc00640 1 -- 172.21.15.123:0/10148524 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12980052d0 con 0x7f12a015e590 2024-04-25T12:38:25.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.118+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a015d6b0 con 0x7f12a015e590 2024-04-25T12:38:25.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.119+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a0076da0 con 0x7f12a015e590 2024-04-25T12:38:25.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.120+0000 7f129cc00640 1 -- 172.21.15.123:0/10148524 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1298005470 con 0x7f12a015e590 2024-04-25T12:38:25.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.121+0000 7f129cc00640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f127c0744b0 0x7f127c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.121+0000 7f129cc00640 1 -- 172.21.15.123:0/10148524 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f12980954c0 con 0x7f12a015e590 2024-04-25T12:38:25.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.122+0000 7f129fe00640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f127c0744b0 0x7f127c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.123+0000 7f129fe00640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f127c0744b0 0x7f127c0769c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1290034000 tx=0x7f129002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:25.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.126+0000 7f129cc00640 1 -- 172.21.15.123:0/10148524 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f129805f0c0 con 0x7f12a015e590 2024-04-25T12:38:25.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.324+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12a00c8e60 con 0x7f12a015e590 2024-04-25T12:38:25.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.326+0000 7f129cc00640 1 -- 172.21.15.123:0/10148524 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+19058 (secure 0 0 0) 0x7f12980633c0 con 0x7f12a015e590 2024-04-25T12:38:25.326 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T12:38:25.326 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":93,"fsid":"2239057a-dc25-4ee3-8341-8c80e403f296","created":"2024-04-25T12:33:54.601604+0000","modified":"2024-04-25T12:37:21.396436+0000","last_up_change":"2024-04-25T12:35:59.575545+0000","last_in_change":"2024-04-25T12:37:00.163243+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:34:06.827674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:34:11.030660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:34:12.395253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T12:34:15.975923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2d9ee216-abfa-45db-a0ac-ef834981be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6817","nonce":1405705428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6819","nonce":1405705428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6823","nonce":1405705428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1405705428},{"type":"v1","addr":"172.21.15.123:6821","nonce":1405705428}]},"public_addr":"172.21.15.123:6817/1405705428","cluster_addr":"172.21.15.123:6819/1405705428","heartbeat_back_addr":"172.21.15.123:6823/1405705428","heartbeat_front_addr":"172.21.15.123:6821/1405705428","state":["exists","up"]},{"osd":1,"uuid":"a60a3a6f-376a-4fb2-baf1-e13fc615f6e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6801","nonce":1234958824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6803","nonce":1234958824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6807","nonce":1234958824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1234958824},{"type":"v1","addr":"172.21.15.123:6805","nonce":1234958824}]},"public_addr":"172.21.15.123:6801/1234958824","cluster_addr":"172.21.15.123:6803/1234958824","heartbeat_back_addr":"172.21.15.123:6807/1234958824","heartbeat_front_addr":"172.21.15.123:6805/1234958824","state":["exists","up"]},{"osd":2,"uuid":"cfb7cbbd-82cd-46fe-b89b-15775df1fe93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6809","nonce":2222173849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6811","nonce":2222173849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6815","nonce":2222173849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2222173849},{"type":"v1","addr":"172.21.15.123:6813","nonce":2222173849}]},"public_addr":"172.21.15.123:6809/2222173849","cluster_addr":"172.21.15.123:6811/2222173849","heartbeat_back_addr":"172.21.15.123:6815/2222173849","heartbeat_front_addr":"172.21.15.123:6813/2222173849","state":["exists","up"]},{"osd":3,"uuid":"9e8ccd97-ab68-4f03-bbb3-5820eccf4568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6825","nonce":1491071423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6827","nonce":1491071423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6831","nonce":1491071423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1491071423},{"type":"v1","addr":"172.21.15.123:6829","nonce":1491071423}]},"public_addr":"172.21.15.123:6825/1491071423","cluster_addr":"172.21.15.123:6827/1491071423","heartbeat_back_addr":"172.21.15.123:6831/1491071423","heartbeat_front_addr":"172.21.15.123:6829/1491071423","state":["exists","up"]},{"osd":4,"uuid":"7d5bfd6e-173a-413d-abdb-df6d82c941d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6803","nonce":3135982807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6805","nonce":3135982807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6809","nonce":3135982807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3135982807},{"type":"v1","addr":"172.21.15.153:6807","nonce":3135982807}]},"public_addr":"172.21.15.153:6803/3135982807","cluster_addr":"172.21.15.153:6805/3135982807","heartbeat_back_addr":"172.21.15.153:6809/3135982807","heartbeat_front_addr":"172.21.15.153:6807/3135982807","state":["exists","up"]},{"osd":5,"uuid":"9141fa91-c547-4711-910a-9ef123017fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":39,"up_from":47,"up_thru":76,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6827","nonce":4270711740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6829","nonce":4270711740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6833","nonce":4270711740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":4270711740},{"type":"v1","addr":"172.21.15.153:6831","nonce":4270711740}]},"public_addr":"172.21.15.153:6827/4270711740","cluster_addr":"172.21.15.153:6829/4270711740","heartbeat_back_addr":"172.21.15.153:6833/4270711740","heartbeat_front_addr":"172.21.15.153:6831/4270711740","state":["exists","up"]},{"osd":6,"uuid":"66fbc935-086e-4b03-aa03-96e0bc8b1d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6811","nonce":3626282289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6813","nonce":3626282289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6817","nonce":3626282289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3626282289},{"type":"v1","addr":"172.21.15.153:6815","nonce":3626282289}]},"public_addr":"172.21.15.153:6811/3626282289","cluster_addr":"172.21.15.153:6813/3626282289","heartbeat_back_addr":"172.21.15.153:6817/3626282289","heartbeat_front_addr":"172.21.15.153:6815/3626282289","state":["exists","up"]},{"osd":7,"uuid":"de59a619-b84a-4623-ac89-447178815d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6819","nonce":3086604284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6821","nonce":3086604284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6825","nonce":3086604284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3086604284},{"type":"v1","addr":"172.21.15.153:6823","nonce":3086604284}]},"public_addr":"172.21.15.153:6819/3086604284","cluster_addr":"172.21.15.153:6821/3086604284","heartbeat_back_addr":"172.21.15.153:6825/3086604284","heartbeat_front_addr":"172.21.15.153:6823/3086604284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.805055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.746266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.706883+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.826277+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.402228+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.536018+0000","dead_epoch":40},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.375016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:34:03.454835+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[3,1]},{"pgid":"4.15","osds":[1,0,7,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":5,"to":4}]},{"pgid":"2.c","mappings":[{"from":0,"to":2}]},{"pgid":"3.3","mappings":[{"from":0,"to":7}]},{"pgid":"3.c","mappings":[{"from":0,"to":1}]},{"pgid":"3.13","mappings":[{"from":0,"to":1}]},{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"3.17","mappings":[{"from":3,"to":7}]},{"pgid":"3.26","mappings":[{"from":3,"to":1}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:38:25.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f127c0744b0 msgr2=0x7f127c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f127c0744b0 0x7f127c0769c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1290034000 tx=0x7f129002e000 comp rx=0 tx=0).stop 2024-04-25T12:38:25.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a015e590 msgr2=0x7f12a006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a015e590 0x7f12a006c570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f129800c7b0 tx=0x7f129800cc70 comp rx=0 tx=0).stop 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 shutdown_connections 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f127c0744b0 0x7f127c0769c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12a015e590 0x7f12a006c570 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a00c9080 0x7f12a00c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.328+0000 7f12a5400640 1 --2- 172.21.15.123:0/10148524 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12a00c82b0 0x7f12a00cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.329+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 >> 172.21.15.123:0/10148524 conn(0x7f12a00cd600 msgr2=0x7f12a00c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.329+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 shutdown_connections 2024-04-25T12:38:25.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.329+0000 7f12a5400640 1 -- 172.21.15.123:0/10148524 wait complete. 2024-04-25T12:38:25.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T12:38:25.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.625+0000 7f6eeca00640 1 Processor -- start 2024-04-25T12:38:25.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.626+0000 7f6eeca00640 1 -- start start 2024-04-25T12:38:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6eeca00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6ee81546d0 0x7f6ee815e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6eeca00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ee815e590 0x7f6ee806c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6eeca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee815a330 con 0x7f6ee80c9080 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6eeca00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee8073b10 con 0x7f6ee815e590 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6eeca00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee8073df0 con 0x7f6ee81546d0 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6ee7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ee815e590 0x7f6ee806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6ee7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ee815e590 0x7f6ee806c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43624/0 (socket says 172.21.15.123:43624) 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6ee7e00640 1 -- 172.21.15.123:0/2986987862 learned_addr learned my addr 172.21.15.123:0/2986987862 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6edea00640 1 --2- 172.21.15.123:0/2986987862 >> v2:172.21.15.153:3300/0 conn(0x7f6ee81546d0 0x7f6ee815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.628+0000 7f6ee7400640 1 --1- 172.21.15.123:0/2986987862 >> v1:172.21.15.123:6789/0 conn(0x7f6ee80c9080 0x7f6ee80c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58746/0 (socket says 172.21.15.123:58746) 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.629+0000 7f6ee7e00640 1 -- 172.21.15.123:0/2986987862 >> v2:172.21.15.153:3300/0 conn(0x7f6ee81546d0 msgr2=0x7f6ee815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.629+0000 7f6ee7e00640 1 --2- 172.21.15.123:0/2986987862 >> v2:172.21.15.153:3300/0 conn(0x7f6ee81546d0 0x7f6ee815e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.629+0000 7f6ee7e00640 1 -- 172.21.15.123:0/2986987862 >> v1:172.21.15.123:6789/0 conn(0x7f6ee80c9080 legacy=0x7f6ee80c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.629+0000 7f6ee7e00640 1 -- 172.21.15.123:0/2986987862 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee80740d0 con 0x7f6ee815e590 2024-04-25T12:38:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.630+0000 7f6ee7e00640 1 --2- 172.21.15.123:0/2986987862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee815e590 0x7f6ee806c570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8072820 tx=0x7f6ee002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04b50847ffcdc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.630+0000 7f6ee6a00640 1 -- 172.21.15.123:0/2986987862 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0036070 con 0x7f6ee815e590 2024-04-25T12:38:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.630+0000 7f6ee6a00640 1 -- 172.21.15.123:0/2986987862 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6ee0031600 con 0x7f6ee815e590 2024-04-25T12:38:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.630+0000 7f6ee6a00640 1 -- 172.21.15.123:0/2986987862 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0031a10 con 0x7f6ee815e590 2024-04-25T12:38:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 -- 172.21.15.123:0/2986987862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee815e590 msgr2=0x7f6ee806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 --2- 172.21.15.123:0/2986987862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee815e590 0x7f6ee806c570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8072820 tx=0x7f6ee002a7c0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 -- 172.21.15.123:0/2986987862 shutdown_connections 2024-04-25T12:38:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 --2- 172.21.15.123:0/2986987862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee815e590 0x7f6ee806c570 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 --2- 172.21.15.123:0/2986987862 >> v2:172.21.15.153:3300/0 conn(0x7f6ee81546d0 0x7f6ee815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 -- 172.21.15.123:0/2986987862 >> 172.21.15.123:0/2986987862 conn(0x7f6ee80cd600 msgr2=0x7f6ee80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:25.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 -- 172.21.15.123:0/2986987862 shutdown_connections 2024-04-25T12:38:25.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.631+0000 7f6eeca00640 1 -- 172.21.15.123:0/2986987862 wait complete. 2024-04-25T12:38:25.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.633+0000 7f6eeca00640 1 Processor -- start 2024-04-25T12:38:25.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.634+0000 7f6eeca00640 1 -- start start 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6eeca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80c9080 0x7f6ee80c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6eeca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ee81546d0 0x7f6ee8151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6eeca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ee815e590 0x7f6ee80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6eeca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee8073760 con 0x7f6ee80c9080 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6eeca00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee815db30 con 0x7f6ee81546d0 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6eeca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee8152310 con 0x7f6ee815e590 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6ee7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80c9080 0x7f6ee80c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6ee7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80c9080 0x7f6ee80c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43632/0 (socket says 172.21.15.123:43632) 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.635+0000 7f6ee7400640 1 -- 172.21.15.123:0/3419540176 learned_addr learned my addr 172.21.15.123:0/3419540176 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6edea00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ee81546d0 0x7f6ee8151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6ee7e00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ee815e590 0x7f6ee80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6ee7400640 1 -- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ee815e590 msgr2=0x7f6ee80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6ee7400640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ee815e590 0x7f6ee80cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6ee7400640 1 -- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ee81546d0 msgr2=0x7f6ee8151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6ee7400640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ee81546d0 0x7f6ee8151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6ee7400640 1 -- 172.21.15.123:0/3419540176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee8151e40 con 0x7f6ee80c9080 2024-04-25T12:38:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6ee7400640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80c9080 0x7f6ee80c5ad0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800c3f0 tx=0x7f6ed800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:25.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.636+0000 7f6edfe00640 1 -- 172.21.15.123:0/3419540176 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80074e0 con 0x7f6ee80c9080 2024-04-25T12:38:25.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.637+0000 7f6edfe00640 1 -- 172.21.15.123:0/3419540176 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6ed8007680 con 0x7f6ee80c9080 2024-04-25T12:38:25.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.637+0000 7f6edfe00640 1 -- 172.21.15.123:0/3419540176 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8004a20 con 0x7f6ee80c9080 2024-04-25T12:38:25.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.637+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee8153d70 con 0x7f6ee80c9080 2024-04-25T12:38:25.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.637+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee81531d0 con 0x7f6ee80c9080 2024-04-25T12:38:25.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.638+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee8076da0 con 0x7f6ee80c9080 2024-04-25T12:38:25.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.639+0000 7f6edfe00640 1 -- 172.21.15.123:0/3419540176 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ed801a020 con 0x7f6ee80c9080 2024-04-25T12:38:25.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.640+0000 7f6edfe00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6ec40744b0 0x7f6ec40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:25.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.640+0000 7f6edfe00640 1 -- 172.21.15.123:0/3419540176 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6891+0+0 (secure 0 0 0) 0x7f6ed8094d20 con 0x7f6ee80c9080 2024-04-25T12:38:25.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.640+0000 7f6edea00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6ec40744b0 0x7f6ec40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:25.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.641+0000 7f6edea00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6ec40744b0 0x7f6ec40769c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80c3e10 tx=0x7f6ed0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:25.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.645+0000 7f6edfe00640 1 -- 172.21.15.123:0/3419540176 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ed8099070 con 0x7f6ee80c9080 2024-04-25T12:38:25.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.883+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6ee80c56f0 con 0x7f6ee80c9080 2024-04-25T12:38:25.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.888+0000 7f6edfe00640 1 -- 172.21.15.123:0/3419540176 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7f6ed8094020 con 0x7f6ee80c9080 2024-04-25T12:38:25.888 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T12:38:25.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6ec40744b0 msgr2=0x7f6ec40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6ec40744b0 0x7f6ec40769c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80c3e10 tx=0x7f6ed0009190 comp rx=0 tx=0).stop 2024-04-25T12:38:25.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80c9080 msgr2=0x7f6ee80c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80c9080 0x7f6ee80c5ad0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800c3f0 tx=0x7f6ed800c8b0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 shutdown_connections 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f6ec40744b0 0x7f6ec40769c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ee815e590 0x7f6ee80cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ee81546d0 0x7f6ee8151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 --2- 172.21.15.123:0/3419540176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80c9080 0x7f6ee80c5ad0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 >> 172.21.15.123:0/3419540176 conn(0x7f6ee80cd600 msgr2=0x7f6ee8072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 shutdown_connections 2024-04-25T12:38:25.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:25.891+0000 7f6eeca00640 1 -- 172.21.15.123:0/3419540176 wait complete. 2024-04-25T12:38:25.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T12:38:25.955 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2024-04-25T12:38:25.956 INFO:tasks.ceph.mds.f.smithi153.stdout:starting mds.f at 2024-04-25T12:38:26.084 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2024-04-25T12:38:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.181+0000 7f5525c00640 1 Processor -- start 2024-04-25T12:38:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.181+0000 7f5525c00640 1 -- start start 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.183+0000 7f5525c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5520074840 0x7f552015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.183+0000 7f5525c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f552006c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.183+0000 7f5525c00640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55200c3f70 con 0x7f552006ccf0 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.183+0000 7f5525c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5520073b00 con 0x7f5520074840 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.183+0000 7f5525c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5520073da0 con 0x7f55200c2720 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.183+0000 7f551fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5520074840 0x7f552015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.184+0000 7f551fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5520074840 0x7f552015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43658/0 (socket says 172.21.15.123:43658) 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.184+0000 7f551fe00640 1 -- 172.21.15.123:0/1955988824 learned_addr learned my addr 172.21.15.123:0/1955988824 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.184+0000 7f551f400640 1 --2- 172.21.15.123:0/1955988824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f552006c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.184+0000 7f551fe00640 1 -- 172.21.15.123:0/1955988824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 msgr2=0x7f552006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.184+0000 7f551fe00640 1 --2- 172.21.15.123:0/1955988824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f552006c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.184+0000 7f551fe00640 1 -- 172.21.15.123:0/1955988824 >> v1:172.21.15.153:6789/0 conn(0x7f552006ccf0 legacy=0x7f55200c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.184+0000 7f551fe00640 1 -- 172.21.15.123:0/1955988824 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5520161350 con 0x7f5520074840 2024-04-25T12:38:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.185+0000 7f551fe00640 1 --2- 172.21.15.123:0/1955988824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 0x7f552015de60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5510009870 tx=0x7f551002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7605187638ff770f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.185+0000 7f551ea00640 1 -- 172.21.15.123:0/1955988824 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510037070 con 0x7f5520074840 2024-04-25T12:38:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.185+0000 7f551ea00640 1 -- 172.21.15.123:0/1955988824 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5510032440 con 0x7f5520074840 2024-04-25T12:38:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.185+0000 7f551ea00640 1 -- 172.21.15.123:0/1955988824 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510032830 con 0x7f5520074840 2024-04-25T12:38:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 -- 172.21.15.123:0/1955988824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 msgr2=0x7f552015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 --2- 172.21.15.123:0/1955988824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 0x7f552015de60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5510009870 tx=0x7f551002a250 comp rx=0 tx=0).stop 2024-04-25T12:38:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 -- 172.21.15.123:0/1955988824 shutdown_connections 2024-04-25T12:38:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 --2- 172.21.15.123:0/1955988824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f552006c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 --2- 172.21.15.123:0/1955988824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 0x7f552015de60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 -- 172.21.15.123:0/1955988824 >> 172.21.15.123:0/1955988824 conn(0x7f5520070b70 msgr2=0x7f5520070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 -- 172.21.15.123:0/1955988824 shutdown_connections 2024-04-25T12:38:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.186+0000 7f5525c00640 1 -- 172.21.15.123:0/1955988824 wait complete. 2024-04-25T12:38:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.188+0000 7f5525c00640 1 Processor -- start 2024-04-25T12:38:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.189+0000 7f5525c00640 1 -- start start 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f5525c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552006ccf0 0x7f55200c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f5525c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 0x7f552015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f5525c00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f55201f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f5525c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55200c44c0 con 0x7f5520074840 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f5525c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55200c41e0 con 0x7f552006ccf0 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f5525c00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f55200c3820 con 0x7f55200c2720 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f551fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552006ccf0 0x7f55200c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f551f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 0x7f552015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f551fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552006ccf0 0x7f55200c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.123:53236/0 (socket says 172.21.15.123:53236) 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f551fe00640 1 -- 172.21.15.123:0/386652546 learned_addr learned my addr 172.21.15.123:0/386652546 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.190+0000 7f5525200640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f55201f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f551fe00640 1 -- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 msgr2=0x7f55201f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f551fe00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f55201f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f551fe00640 1 -- 172.21.15.123:0/386652546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 msgr2=0x7f552015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f551fe00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 0x7f552015d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f551fe00640 1 -- 172.21.15.123:0/386652546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55201f7720 con 0x7f552006ccf0 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f551fe00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552006ccf0 0x7f55200c65f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f551002a290 tx=0x7f55100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f5517e00640 1 -- 172.21.15.123:0/386652546 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510037040 con 0x7f552006ccf0 2024-04-25T12:38:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f5517e00640 1 -- 172.21.15.123:0/386652546 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5510042070 con 0x7f552006ccf0 2024-04-25T12:38:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f5517e00640 1 -- 172.21.15.123:0/386652546 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510030900 con 0x7f552006ccf0 2024-04-25T12:38:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55200c5110 con 0x7f552006ccf0 2024-04-25T12:38:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.191+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55201f69e0 con 0x7f552006ccf0 2024-04-25T12:38:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.192+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520076da0 con 0x7f552006ccf0 2024-04-25T12:38:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.194+0000 7f5517e00640 1 -- 172.21.15.123:0/386652546 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5510030aa0 con 0x7f552006ccf0 2024-04-25T12:38:26.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.195+0000 7f5517e00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55000744b0 0x7f55000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:26.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.195+0000 7f5517e00640 1 -- 172.21.15.123:0/386652546 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7149+0+0 (secure 0 0 0) 0x7f55100b5ff0 con 0x7f552006ccf0 2024-04-25T12:38:26.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.195+0000 7f551f400640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55000744b0 0x7f55000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:26.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.195+0000 7f551f400640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55000744b0 0x7f55000769c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5520152f40 tx=0x7f550c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.199+0000 7f5517e00640 1 -- 172.21.15.123:0/386652546 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f551007fba0 con 0x7f552006ccf0 2024-04-25T12:38:26.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.423+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f55200b35a0 con 0x7f552006ccf0 2024-04-25T12:38:26.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.961+0000 7f5517e00640 1 -- 172.21.15.123:0/386652546 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f5510083ea0 con 0x7f552006ccf0 2024-04-25T12:38:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55000744b0 msgr2=0x7f55000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55000744b0 0x7f55000769c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5520152f40 tx=0x7f550c009190 comp rx=0 tx=0).stop 2024-04-25T12:38:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552006ccf0 msgr2=0x7f55200c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552006ccf0 0x7f55200c65f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f551002a290 tx=0x7f55100046c0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 shutdown_connections 2024-04-25T12:38:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f55000744b0 0x7f55000769c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55200c2720 0x7f55201f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5520074840 0x7f552015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 --2- 172.21.15.123:0/386652546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552006ccf0 0x7f55200c65f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 >> 172.21.15.123:0/386652546 conn(0x7f5520070b70 msgr2=0x7f552015e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.965+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 shutdown_connections 2024-04-25T12:38:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:26.966+0000 7f5525c00640 1 -- 172.21.15.123:0/386652546 wait complete. 2024-04-25T12:38:26.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T12:38:27.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.263+0000 7ff97ca00640 1 Processor -- start 2024-04-25T12:38:27.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.264+0000 7ff97ca00640 1 -- start start 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff97ca00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff97815c390 0x7ff978156540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff97ca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97806d680 0x7ff978070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff97ca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97815c8f0 con 0x7ff978071350 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff97ca00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff97806e130 con 0x7ff97815c390 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff97ca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff97815cbd0 con 0x7ff97806d680 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff977e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff978071350 0x7ff97806e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58770/0 (socket says 172.21.15.123:58770) 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff977e00640 1 -- 172.21.15.123:0/4110383397 learned_addr learned my addr 172.21.15.123:0/4110383397 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff96ea00640 1 --2- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97806d680 0x7ff978070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:27.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.266+0000 7ff977400640 1 --2- 172.21.15.123:0/4110383397 >> v2:172.21.15.153:3300/0 conn(0x7ff97815c390 0x7ff978156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:27.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.267+0000 7ff976a00640 1 -- 172.21.15.123:0/4110383397 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3732373025 0 0) 0x7ff97815c8f0 con 0x7ff978071350 2024-04-25T12:38:27.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.267+0000 7ff977400640 1 -- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97806d680 msgr2=0x7ff978070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:27.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.267+0000 7ff977400640 1 --2- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97806d680 0x7ff978070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.267+0000 7ff977400640 1 -- 172.21.15.123:0/4110383397 >> v1:172.21.15.123:6789/0 conn(0x7ff978071350 legacy=0x7ff97806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:27.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.267+0000 7ff977400640 1 -- 172.21.15.123:0/4110383397 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97815ceb0 con 0x7ff97815c390 2024-04-25T12:38:27.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.267+0000 7ff96ea00640 1 --2- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97806d680 0x7ff978070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:38:27.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.268+0000 7ff977400640 1 --2- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff978156540 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff968009d90 tx=0x7ff96802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c100b6e2a7ce1b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:27.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.268+0000 7ff976a00640 1 -- 172.21.15.123:0/4110383397 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968007740 con 0x7ff97815c390 2024-04-25T12:38:27.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.268+0000 7ff976a00640 1 -- 172.21.15.123:0/4110383397 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff9680078e0 con 0x7ff97815c390 2024-04-25T12:38:27.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.268+0000 7ff976a00640 1 -- 172.21.15.123:0/4110383397 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968007cd0 con 0x7ff97815c390 2024-04-25T12:38:27.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 -- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 msgr2=0x7ff978156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:27.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 --2- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff978156540 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff968009d90 tx=0x7ff96802a660 comp rx=0 tx=0).stop 2024-04-25T12:38:27.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 -- 172.21.15.123:0/4110383397 shutdown_connections 2024-04-25T12:38:27.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 --2- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97806d680 0x7ff978070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 --2- 172.21.15.123:0/4110383397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff978156540 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 -- 172.21.15.123:0/4110383397 >> 172.21.15.123:0/4110383397 conn(0x7ff978073390 msgr2=0x7ff9780cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:27.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 -- 172.21.15.123:0/4110383397 shutdown_connections 2024-04-25T12:38:27.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.269+0000 7ff97ca00640 1 -- 172.21.15.123:0/4110383397 wait complete. 2024-04-25T12:38:27.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.271+0000 7ff97ca00640 1 Processor -- start 2024-04-25T12:38:27.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.271+0000 7ff97ca00640 1 -- start start 2024-04-25T12:38:27.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff97ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff97806d680 0x7ff978151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:27.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff97ca00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff978071350 0x7ff9780b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:27.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff97ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff9780cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:27.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff97ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff978156a80 con 0x7ff97806d680 2024-04-25T12:38:27.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff97ca00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff97806e380 con 0x7ff97815c390 2024-04-25T12:38:27.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff97ca00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff978072310 con 0x7ff978071350 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff977400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff97806d680 0x7ff978151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff977e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff9780cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff977400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff97806d680 0x7ff978151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43678/0 (socket says 172.21.15.123:43678) 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff978071350 0x7ff9780b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.273+0000 7ff977400640 1 -- 172.21.15.123:0/3797790924 learned_addr learned my addr 172.21.15.123:0/3797790924 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff977e00640 1 -- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff978071350 msgr2=0x7ff9780b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff977e00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff978071350 0x7ff9780b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff977e00640 1 -- 172.21.15.123:0/3797790924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff97806d680 msgr2=0x7ff978151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff977e00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff97806d680 0x7ff978151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff977e00640 1 -- 172.21.15.123:0/3797790924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9780b5410 con 0x7ff97815c390 2024-04-25T12:38:27.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff977e00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff9780cae90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c3e0 tx=0x7ff97000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:27.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff96fe00640 1 -- 172.21.15.123:0/3797790924 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700074e0 con 0x7ff97815c390 2024-04-25T12:38:27.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff96fe00640 1 -- 172.21.15.123:0/3797790924 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff9700047e0 con 0x7ff97815c390 2024-04-25T12:38:27.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9781506c0 con 0x7ff97815c390 2024-04-25T12:38:27.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff96fe00640 1 -- 172.21.15.123:0/3797790924 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970004c30 con 0x7ff97815c390 2024-04-25T12:38:27.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.274+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9780b46d0 con 0x7ff97815c390 2024-04-25T12:38:27.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.275+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff978076da0 con 0x7ff97815c390 2024-04-25T12:38:27.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.277+0000 7ff96fe00640 1 -- 172.21.15.123:0/3797790924 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff970011070 con 0x7ff97815c390 2024-04-25T12:38:27.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.278+0000 7ff96fe00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9540744b0 0x7ff9540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:27.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.278+0000 7ff96fe00640 1 -- 172.21.15.123:0/3797790924 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7149+0+0 (secure 0 0 0) 0x7ff970095210 con 0x7ff97815c390 2024-04-25T12:38:27.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.278+0000 7ff977400640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9540744b0 0x7ff9540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:27.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.279+0000 7ff977400640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9540744b0 0x7ff9540769c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff968000f80 tx=0x7ff96802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:27.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.283+0000 7ff96fe00640 1 -- 172.21.15.123:0/3797790924 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff97005ed10 con 0x7ff97815c390 2024-04-25T12:38:27.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.483+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff978150410 con 0x7ff97815c390 2024-04-25T12:38:27.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.988+0000 7ff96fe00640 1 -- 172.21.15.123:0/3797790924 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7ff970063010 con 0x7ff97815c390 2024-04-25T12:38:27.988 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.991+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9540744b0 msgr2=0x7ff9540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.991+0000 7ff97ca00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9540744b0 0x7ff9540769c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff968000f80 tx=0x7ff96802f000 comp rx=0 tx=0).stop 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.991+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 msgr2=0x7ff9780cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.991+0000 7ff97ca00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff9780cae90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c3e0 tx=0x7ff97000c8a0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 shutdown_connections 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7ff9540744b0 0x7ff9540769c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97815c390 0x7ff9780cae90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff978071350 0x7ff9780b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 --2- 172.21.15.123:0/3797790924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff97806d680 0x7ff978151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:27.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 >> 172.21.15.123:0/3797790924 conn(0x7ff978073390 msgr2=0x7ff97814f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:27.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 shutdown_connections 2024-04-25T12:38:27.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:27.992+0000 7ff97ca00640 1 -- 172.21.15.123:0/3797790924 wait complete. 2024-04-25T12:38:28.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T12:38:28.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.301+0000 7f89fa400640 1 Processor -- start 2024-04-25T12:38:28.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.301+0000 7f89fa400640 1 -- start start 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89fa400640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89f4153680 0x7f89f40c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89fa400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89f40c2870 0x7f89f4074170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89fa400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f40c83f0 con 0x7f89f40746b0 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89fa400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f89f4153a90 con 0x7f89f40c2870 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89fa400640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f89f40c9c70 con 0x7f89f4153680 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89f3e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f89f40746b0 0x7f89f4159830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58780/0 (socket says 172.21.15.123:58780) 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89f3e00640 1 -- 172.21.15.123:0/1939186391 learned_addr learned my addr 172.21.15.123:0/1939186391 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89f2a00640 1 --2- 172.21.15.123:0/1939186391 >> v2:172.21.15.123:3300/0 conn(0x7f89f40c2870 0x7f89f4074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:28.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.303+0000 7f89f3400640 1 --2- 172.21.15.123:0/1939186391 >> v2:172.21.15.153:3300/0 conn(0x7f89f4153680 0x7f89f40c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:28.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.304+0000 7f89f2000640 1 -- 172.21.15.123:0/1939186391 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970981367 0 0) 0x7f89f40c83f0 con 0x7f89f40746b0 2024-04-25T12:38:28.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.304+0000 7f89f2000640 1 -- 172.21.15.123:0/1939186391 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89d8005730 con 0x7f89f40746b0 2024-04-25T12:38:28.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.304+0000 7f89f3400640 1 -- 172.21.15.123:0/1939186391 >> v2:172.21.15.123:3300/0 conn(0x7f89f40c2870 msgr2=0x7f89f4074170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:28.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.304+0000 7f89f3400640 1 --2- 172.21.15.123:0/1939186391 >> v2:172.21.15.123:3300/0 conn(0x7f89f40c2870 0x7f89f4074170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.304+0000 7f89f3400640 1 -- 172.21.15.123:0/1939186391 >> v1:172.21.15.123:6789/0 conn(0x7f89f40746b0 legacy=0x7f89f4159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:28.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.304+0000 7f89f3400640 1 -- 172.21.15.123:0/1939186391 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f40c8690 con 0x7f89f4153680 2024-04-25T12:38:28.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.304+0000 7f89f2a00640 1 --2- 172.21.15.123:0/1939186391 >> v2:172.21.15.123:3300/0 conn(0x7f89f40c2870 0x7f89f4074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:38:28.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.305+0000 7f89f3400640 1 --2- 172.21.15.123:0/1939186391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f4153680 0x7f89f40c3be0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f89f40cb380 tx=0x7f89e402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89504c97c86a02af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:28.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.306+0000 7f89f2000640 1 -- 172.21.15.123:0/1939186391 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4037070 con 0x7f89f4153680 2024-04-25T12:38:28.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.306+0000 7f89f2000640 1 -- 172.21.15.123:0/1939186391 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f89e4032440 con 0x7f89f4153680 2024-04-25T12:38:28.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.306+0000 7f89f2000640 1 -- 172.21.15.123:0/1939186391 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40327f0 con 0x7f89f4153680 2024-04-25T12:38:28.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.306+0000 7f89fa400640 1 -- 172.21.15.123:0/1939186391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f4153680 msgr2=0x7f89f40c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:28.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.306+0000 7f89fa400640 1 --2- 172.21.15.123:0/1939186391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f4153680 0x7f89f40c3be0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f89f40cb380 tx=0x7f89e402a250 comp rx=0 tx=0).stop 2024-04-25T12:38:28.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.307+0000 7f89fa400640 1 -- 172.21.15.123:0/1939186391 shutdown_connections 2024-04-25T12:38:28.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.307+0000 7f89fa400640 1 --2- 172.21.15.123:0/1939186391 >> v2:172.21.15.123:3300/0 conn(0x7f89f40c2870 0x7f89f4074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.307+0000 7f89fa400640 1 --2- 172.21.15.123:0/1939186391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f4153680 0x7f89f40c3be0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.307+0000 7f89fa400640 1 -- 172.21.15.123:0/1939186391 >> 172.21.15.123:0/1939186391 conn(0x7f89f406c570 msgr2=0x7f89f406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:28.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.307+0000 7f89fa400640 1 -- 172.21.15.123:0/1939186391 shutdown_connections 2024-04-25T12:38:28.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.307+0000 7f89fa400640 1 -- 172.21.15.123:0/1939186391 wait complete. 2024-04-25T12:38:28.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.309+0000 7f89fa400640 1 Processor -- start 2024-04-25T12:38:28.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.309+0000 7f89fa400640 1 -- start start 2024-04-25T12:38:28.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.310+0000 7f89fa400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f40746b0 0x7f89f40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:28.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89fa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f40c2870 0x7f89f414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:28.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89fa400640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89f4153680 0x7f89f414af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89fa400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41592a0 con 0x7f89f40c2870 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89fa400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89f40c9f50 con 0x7f89f40746b0 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89fa400640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f89f40c9910 con 0x7f89f4153680 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f40c2870 0x7f89f414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f40c2870 0x7f89f414f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43694/0 (socket says 172.21.15.123:43694) 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 -- 172.21.15.123:0/2029177699 learned_addr learned my addr 172.21.15.123:0/2029177699 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f3e00640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89f4153680 0x7f89f414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f3400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f40746b0 0x7f89f40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 -- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89f4153680 msgr2=0x7f89f414af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89f4153680 0x7f89f414af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 -- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f40746b0 msgr2=0x7f89f40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:28.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f40746b0 0x7f89f40ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.311+0000 7f89f2a00640 1 -- 172.21.15.123:0/2029177699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f40cd110 con 0x7f89f40c2870 2024-04-25T12:38:28.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.312+0000 7f89f2a00640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f40c2870 0x7f89f414f9f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f89e000e900 tx=0x7f89e000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:28.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.312+0000 7f89eb400640 1 -- 172.21.15.123:0/2029177699 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0017070 con 0x7f89f40c2870 2024-04-25T12:38:28.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.312+0000 7f89eb400640 1 -- 172.21.15.123:0/2029177699 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f89e000ca00 con 0x7f89f40c2870 2024-04-25T12:38:28.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.312+0000 7f89eb400640 1 -- 172.21.15.123:0/2029177699 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e000ce10 con 0x7f89f40c2870 2024-04-25T12:38:28.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.312+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f40b2690 con 0x7f89f40c2870 2024-04-25T12:38:28.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.312+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f40b2930 con 0x7f89f40c2870 2024-04-25T12:38:28.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.314+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f4076700 con 0x7f89f40c2870 2024-04-25T12:38:28.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.315+0000 7f89eb400640 1 -- 172.21.15.123:0/2029177699 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f89e001a070 con 0x7f89f40c2870 2024-04-25T12:38:28.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.316+0000 7f89eb400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f89d40744b0 0x7f89d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:28.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.316+0000 7f89eb400640 1 -- 172.21.15.123:0/2029177699 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6653+0+0 (secure 0 0 0) 0x7f89e0095550 con 0x7f89f40c2870 2024-04-25T12:38:28.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.316+0000 7f89f3400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f89d40744b0 0x7f89d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:28.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.317+0000 7f89f3400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f89d40744b0 0x7f89d40769c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f89e4034000 tx=0x7f89e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:28.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.320+0000 7f89eb400640 1 -- 172.21.15.123:0/2029177699 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89e005f2e0 con 0x7f89f40c2870 2024-04-25T12:38:28.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.518+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89f4069db0 con 0x7f89f40c2870 2024-04-25T12:38:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.992+0000 7f89eb400640 1 -- 172.21.15.123:0/2029177699 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7f89e00635e0 con 0x7f89f40c2870 2024-04-25T12:38:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2024-04-25T12:38:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f89d40744b0 msgr2=0x7f89d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f89d40744b0 0x7f89d40769c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f89e4034000 tx=0x7f89e402e000 comp rx=0 tx=0).stop 2024-04-25T12:38:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f40c2870 msgr2=0x7f89f414f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f40c2870 0x7f89f414f9f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f89e000e900 tx=0x7f89e000edc0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 shutdown_connections 2024-04-25T12:38:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f89d40744b0 0x7f89d40769c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89f4153680 0x7f89f414af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f40c2870 0x7f89f414f9f0 secure :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f89e000e900 tx=0x7f89e000edc0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 --2- 172.21.15.123:0/2029177699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89f40746b0 0x7f89f40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:28.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.996+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 >> 172.21.15.123:0/2029177699 conn(0x7f89f406c570 msgr2=0x7f89f406edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:29.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:28.999+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 shutdown_connections 2024-04-25T12:38:29.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.000+0000 7f89fa400640 1 -- 172.21.15.123:0/2029177699 wait complete. 2024-04-25T12:38:29.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T12:38:29.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.297+0000 7f4403e00640 1 Processor -- start 2024-04-25T12:38:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.297+0000 7f4403e00640 1 -- start start 2024-04-25T12:38:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.299+0000 7f4403e00640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44040749b0 0x7f4404152be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.299+0000 7f4403e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f440406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.299+0000 7f4403e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44040b3480 con 0x7f4404072d50 2024-04-25T12:38:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.299+0000 7f4403e00640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4404073260 con 0x7f44040749b0 2024-04-25T12:38:29.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.299+0000 7f4403e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4404073540 con 0x7f440406f500 2024-04-25T12:38:29.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.299+0000 7f4402a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4404072d50 0x7f44040c9880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58782/0 (socket says 172.21.15.123:58782) 2024-04-25T12:38:29.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.299+0000 7f4402a00640 1 -- 172.21.15.123:0/1266145849 learned_addr learned my addr 172.21.15.123:0/1266145849 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T12:38:29.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f43fa000640 1 --2- 172.21.15.123:0/1266145849 >> v2:172.21.15.153:3300/0 conn(0x7f44040749b0 0x7f4404152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:29.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f4403400640 1 --2- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f440406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:29.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f4402000640 1 -- 172.21.15.123:0/1266145849 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409925281 0 0) 0x7f44040b3480 con 0x7f4404072d50 2024-04-25T12:38:29.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f4402000640 1 -- 172.21.15.123:0/1266145849 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43e8003520 con 0x7f4404072d50 2024-04-25T12:38:29.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f43fa000640 1 -- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 msgr2=0x7f440406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:38:29.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f43fa000640 1 --2- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f440406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:29.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f43fa000640 1 -- 172.21.15.123:0/1266145849 >> v1:172.21.15.123:6789/0 conn(0x7f4404072d50 legacy=0x7f44040c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:29.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.300+0000 7f43fa000640 1 -- 172.21.15.123:0/1266145849 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4404074670 con 0x7f44040749b0 2024-04-25T12:38:29.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.301+0000 7f43fa000640 1 --2- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44040749b0 0x7f4404152be0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0097a0 tx=0x7f43ec02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46fa0936772fd79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:29.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.301+0000 7f4402000640 1 -- 172.21.15.123:0/1266145849 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec037070 con 0x7f44040749b0 2024-04-25T12:38:29.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4402000640 1 -- 172.21.15.123:0/1266145849 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f43ec032440 con 0x7f44040749b0 2024-04-25T12:38:29.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4402000640 1 -- 172.21.15.123:0/1266145849 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec032830 con 0x7f44040749b0 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4403e00640 1 -- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44040749b0 msgr2=0x7f4404152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4403e00640 1 --2- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44040749b0 0x7f4404152be0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0097a0 tx=0x7f43ec02a180 comp rx=0 tx=0).stop 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4403e00640 1 -- 172.21.15.123:0/1266145849 shutdown_connections 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4403e00640 1 --2- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f440406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4403e00640 1 --2- 172.21.15.123:0/1266145849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44040749b0 0x7f4404152be0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4403e00640 1 -- 172.21.15.123:0/1266145849 >> 172.21.15.123:0/1266145849 conn(0x7f440406c570 msgr2=0x7f440406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.302+0000 7f4403e00640 1 -- 172.21.15.123:0/1266145849 shutdown_connections 2024-04-25T12:38:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.303+0000 7f4403e00640 1 -- 172.21.15.123:0/1266145849 wait complete. 2024-04-25T12:38:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.305+0000 7f4403e00640 1 Processor -- start 2024-04-25T12:38:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.305+0000 7f4403e00640 1 -- start start 2024-04-25T12:38:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.306+0000 7f4403e00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f4404158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.306+0000 7f4403e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4404072d50 0x7f44041f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.306+0000 7f4403e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44040749b0 0x7f4404146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.306+0000 7f4403e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f440415bee0 con 0x7f44040749b0 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.306+0000 7f4403e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4404153120 con 0x7f4404072d50 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.306+0000 7f4403e00640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4404073f10 con 0x7f440406f500 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f4403400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44040749b0 0x7f4404146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f4403400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44040749b0 0x7f4404146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43704/0 (socket says 172.21.15.123:43704) 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f4402a00640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f4404158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f4403400640 1 -- 172.21.15.123:0/3438281051 learned_addr learned my addr 172.21.15.123:0/3438281051 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T12:38:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f43fa000640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4404072d50 0x7f44041f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f43fa000640 1 -- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 msgr2=0x7f4404158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f43fa000640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f4404158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f43fa000640 1 -- 172.21.15.123:0/3438281051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44040749b0 msgr2=0x7f4404146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f43fa000640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44040749b0 0x7f4404146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f43fa000640 1 -- 172.21.15.123:0/3438281051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44041469d0 con 0x7f4404072d50 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.307+0000 7f4403400640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44040749b0 0x7f4404146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.308+0000 7f43fa000640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4404072d50 0x7f44041f3dd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f43ec031000 tx=0x7f43ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.308+0000 7f43fbe00640 1 -- 172.21.15.123:0/3438281051 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec041030 con 0x7f4404072d50 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.308+0000 7f43fbe00640 1 -- 172.21.15.123:0/3438281051 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f43ec0304c0 con 0x7f4404072d50 2024-04-25T12:38:29.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.308+0000 7f43fbe00640 1 -- 172.21.15.123:0/3438281051 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec037040 con 0x7f4404072d50 2024-04-25T12:38:29.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.308+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4404156b20 con 0x7f4404072d50 2024-04-25T12:38:29.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.308+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4404156e00 con 0x7f4404072d50 2024-04-25T12:38:29.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.309+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4404076da0 con 0x7f4404072d50 2024-04-25T12:38:29.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.310+0000 7f43fbe00640 1 -- 172.21.15.123:0/3438281051 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f43ec0308b0 con 0x7f4404072d50 2024-04-25T12:38:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.311+0000 7f43fbe00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f43e00744b0 0x7f43e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:38:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.312+0000 7f43fbe00640 1 -- 172.21.15.123:0/3438281051 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6065+0+0 (secure 0 0 0) 0x7f43ec0b5710 con 0x7f4404072d50 2024-04-25T12:38:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.312+0000 7f4402a00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f43e00744b0 0x7f43e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:38:29.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.312+0000 7f4402a00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f43e00744b0 0x7f43e00769c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f43f4005c00 tx=0x7f43f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:38:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.316+0000 7f43fbe00640 1 -- 172.21.15.123:0/3438281051 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43ec04b020 con 0x7f4404072d50 2024-04-25T12:38:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.517+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f440406cab0 con 0x7f4404072d50 2024-04-25T12:38:29.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:29.999+0000 7f43fbe00640 1 -- 172.21.15.123:0/3438281051 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) ==== 178+0+0 (secure 0 0 0) 0x7f43ec07f640 con 0x7f4404072d50 2024-04-25T12:38:29.999 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data_ec' removed 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f43e00744b0 msgr2=0x7f43e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f43e00744b0 0x7f43e00769c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f43f4005c00 tx=0x7f43f4009040 comp rx=0 tx=0).stop 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4404072d50 msgr2=0x7f44041f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4404072d50 0x7f44041f3dd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f43ec031000 tx=0x7f43ec0046c0 comp rx=0 tx=0).stop 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 shutdown_connections 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:6800/2246116237,v1:172.21.15.153:6801/2246116237] conn(0x7f43e00744b0 0x7f43e00769c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44040749b0 0x7f4404146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4404072d50 0x7f44041f3dd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 --2- 172.21.15.123:0/3438281051 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f440406f500 0x7f4404158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:38:30.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 >> 172.21.15.123:0/3438281051 conn(0x7f440406c570 msgr2=0x7f44040c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:38:30.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 shutdown_connections 2024-04-25T12:38:30.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T12:38:30.003+0000 7f4403e00640 1 -- 172.21.15.123:0/3438281051 wait complete. 2024-04-25T12:38:30.035 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T12:38:30.035 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T12:38:30.035 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:30.078 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T12:38:30.079 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T12:38:30.079 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:30.110 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T12:38:30.110 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-25T12:38:30.110 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:30.205 INFO:tasks.ceph.mds.e:Stopped 2024-04-25T12:38:30.206 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T12:38:30.206 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:30.217 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T12:38:30.218 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T12:38:30.218 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:30.272 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T12:38:30.272 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-25T12:38:30.273 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:30.325 INFO:tasks.ceph.mds.f:Stopped 2024-04-25T12:38:30.326 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T12:38:30.326 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T12:38:30.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:30.425 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T12:38:30.425 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T12:38:30.425 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:32.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-25T12:38:36.527 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T12:38:36.528 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T12:38:36.528 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:36.623 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T12:38:36.623 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T12:38:36.623 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:36.693 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T12:38:36.693 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T12:38:36.693 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:36.726 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T12:38:36.726 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T12:38:36.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:36.796 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T12:38:36.797 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T12:38:36.797 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:36.869 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T12:38:36.870 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T12:38:36.870 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:36.935 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T12:38:36.936 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T12:38:36.936 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T12:38:36.936 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:38.043 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-25T12:38:38.043 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-25T12:38:38.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:38.338 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:43.038 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T12:38:43.038 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T12:38:43.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:43.069 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T12:38:43.069 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T12:38:43.070 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T12:38:43.070 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:43.143 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T12:38:43.143 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T12:38:43.143 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:43.157 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T12:38:43.157 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T12:38:43.157 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T12:38:43.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-04-25T12:38:43.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-25T12:38:43.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-25T12:38:43.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-25T12:38:43.210 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T12:38:43.210 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T12:38:43.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v MDS_SLOW_METADATA_IO | egrep -v MDS_TRIM | head -n 1 2024-04-25T12:38:43.342 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:38:43.342 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T12:38:43.526 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:38:43.526 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T12:38:43.654 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:38:43.654 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T12:38:43.822 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:38:43.823 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T12:38:43.978 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:38:43.979 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T12:38:44.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:38:44.134 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T12:38:44.198 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:38:44.198 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T12:38:44.258 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:38:44.258 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T12:38:44.331 INFO:tasks.ceph:Archiving mon data... 2024-04-25T12:38:44.332 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/data/mon.a.tgz 2024-04-25T12:38:44.333 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-25T12:38:44.390 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.BkC815OkcG 2024-04-25T12:38:44.391 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.BkC815OkcG 2024-04-25T12:38:44.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.BkC815OkcG 2024-04-25T12:38:44.707 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.BkC815OkcG is 400KB 2024-04-25T12:38:44.809 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.BkC815OkcG 2024-04-25T12:38:44.866 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/data/mon.b.tgz 2024-04-25T12:38:44.867 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2024-04-25T12:38:44.885 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.Fx6AY8qbTD 2024-04-25T12:38:44.885 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Fx6AY8qbTD 2024-04-25T12:38:45.085 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.Fx6AY8qbTD 2024-04-25T12:38:45.166 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.Fx6AY8qbTD is 559KB 2024-04-25T12:38:45.266 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.Fx6AY8qbTD 2024-04-25T12:38:45.280 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/data/mon.c.tgz 2024-04-25T12:38:45.280 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2024-04-25T12:38:45.334 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.UiepsOfX1Y 2024-04-25T12:38:45.334 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.UiepsOfX1Y 2024-04-25T12:38:45.511 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.UiepsOfX1Y 2024-04-25T12:38:45.592 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.UiepsOfX1Y is 558KB 2024-04-25T12:38:45.691 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.UiepsOfX1Y 2024-04-25T12:38:45.705 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T12:38:45.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T12:38:45.714 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T12:38:45.847 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T12:38:45.861 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T12:38:45.862 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T12:38:45.863 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/remote/smithi123/crash 2024-04-25T12:38:45.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T12:38:45.947 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/remote/smithi153/crash 2024-04-25T12:38:45.948 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T12:38:45.970 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T12:38:45.971 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:45.978 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:46.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:46.171 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T12:38:46.194 INFO:tasks.ceph:Compressing logs... 2024-04-25T12:38:46.194 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T12:38:46.202 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T12:38:46.265 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.39889.log 2024-04-25T12:38:46.265 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40100.log 2024-04-25T12:38:46.265 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/objectstore_tool.39889.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T12:38:46.265 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/objectstore_tool.40100.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-25T12:38:46.265 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T12:38:46.265 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34413.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mds.f.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34533.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34413.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34533.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34413.log.gz 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34533.log.gz 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-mds.b.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40311.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/objectstore_tool.40311.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-04-25T12:38:46.266 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34453.log 2024-04-25T12:38:46.267 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34613.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-04-25T12:38:46.267 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T12:38:46.267 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34453.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34453.log.gz 2024-04-25T12:38:46.267 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T12:38:46.267 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mgr.x.log: 93.7% -- replaced with /var/log/ceph/objectstore_tool.39889.log.gz 2024-04-25T12:38:46.268 INFO:teuthology.orchestra.run.smithi153.stderr: 93.6% -- replaced with /var/log/ceph/objectstore_tool.40100.log.gz 2024-04-25T12:38:46.275 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T12:38:46.275 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-04-25T12:38:46.276 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46046.log 2024-04-25T12:38:46.277 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54582.log 2024-04-25T12:38:46.278 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52343.log 2024-04-25T12:38:46.278 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46046.log.gz 2024-04-25T12:38:46.279 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50452.log 2024-04-25T12:38:46.280 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54582.log: /var/log/ceph/ceph-client.admin.52343.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54582.log.gz 2024-04-25T12:38:46.281 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52343.log.gz 2024-04-25T12:38:46.281 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43171.log 2024-04-25T12:38:46.281 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54010.log 2024-04-25T12:38:46.282 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.6.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-25T12:38:46.283 INFO:teuthology.orchestra.run.smithi123.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.50452.log.gz 2024-04-25T12:38:46.283 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44324.log 2024-04-25T12:38:46.283 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.43171.log.gz/var/log/ceph/ceph-client.admin.54010.log: 2024-04-25T12:38:46.285 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34333.log 2024-04-25T12:38:46.285 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34373.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-04-25T12:38:46.285 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T12:38:46.285 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34333.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34333.log.gz 2024-04-25T12:38:46.285 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T12:38:46.287 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54010.log.gz 2024-04-25T12:38:46.287 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52702.log 2024-04-25T12:38:46.288 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55763.log 2024-04-25T12:38:46.288 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44324.log.gz 2024-04-25T12:38:46.288 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43651.log 2024-04-25T12:38:46.288 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52702.log.gz 2024-04-25T12:38:46.288 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55763.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55763.log.gz 2024-04-25T12:38:46.288 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47677.log 2024-04-25T12:38:46.288 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40608.log 2024-04-25T12:38:46.289 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47677.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43651.log.gz 2024-04-25T12:38:46.289 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55285.log 2024-04-25T12:38:46.289 INFO:teuthology.orchestra.run.smithi123.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47677.log.gz 2024-04-25T12:38:46.289 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49422.log 2024-04-25T12:38:46.290 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40608.log.gz 2024-04-25T12:38:46.290 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38906.log 2024-04-25T12:38:46.291 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55285.log.gz 2024-04-25T12:38:46.291 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-04-25T12:38:46.292 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49422.log.gz 2024-04-25T12:38:46.292 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44166.log 2024-04-25T12:38:46.292 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38906.log.gz 2024-04-25T12:38:46.293 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40824.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-04-25T12:38:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54805.log 2024-04-25T12:38:46.294 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46521.log 2024-04-25T12:38:46.295 INFO:teuthology.orchestra.run.smithi123.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.44166.log.gz 2024-04-25T12:38:46.295 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40255.log 2024-04-25T12:38:46.296 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54805.log.gz 2024-04-25T12:38:46.296 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46521.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.46521.log.gz -- 2024-04-25T12:38:46.296 INFO:teuthology.orchestra.run.smithi123.stderr: /var/log/ceph/ceph-client.admin.42995.log 2024-04-25T12:38:46.297 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-04-25T12:38:46.298 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41228.log 2024-04-25T12:38:46.299 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40255.log.gz 2024-04-25T12:38:46.299 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37964.log 2024-04-25T12:38:46.299 INFO:teuthology.orchestra.run.smithi123.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.42995.log.gz 2024-04-25T12:38:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39732.log 2024-04-25T12:38:46.300 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-04-25T12:38:46.300 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47032.log 2024-04-25T12:38:46.301 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39732.log: gzip 90.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.37964.log.gz 2024-04-25T12:38:46.301 INFO:teuthology.orchestra.run.smithi123.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37796.log 2024-04-25T12:38:46.302 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39732.log.gz 2024-04-25T12:38:46.302 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37403.log 2024-04-25T12:38:46.302 INFO:teuthology.orchestra.run.smithi123.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47032.log.gz 2024-04-25T12:38:46.303 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53332.log 2024-04-25T12:38:46.304 INFO:teuthology.orchestra.run.smithi123.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37796.log.gz 2024-04-25T12:38:46.304 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mon.b.log: 93.8% -- replaced with /var/log/ceph/objectstore_tool.40311.log.gz 2024-04-25T12:38:46.305 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39948.log 2024-04-25T12:38:46.305 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37403.log.gz 2024-04-25T12:38:46.306 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41382.log 2024-04-25T12:38:46.306 INFO:teuthology.orchestra.run.smithi123.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53332.log.gz 2024-04-25T12:38:46.306 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39948.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39948.log.gz 2024-04-25T12:38:46.307 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50911.log 2024-04-25T12:38:46.307 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53080.log 2024-04-25T12:38:46.308 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41382.log.gz 2024-04-25T12:38:46.308 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53346.log 2024-04-25T12:38:46.309 INFO:teuthology.orchestra.run.smithi123.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50911.log.gz 2024-04-25T12:38:46.309 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53080.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53080.log.gz 2024-04-25T12:38:46.310 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45535.log 2024-04-25T12:38:46.310 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53346.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T12:38:46.312 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53346.log.gz 2024-04-25T12:38:46.312 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49475.log 2024-04-25T12:38:46.312 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45535.log.gz 2024-04-25T12:38:46.312 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34573.log 2024-04-25T12:38:46.312 INFO:teuthology.orchestra.run.smithi153.stderr: 92.2%/var/log/ceph/objectstore_tool.41228.log: -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T12:38:46.313 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54847.log 2024-04-25T12:38:46.313 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49475.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49475.log.gz 2024-04-25T12:38:46.313 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-04-25T12:38:46.314 INFO:teuthology.orchestra.run.smithi123.stderr: 85.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T12:38:46.314 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47956.log 2024-04-25T12:38:46.315 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54847.log.gz 2024-04-25T12:38:46.315 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-04-25T12:38:46.315 INFO:teuthology.orchestra.run.smithi123.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-04-25T12:38:46.316 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48339.log 2024-04-25T12:38:46.316 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47956.log.gz 2024-04-25T12:38:46.317 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47816.log 2024-04-25T12:38:46.318 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-04-25T12:38:46.318 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55453.log 2024-04-25T12:38:46.318 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.48339.log.gz 2024-04-25T12:38:46.319 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39947.log 2024-04-25T12:38:46.319 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47816.log.gz 2024-04-25T12:38:46.320 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39944.log 2024-04-25T12:38:46.320 INFO:teuthology.orchestra.run.smithi123.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55453.log.gz 2024-04-25T12:38:46.322 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53336.log 2024-04-25T12:38:46.322 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39947.log.gz 2024-04-25T12:38:46.322 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39944.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T12:38:46.323 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39944.log.gz 2024-04-25T12:38:46.323 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37489.log 2024-04-25T12:38:46.325 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53336.log.gz 2024-04-25T12:38:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39942.log 2024-04-25T12:38:46.325 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34493.log 2024-04-25T12:38:46.325 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34573.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34573.log.gz 2024-04-25T12:38:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-04-25T12:38:46.326 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37489.log.gz 2024-04-25T12:38:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37922.log 2024-04-25T12:38:46.327 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39942.log.gz 2024-04-25T12:38:46.328 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42807.log 2024-04-25T12:38:46.328 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-04-25T12:38:46.329 INFO:teuthology.orchestra.run.smithi153.stderr: 93.2% -- replaced with /var/log/ceph/objectstore_tool.41228.log.gz 2024-04-25T12:38:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39955.log 2024-04-25T12:38:46.330 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37922.log.gz 2024-04-25T12:38:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54016.log 2024-04-25T12:38:46.331 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.34493.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34493.log.gz 2024-04-25T12:38:46.331 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42807.log.gz 2024-04-25T12:38:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39564.log 2024-04-25T12:38:46.331 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39955.log.gz 2024-04-25T12:38:46.332 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53682.log 2024-04-25T12:38:46.332 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54016.log.gz 2024-04-25T12:38:46.333 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-04-25T12:38:46.334 INFO:teuthology.orchestra.run.smithi123.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39564.log.gz 2024-04-25T12:38:46.334 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53682.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T12:38:46.334 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53682.log.gz 2024-04-25T12:38:46.335 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53676.log 2024-04-25T12:38:46.335 INFO:teuthology.orchestra.run.smithi123.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-04-25T12:38:46.336 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40577.log 2024-04-25T12:38:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46738.log 2024-04-25T12:38:46.337 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53676.log.gz 2024-04-25T12:38:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40577.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40577.log.gz 2024-04-25T12:38:46.338 INFO:teuthology.orchestra.run.smithi123.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T12:38:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52828.log 2024-04-25T12:38:46.339 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44008.log 2024-04-25T12:38:46.339 INFO:teuthology.orchestra.run.smithi123.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46738.log.gz 2024-04-25T12:38:46.340 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56034.log 2024-04-25T12:38:46.341 INFO:teuthology.orchestra.run.smithi123.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52828.log.gz 2024-04-25T12:38:46.341 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50392.log 2024-04-25T12:38:46.342 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44008.log.gz 2024-04-25T12:38:46.342 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.56034.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.56034.log.gz --verbose 2024-04-25T12:38:46.342 INFO:teuthology.orchestra.run.smithi123.stderr: -- /var/log/ceph/ceph-client.admin.52870.log 2024-04-25T12:38:46.344 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40561.log 2024-04-25T12:38:46.344 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52870.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50392.log.gz 2024-04-25T12:38:46.345 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52870.log.gz 2024-04-25T12:38:46.345 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55889.log 2024-04-25T12:38:46.346 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38964.log 2024-04-25T12:38:46.346 INFO:teuthology.orchestra.run.smithi153.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T12:38:46.346 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40561.log.gz 2024-04-25T12:38:46.347 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47271.log 2024-04-25T12:38:46.347 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55889.log.gz 2024-04-25T12:38:46.348 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53671.log 2024-04-25T12:38:46.349 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38964.log.gz 2024-04-25T12:38:46.349 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47271.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47271.log.gz 2024-04-25T12:38:46.349 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51878.log 2024-04-25T12:38:46.350 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50509.log 2024-04-25T12:38:46.350 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53671.log.gz 2024-04-25T12:38:46.351 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51878.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51878.log.gz 2024-04-25T12:38:46.351 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55099.log 2024-04-25T12:38:46.352 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50509.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T12:38:46.353 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55099.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50509.log.gz 2024-04-25T12:38:46.353 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55099.log.gz 2024-04-25T12:38:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43390.log 2024-04-25T12:38:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50985.log 2024-04-25T12:38:46.356 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52996.log 2024-04-25T12:38:46.356 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43390.log.gz 2024-04-25T12:38:46.357 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55847.log 2024-04-25T12:38:46.357 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52996.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50985.log.gz 2024-04-25T12:38:46.358 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52996.log.gz 2024-04-25T12:38:46.358 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47836.log 2024-04-25T12:38:46.359 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55847.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55847.log.gz 2024-04-25T12:38:46.359 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52152.log 2024-04-25T12:38:46.360 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53667.log 2024-04-25T12:38:46.361 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52152.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47836.log.gz 2024-04-25T12:38:46.362 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52152.log.gz 2024-04-25T12:38:46.362 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48518.log 2024-04-25T12:38:46.363 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53667.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.54298.log -- replaced with /var/log/ceph/ceph-client.admin.53667.log.gz 2024-04-25T12:38:46.363 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T12:38:46.364 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37754.log 2024-04-25T12:38:46.365 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54298.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48518.log.gz 2024-04-25T12:38:46.366 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-04-25T12:38:46.366 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52912.log 2024-04-25T12:38:46.367 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45298.log 2024-04-25T12:38:46.367 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37754.log.gz 2024-04-25T12:38:46.368 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42383.log 2024-04-25T12:38:46.368 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45298.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52912.log.gz 2024-04-25T12:38:46.369 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45298.log.gz 2024-04-25T12:38:46.369 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49613.log 2024-04-25T12:38:46.370 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42383.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42383.log.gz 2024-04-25T12:38:46.371 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49856.log 2024-04-25T12:38:46.372 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40908.log 2024-04-25T12:38:46.372 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49856.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49613.log.gz 2024-04-25T12:38:46.373 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49856.log.gz 2024-04-25T12:38:46.373 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53344.log 2024-04-25T12:38:46.374 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40908.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40908.log.gz 2024-04-25T12:38:46.374 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44067.log 2024-04-25T12:38:46.375 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39132.log 2024-04-25T12:38:46.376 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44067.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53344.log.gz 2024-04-25T12:38:46.377 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.44067.log.gz 2024-04-25T12:38:46.377 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38693.log 2024-04-25T12:38:46.378 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38780.log 2024-04-25T12:38:46.378 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39132.log.gz 2024-04-25T12:38:46.379 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38693.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38693.log.gz 2024-04-25T12:38:46.379 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48896.log 2024-04-25T12:38:46.380 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38780.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T12:38:46.380 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38780.log.gz 2024-04-25T12:38:46.381 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52007.log 2024-04-25T12:38:46.381 INFO:teuthology.orchestra.run.smithi123.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.48896.log.gz 2024-04-25T12:38:46.382 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53343.log 2024-04-25T12:38:46.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52007.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52007.log.gz 2024-04-25T12:38:46.384 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51862.log 2024-04-25T12:38:46.385 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37093.log 2024-04-25T12:38:46.385 INFO:teuthology.orchestra.run.smithi123.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53343.log.gz 2024-04-25T12:38:46.386 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51862.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51862.log.gz 2024-04-25T12:38:46.386 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40950.log 2024-04-25T12:38:46.388 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48160.log 2024-04-25T12:38:46.389 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37093.log: /var/log/ceph/ceph-client.admin.40950.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37093.log.gz 2024-04-25T12:38:46.389 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46345.log 2024-04-25T12:38:46.389 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40950.log.gz 2024-04-25T12:38:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48160.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48160.log.gz 2024-04-25T12:38:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49292.log 2024-04-25T12:38:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48737.log 2024-04-25T12:38:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46345.log: /var/log/ceph/ceph-client.admin.49292.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.46345.log.gz 2024-04-25T12:38:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50629.log 2024-04-25T12:38:46.393 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49292.log.gz 2024-04-25T12:38:46.394 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37307.log 2024-04-25T12:38:46.394 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.48737.log.gz 2024-04-25T12:38:46.395 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50629.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50629.log.gz 2024-04-25T12:38:46.395 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52259.log 2024-04-25T12:38:46.396 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54889.log 2024-04-25T12:38:46.397 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37307.log: /var/log/ceph/ceph-client.admin.52259.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37307.log.gz 2024-04-25T12:38:46.398 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 87.9% 2024-04-25T12:38:46.398 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52259.log.gz 2024-04-25T12:38:46.399 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54889.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T12:38:46.399 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54889.log.gz 2024-04-25T12:38:46.400 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41810.log 2024-04-25T12:38:46.401 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39774.log 2024-04-25T12:38:46.403 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45931.log 2024-04-25T12:38:46.403 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41810.log.gz 2024-04-25T12:38:46.405 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38864.log 2024-04-25T12:38:46.405 INFO:teuthology.orchestra.run.smithi123.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39774.log.gz 2024-04-25T12:38:46.406 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51469.log 2024-04-25T12:38:46.406 INFO:teuthology.orchestra.run.smithi123.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.45931.log.gz 2024-04-25T12:38:46.408 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38864.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38864.log.gz 2024-04-25T12:38:46.408 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54214.log 2024-04-25T12:38:46.409 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51469.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T12:38:46.409 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T12:38:46.409 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.51469.log.gz 2024-04-25T12:38:46.410 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42166.log 2024-04-25T12:38:46.410 INFO:teuthology.orchestra.run.smithi123.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54214.log.gz 2024-04-25T12:38:46.411 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42299.log 2024-04-25T12:38:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.0.42166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42166.log.gz 2024-04-25T12:38:46.413 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55681.log 2024-04-25T12:38:46.414 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44564.log 2024-04-25T12:38:46.414 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42299.log.gz 2024-04-25T12:38:46.416 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54372.log 2024-04-25T12:38:46.416 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55681.log.gz 2024-04-25T12:38:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44564.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44564.log.gz 2024-04-25T12:38:46.417 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39270.log 2024-04-25T12:38:46.418 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54372.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54372.log.gz 2024-04-25T12:38:46.419 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54414.log 2024-04-25T12:38:46.420 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44763.log 2024-04-25T12:38:46.420 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39270.log.gz 2024-04-25T12:38:46.421 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54414.log: gzip -5 87.7% --verbose -- /var/log/ceph/ceph-client.admin.51772.log 2024-04-25T12:38:46.421 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54414.log.gz 2024-04-25T12:38:46.423 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44763.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44763.log.gz -5 2024-04-25T12:38:46.423 INFO:teuthology.orchestra.run.smithi123.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48359.log 2024-04-25T12:38:46.424 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52217.log 2024-04-25T12:38:46.425 INFO:teuthology.orchestra.run.smithi123.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51772.log.gz 2024-04-25T12:38:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50731.log 2024-04-25T12:38:46.426 INFO:teuthology.orchestra.run.smithi123.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48359.log.gz 2024-04-25T12:38:46.426 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52217.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52217.log.gz 2024-04-25T12:38:46.427 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53672.log 2024-04-25T12:38:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49373.log 2024-04-25T12:38:46.429 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz 2024-04-25T12:38:46.430 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54017.log 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53672.log.gz 2024-04-25T12:38:46.430 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T12:38:46.431 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51130.log 2024-04-25T12:38:46.432 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49373.log.gz 2024-04-25T12:38:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-04-25T12:38:46.433 INFO:teuthology.orchestra.run.smithi123.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54017.log.gz 2024-04-25T12:38:46.434 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54456.log 2024-04-25T12:38:46.434 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51130.log.gz 2024-04-25T12:38:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46581.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-04-25T12:38:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40305.log 2024-04-25T12:38:46.437 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54456.log.gz 2024-04-25T12:38:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50788.log 2024-04-25T12:38:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47417.log 2024-04-25T12:38:46.439 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40305.log.gz 2024-04-25T12:38:46.441 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50788.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50788.log.gz 2024-04-25T12:38:46.441 INFO:teuthology.orchestra.run.smithi123.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37361.log 2024-04-25T12:38:46.442 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47417.log: 90.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50312.log 2024-04-25T12:38:46.442 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47417.log.gz 2024-04-25T12:38:46.443 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43568.log 2024-04-25T12:38:46.443 INFO:teuthology.orchestra.run.smithi123.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37361.log.gz 2024-04-25T12:38:46.444 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50312.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50312.log.gz 2024-04-25T12:38:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39954.log 2024-04-25T12:38:46.446 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53677.log 2024-04-25T12:38:46.446 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43568.log.gz 2024-04-25T12:38:46.447 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52062.log 2024-04-25T12:38:46.448 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39954.log.gz 2024-04-25T12:38:46.449 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53677.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53677.log.gz 2024-04-25T12:38:46.450 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47335.log/var/log/ceph/ceph-client.admin.52062.log: 2024-04-25T12:38:46.450 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52062.log.gz 2024-04-25T12:38:46.450 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55141.log 2024-04-25T12:38:46.452 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55238.log 2024-04-25T12:38:46.452 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47335.log.gz 2024-04-25T12:38:46.454 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55141.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55141.log.gz 2024-04-25T12:38:46.454 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46169.log 2024-04-25T12:38:46.459 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55238.log.gz 2024-04-25T12:38:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42624.log 2024-04-25T12:38:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37533.log 2024-04-25T12:38:46.459 INFO:teuthology.orchestra.run.smithi123.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46169.log.gz 2024-04-25T12:38:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40604.log 2024-04-25T12:38:46.459 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.42624.log.gz 2024-04-25T12:38:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37533.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37533.log.gz 2024-04-25T12:38:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55537.log 2024-04-25T12:38:46.461 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40604.log: gzip 89.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40604.log.gz -- 2024-04-25T12:38:46.461 INFO:teuthology.orchestra.run.smithi123.stderr: /var/log/ceph/ceph-client.admin.55805.log 2024-04-25T12:38:46.462 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40264.log 2024-04-25T12:38:46.463 INFO:teuthology.orchestra.run.smithi123.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55537.log.gz 2024-04-25T12:38:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55805.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55805.log.gz 2024-04-25T12:38:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2024-04-25T12:38:46.465 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40264.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40264.log.gz 2024-04-25T12:38:46.466 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38459.log 2024-04-25T12:38:46.467 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51965.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T12:38:46.467 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2024-04-25T12:38:46.471 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38459.log: /var/log/ceph/ceph-mon.a.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.38459.log.gz 2024-04-25T12:38:46.471 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52301.log 2024-04-25T12:38:46.472 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54498.log 2024-04-25T12:38:46.473 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-04-25T12:38:46.474 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52301.log.gz 2024-04-25T12:38:46.475 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45009.log 2024-04-25T12:38:46.475 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54498.log.gz 2024-04-25T12:38:46.476 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45869.log 2024-04-25T12:38:46.478 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44908.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T12:38:46.478 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-04-25T12:38:46.478 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-04-25T12:38:46.479 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45869.log: /var/log/ceph/ceph-client.admin.45009.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45869.log.gz 2024-04-25T12:38:46.480 INFO:teuthology.orchestra.run.smithi123.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.45009.log.gz 2024-04-25T12:38:46.480 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.36960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55015.log 2024-04-25T12:38:46.481 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-04-25T12:38:46.481 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37669.log 2024-04-25T12:38:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41340.log 2024-04-25T12:38:46.484 INFO:teuthology.orchestra.run.smithi123.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55015.log.gz 2024-04-25T12:38:46.484 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37669.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55621.log 2024-04-25T12:38:46.485 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37669.log.gz 2024-04-25T12:38:46.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41340.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41340.log.gz 2024-04-25T12:38:46.486 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50035.log 2024-04-25T12:38:46.487 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55621.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55621.log.gz 2024-04-25T12:38:46.487 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37880.log 2024-04-25T12:38:46.489 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39522.log 2024-04-25T12:38:46.489 INFO:teuthology.orchestra.run.smithi123.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50035.log.gz 2024-04-25T12:38:46.491 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37880.log: gzip -5 --verbose 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37880.log.gz 2024-04-25T12:38:46.491 INFO:teuthology.orchestra.run.smithi123.stderr: -- /var/log/ceph/ceph-client.admin.42487.log 2024-04-25T12:38:46.491 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39522.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39522.log.gz 2024-04-25T12:38:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40680.log 2024-04-25T12:38:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52618.log 2024-04-25T12:38:46.494 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42487.log.gz 2024-04-25T12:38:46.495 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-04-25T12:38:46.495 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40680.log.gz 2024-04-25T12:38:46.496 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52618.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52618.log.gz 2024-04-25T12:38:46.496 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55369.log 2024-04-25T12:38:46.498 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47558.log 2024-04-25T12:38:46.498 INFO:teuthology.orchestra.run.smithi123.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz 2024-04-25T12:38:46.499 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43410.log 2024-04-25T12:38:46.499 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55369.log.gz 2024-04-25T12:38:46.500 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52576.log 2024-04-25T12:38:46.501 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47558.log.gz 2024-04-25T12:38:46.502 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52516.log 2024-04-25T12:38:46.502 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43410.log.gz 2024-04-25T12:38:46.503 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42425.log 2024-04-25T12:38:46.503 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52576.log.gz 2024-04-25T12:38:46.504 INFO:teuthology.orchestra.run.smithi153.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T12:38:46.505 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52516.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52516.log.gz 2024-04-25T12:38:46.506 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.42425.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37192.log 2024-04-25T12:38:46.506 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42425.log.gz 2024-04-25T12:38:46.506 INFO:teuthology.orchestra.run.smithi153.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T12:38:46.506 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44846.log 2024-04-25T12:38:46.508 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46779.log 2024-04-25T12:38:46.508 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37192.log.gz 2024-04-25T12:38:46.510 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44846.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44846.log.gz 2024-04-25T12:38:46.510 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45170.log 2024-04-25T12:38:46.510 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46779.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46779.log.gz 2024-04-25T12:38:46.511 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48140.log 2024-04-25T12:38:46.513 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44233.log 2024-04-25T12:38:46.513 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45170.log.gz 2024-04-25T12:38:46.515 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48140.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48140.log.gz 2024-04-25T12:38:46.515 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.44233.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41034.log 2024-04-25T12:38:46.515 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44233.log.gz 2024-04-25T12:38:46.516 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54018.log 2024-04-25T12:38:46.518 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2024-04-25T12:38:46.518 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41034.log.gz 2024-04-25T12:38:46.519 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54018.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54018.log.gz 2024-04-25T12:38:46.519 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37625.log 2024-04-25T12:38:46.520 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50144.log: 86.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54004.log 2024-04-25T12:38:46.521 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2024-04-25T12:38:46.522 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52069.log 2024-04-25T12:38:46.522 INFO:teuthology.orchestra.run.smithi123.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37625.log.gz 2024-04-25T12:38:46.523 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41076.log 2024-04-25T12:38:46.523 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54004.log.gz 2024-04-25T12:38:46.525 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52069.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52069.log.gz 2024-04-25T12:38:46.526 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.41076.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-04-25T12:38:46.526 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41076.log.gz 2024-04-25T12:38:46.527 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40246.log 2024-04-25T12:38:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41169.log 2024-04-25T12:38:46.528 INFO:teuthology.orchestra.run.smithi123.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-04-25T12:38:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40246.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40246.log.gz 2024-04-25T12:38:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39480.log 2024-04-25T12:38:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41169.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41169.log.gz 2024-04-25T12:38:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39006.log 2024-04-25T12:38:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41213.log 2024-04-25T12:38:46.533 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39480.log.gz 2024-04-25T12:38:46.535 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39006.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39006.log.gz 2024-04-25T12:38:46.535 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39090.log 2024-04-25T12:38:46.536 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41213.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41213.log.gz 2024-04-25T12:38:46.536 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39438.log 2024-04-25T12:38:46.538 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33196.log 2024-04-25T12:38:46.538 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39090.log.gz 2024-04-25T12:38:46.539 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39438.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39438.log.gz 2024-04-25T12:38:46.540 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2024-04-25T12:38:46.540 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.33196.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.33196.log.gz 2024-04-25T12:38:46.541 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53164.log 2024-04-25T12:38:46.542 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49033.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-25T12:38:46.542 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2024-04-25T12:38:46.544 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53164.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53164.log.gz 2024-04-25T12:38:46.545 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-mds.e.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48595.log 2024-04-25T12:38:46.547 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose 92.6% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-25T12:38:46.547 INFO:teuthology.orchestra.run.smithi123.stderr: -- /var/log/ceph/ceph-client.admin.52432.log 2024-04-25T12:38:46.548 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48595.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-04-25T12:38:46.548 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48595.log.gz 2024-04-25T12:38:46.549 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52432.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52432.log.gz 2024-04-25T12:38:46.549 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40265.log 2024-04-25T12:38:46.550 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37838.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-04-25T12:38:46.551 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42625.log 2024-04-25T12:38:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2024-04-25T12:38:46.552 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40265.log.gz 2024-04-25T12:38:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42625.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42625.log.gz 2024-04-25T12:38:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54973.log 2024-04-25T12:38:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46304.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2024-04-25T12:38:46.555 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55931.log 2024-04-25T12:38:46.556 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43829.log 2024-04-25T12:38:46.556 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54973.log.gz 2024-04-25T12:38:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55931.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55931.log.gz 2024-04-25T12:38:46.558 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44452.log 2024-04-25T12:38:46.559 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45003.log 2024-04-25T12:38:46.559 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43829.log.gz 2024-04-25T12:38:46.561 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40582.log 2024-04-25T12:38:46.561 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.44452.log.gz 2024-04-25T12:38:46.562 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45003.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45003.log.gz 2024-04-25T12:38:46.562 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44626.log 2024-04-25T12:38:46.563 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40582.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40582.log.gz 2024-04-25T12:38:46.564 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45555.log 2024-04-25T12:38:46.566 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37583.log 2024-04-25T12:38:46.566 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44626.log.gz 2024-04-25T12:38:46.567 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52954.log 2024-04-25T12:38:46.567 INFO:teuthology.orchestra.run.smithi123.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.45555.log.gz 2024-04-25T12:38:46.568 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37583.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37583.log.gz 2024-04-25T12:38:46.568 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43228.log 2024-04-25T12:38:46.569 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41466.log 2024-04-25T12:38:46.570 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52954.log.gz 2024-04-25T12:38:46.571 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48037.log 2024-04-25T12:38:46.571 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43228.log.gz 2024-04-25T12:38:46.572 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53681.log 2024-04-25T12:38:46.572 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41466.log.gz 2024-04-25T12:38:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48037.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48037.log.gz 2024-04-25T12:38:46.574 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42623.log 2024-04-25T12:38:46.575 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53681.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53681.log.gz 2024-04-25T12:38:46.575 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49836.log 2024-04-25T12:38:46.577 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52385.log 2024-04-25T12:38:46.577 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42623.log.gz 2024-04-25T12:38:46.578 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49836.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49836.log.gz 2024-04-25T12:38:46.579 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52153.log 2024-04-25T12:38:46.580 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33564.log 2024-04-25T12:38:46.580 INFO:teuthology.orchestra.run.smithi123.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52385.log.gz 2024-04-25T12:38:46.582 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53680.log 2024-04-25T12:38:46.582 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52153.log.gz 2024-04-25T12:38:46.583 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.33564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51648.log 2024-04-25T12:38:46.583 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33564.log.gz 2024-04-25T12:38:46.584 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41616.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53680.log.gz 2024-04-25T12:38:46.584 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T12:38:46.585 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51648.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51648.log.gz 2024-04-25T12:38:46.586 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52660.log 2024-04-25T12:38:46.587 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37711.log 2024-04-25T12:38:46.588 INFO:teuthology.orchestra.run.smithi123.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41616.log.gz 2024-04-25T12:38:46.588 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52660.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52660.log.gz 2024-04-25T12:38:46.589 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39690.log 2024-04-25T12:38:46.590 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47174.log 2024-04-25T12:38:46.590 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37711.log.gz 2024-04-25T12:38:46.591 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42147.log 2024-04-25T12:38:46.592 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39690.log.gz 2024-04-25T12:38:46.593 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47174.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47174.log.gz 2024-04-25T12:38:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53340.log 2024-04-25T12:38:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.0.42147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42147.log.gz 2024-04-25T12:38:46.594 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2024-04-25T12:38:46.596 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52786.log 2024-04-25T12:38:46.596 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53340.log.gz 2024-04-25T12:38:46.598 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43728.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 2024-04-25T12:38:46.598 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55579.log 2024-04-25T12:38:46.599 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52786.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52786.log.gz 2024-04-25T12:38:46.600 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39354.log 2024-04-25T12:38:46.601 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48834.log 2024-04-25T12:38:46.601 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55579.log.gz 2024-04-25T12:38:46.603 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39354.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39354.log.gz 2024-04-25T12:38:46.603 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.48834.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51251.log 2024-04-25T12:38:46.603 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48834.log.gz 2024-04-25T12:38:46.604 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50147.log 2024-04-25T12:38:46.606 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.51251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41550.log 2024-04-25T12:38:46.606 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51251.log.gz 2024-04-25T12:38:46.608 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50147.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50147.log.gz 2024-04-25T12:38:46.608 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-04-25T12:38:46.609 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41256.log 2024-04-25T12:38:46.609 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41550.log.gz 2024-04-25T12:38:46.611 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39648.log 2024-04-25T12:38:46.611 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz 2024-04-25T12:38:46.612 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41256.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41256.log.gz 2024-04-25T12:38:46.613 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.39648.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55327.log 2024-04-25T12:38:46.613 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39648.log.gz 2024-04-25T12:38:46.614 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41424.log 2024-04-25T12:38:46.616 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2024-04-25T12:38:46.616 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55327.log.gz 2024-04-25T12:38:46.617 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41424.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41424.log.gz 2024-04-25T12:38:46.617 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39606.log 2024-04-25T12:38:46.618 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49617.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-04-25T12:38:46.619 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53038.log 2024-04-25T12:38:46.620 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41119.log 2024-04-25T12:38:46.620 INFO:teuthology.orchestra.run.smithi123.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39606.log.gz 2024-04-25T12:38:46.622 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53038.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53038.log.gz 2024-04-25T12:38:46.622 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.41119.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52474.log 2024-04-25T12:38:46.623 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41119.log.gz 2024-04-25T12:38:46.623 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T12:38:46.625 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51409.log 2024-04-25T12:38:46.625 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52474.log.gz 2024-04-25T12:38:46.627 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.51409.log: 92.5% 88.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz -- replaced with /var/log/ceph/ceph-client.admin.51409.log.gz 2024-04-25T12:38:46.628 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T12:38:46.628 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39048.log 2024-04-25T12:38:46.629 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47231.log 2024-04-25T12:38:46.631 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37445.log 2024-04-25T12:38:46.631 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39048.log.gz 2024-04-25T12:38:46.632 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43032.log 2024-04-25T12:38:46.633 INFO:teuthology.orchestra.run.smithi123.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.47231.log.gz 2024-04-25T12:38:46.633 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37445.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37445.log.gz 2024-04-25T12:38:46.635 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43032.log.gz 2024-04-25T12:38:46.635 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39952.log 2024-04-25T12:38:46.637 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55057.log 2024-04-25T12:38:46.638 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53122.log 2024-04-25T12:38:46.638 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39952.log.gz 2024-04-25T12:38:46.639 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55057.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55057.log.gz 2024-04-25T12:38:46.640 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40271.log 2024-04-25T12:38:46.641 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53122.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53122.log.gz 2024-04-25T12:38:46.642 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42341.log 2024-04-25T12:38:46.642 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40271.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40266.log 2024-04-25T12:38:46.643 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40271.log.gz 2024-04-25T12:38:46.644 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42341.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25697.log 2024-04-25T12:38:46.644 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42341.log.gz 2024-04-25T12:38:46.646 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40266.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40266.log.gz 2024-04-25T12:38:46.646 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54763.log 2024-04-25T12:38:46.646 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.tmp-client.admin.25697.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25697.log.gz 2024-04-25T12:38:46.647 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53959.log 2024-04-25T12:38:46.649 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55495.log 2024-04-25T12:38:46.649 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54763.log.gz 2024-04-25T12:38:46.651 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53959.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53959.log.gz 2024-04-25T12:38:46.651 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5/var/log/ceph/ceph-client.admin.55495.log: --verbose -- /var/log/ceph/ceph-client.admin.54931.log 2024-04-25T12:38:46.651 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55495.log.gz 2024-04-25T12:38:46.652 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49955.log 2024-04-25T12:38:46.654 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38737.log 2024-04-25T12:38:46.654 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54931.log.gz 2024-04-25T12:38:46.655 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49955.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49955.log.gz 2024-04-25T12:38:46.656 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46941.log 2024-04-25T12:38:46.656 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38737.log.gz 2024-04-25T12:38:46.657 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47775.log 2024-04-25T12:38:46.664 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-04-25T12:38:46.664 INFO:teuthology.orchestra.run.smithi123.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.46941.log.gz 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38034.log 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47775.log.gz 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45232.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55411.log 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39174.log 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38034.log.gz 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55411.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55411.log.gz 2024-04-25T12:38:46.665 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45775.log 2024-04-25T12:38:46.666 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39174.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39174.log.gz 2024-04-25T12:38:46.666 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54256.log 2024-04-25T12:38:46.668 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56076.log 2024-04-25T12:38:46.668 INFO:teuthology.orchestra.run.smithi123.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45775.log.gz 2024-04-25T12:38:46.669 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54256.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54256.log.gz 2024-04-25T12:38:46.670 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45100.log 2024-04-25T12:38:46.671 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.56076.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56076.log.gz 2024-04-25T12:38:46.671 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55183.log 2024-04-25T12:38:46.672 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45100.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45100.log.gz 2024-04-25T12:38:46.672 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53337.log 2024-04-25T12:38:46.674 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55992.log 2024-04-25T12:38:46.674 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55183.log.gz 2024-04-25T12:38:46.675 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53337.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53337.log.gz 2024-04-25T12:38:46.676 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33464.log 2024-04-25T12:38:46.676 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55992.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55992.log.gz 2024-04-25T12:38:46.677 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2024-04-25T12:38:46.678 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.33464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53341.log 2024-04-25T12:38:46.678 INFO:teuthology.orchestra.run.smithi123.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33464.log.gz 2024-04-25T12:38:46.681 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40507.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2024-04-25T12:38:46.681 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.53341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39312.log 2024-04-25T12:38:46.681 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53341.log.gz 2024-04-25T12:38:46.682 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54721.log 2024-04-25T12:38:46.684 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54015.log 2024-04-25T12:38:46.684 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39312.log.gz 2024-04-25T12:38:46.685 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54721.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54721.log.gz 2024-04-25T12:38:46.685 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40272.log 2024-04-25T12:38:46.687 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39956.log 2024-04-25T12:38:46.687 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54015.log.gz 2024-04-25T12:38:46.688 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40272.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40272.log.gz 2024-04-25T12:38:46.688 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40866.log 2024-04-25T12:38:46.690 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39956.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log 2024-04-25T12:38:46.690 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39956.log.gz 2024-04-25T12:38:46.691 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38822.log 2024-04-25T12:38:46.691 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40866.log.gz 2024-04-25T12:38:46.692 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49135.log 2024-04-25T12:38:46.692 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2024-04-25T12:38:46.694 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54540.log 2024-04-25T12:38:46.694 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38822.log.gz 2024-04-25T12:38:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.49135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54014.log 2024-04-25T12:38:46.696 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49135.log.gz 2024-04-25T12:38:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41508.log 2024-04-25T12:38:46.697 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54540.log.gz 2024-04-25T12:38:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.54014.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50515.log 2024-04-25T12:38:46.698 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54014.log.gz 2024-04-25T12:38:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37237.log 2024-04-25T12:38:46.699 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41508.log.gz 2024-04-25T12:38:46.701 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50515.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50515.log.gz 2024-04-25T12:38:46.701 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55721.log 2024-04-25T12:38:46.702 INFO:teuthology.orchestra.run.smithi123.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37237.log.gz 2024-04-25T12:38:46.702 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39396.log 2024-04-25T12:38:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.55721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50034.log 2024-04-25T12:38:46.704 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55721.log.gz 2024-04-25T12:38:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39396.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39396.log.gz 2024-04-25T12:38:46.706 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52744.log 2024-04-25T12:38:46.707 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.50034.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50034.log.gz 2024-04-25T12:38:46.707 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41682.log 2024-04-25T12:38:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52744.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52744.log.gz 2024-04-25T12:38:46.710 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41682.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41682.log.gz 2024-04-25T12:38:46.902 INFO:teuthology.orchestra.run.smithi123.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T12:38:47.066 INFO:teuthology.orchestra.run.smithi153.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T12:38:47.158 INFO:teuthology.orchestra.run.smithi153.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T12:38:47.169 INFO:teuthology.orchestra.run.smithi153.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T12:38:47.174 INFO:teuthology.orchestra.run.smithi153.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T12:38:47.176 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-04-25T12:38:47.176 INFO:teuthology.orchestra.run.smithi153.stderr:real 0m0.927s 2024-04-25T12:38:47.176 INFO:teuthology.orchestra.run.smithi153.stderr:user 0m4.002s 2024-04-25T12:38:47.176 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m0.174s 2024-04-25T12:38:47.260 INFO:teuthology.orchestra.run.smithi123.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T12:38:47.377 INFO:teuthology.orchestra.run.smithi123.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T12:38:47.391 INFO:teuthology.orchestra.run.smithi123.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T12:38:47.420 INFO:teuthology.orchestra.run.smithi123.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T12:38:47.423 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T12:38:47.423 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m1.177s 2024-04-25T12:38:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m4.301s 2024-04-25T12:38:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m1.218s 2024-04-25T12:38:47.425 INFO:tasks.ceph:Archiving logs... 2024-04-25T12:38:47.425 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/remote/smithi123/log 2024-04-25T12:38:47.426 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T12:38:48.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-04-25T12:38:48.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-25T12:38:48.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-25T12:38:48.322 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/remote/smithi153/log 2024-04-25T12:38:48.323 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T12:38:48.931 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T12:38:48.941 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T12:38:48.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T12:38:48.949 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T12:38:49.082 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T12:38:49.091 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T12:38:49.092 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:38:49.100 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:38:49.119 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:38:49.119 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:38:49.120 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-04-25T12:38:49.120 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +774us[ +768us] +/- 45ms 2024-04-25T12:38:49.120 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -1138us[-1144us] +/- 54ms 2024-04-25T12:38:49.120 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +179us[ +173us] +/- 61ms 2024-04-25T12:38:49.120 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:38:49.147 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:38:49.156 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:38:49.156 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-25T12:38:49.156 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +757us[ +753us] +/- 45ms 2024-04-25T12:38:49.156 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -1168us[-1172us] +/- 54ms 2024-04-25T12:38:49.156 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +150us[ +145us] +/- 61ms 2024-04-25T12:38:49.156 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:38:49.157 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T12:38:49.167 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T12:38:49.168 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T12:38:49.177 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T12:38:49.386 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T12:38:49.454 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:38:49.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:38:49.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:38:49.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:38:49.577 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T12:38:49.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T12:38:49.597 INFO:teuthology.task.internal:Duration was 763.258495 seconds 2024-04-25T12:38:49.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T12:38:49.606 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T12:38:49.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T12:38:49.614 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T12:38:49.656 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:38:49.749 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:38:49.916 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T12:38:49.916 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T12:38:49.916 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T12:38:49.982 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-04-25T12:38:49.982 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T12:38:50.009 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T12:38:50.009 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T12:38:50.014 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T12:38:50.087 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T12:38:50.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T12:38:50.093 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T12:38:50.269 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T12:38:50.279 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T12:38:50.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T12:38:50.352 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T12:38:50.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T12:38:50.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:38:50.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:38:50.437 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-04-25T12:38:50.488 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-04-25T12:38:50.529 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:38:50.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:38:50.576 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:38:50.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:38:50.591 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T12:38:50.602 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T12:38:50.602 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/remote/smithi123 2024-04-25T12:38:50.603 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T12:38:50.711 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673146/remote/smithi153 2024-04-25T12:38:50.712 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T12:38:50.744 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T12:38:50.744 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T12:38:50.752 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T12:38:50.801 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T12:38:50.812 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T12:38:50.812 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T12:38:50.821 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T12:38:50.822 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T12:38:50.829 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T12:38:50.848 INFO:teuthology.orchestra.run.smithi153.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 12:38 /home/ubuntu/cephtest 2024-04-25T12:38:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 12:38 /home/ubuntu/cephtest 2024-04-25T12:38:50.879 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T12:38:50.890 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T12:38:50.913 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} duration: 763.258495092392 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T12:38:50.913 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T12:38:50.996 INFO:teuthology.run:pass